04 Jan

Observium pollers vertical scaling

Observium in containers

There are several free monitoring tools available for commercial or non-commercial use. I decided to use Observium to monitor my home and remote labs. I had no specific requirements except that it had to have an option to run in Docker containers on RaspberryPi (ARMv6/8 CPUs). The downside of all monitoring solutions are the resource requirements. There are three resources that matter – storage, memory, and CPUs. Vertical scaling of Observium pollers and other processes is one of the solutions if your hardware resources are limited.

The vertical scaling means adding more workers doing the same process in parallel, but tasks assigned to each process should not overlap. In my case, I wanted to spread the polling across my RaspberryPi cluster. Polling process can consume the CPU and make RaspberriPi unresponsible. In worst scenario, if you have many devices or you are poling lots of data from your devices the polling processes may not finish their work within 5 minutes (this is how often devices have to be queried) so you will miss some data. You may of course tune number of threads the polling process start but any platform has it resources limitations you cannot bypass.  

In this article I will present my solution based on Docker Swarm cluster. It has some limitations and downsides but may work for many people, not only on RaspberyPi.

Read More
13 Apr

Microsoft Operations Management Suite – powerful log analyzer in Azure (in 10 minutes for free)

Collecting and processing logs from all systems and network devices can be a nightmare for any systems admin. Searching through them and performing security audits can be a nightmare for security team if collector engine is not powerful enough to process queries in efficient time. Microsoft Operations Management Suite is interesting solution to answer both those problems and add much more analysis giving administrators visibility and control across on-premise and cloud installations.

Microsoft Operations Management Suite runs in Azure which means it’s extremely fast in processing the data. Millions of records are not problem for OMS so we can get Insights and Analytics of what is happening on our servers or workstations, detect and respond to threads or apply proper protection or even put in place some automation in controlling. It’s quick to setup and for many users it can be for free!

Read More