Server monitoring is the process of monitoring a server’s system resources like CPU Usage, Memory Consumption, I/O, Network, Disk Usage, Process etc. Server Monitoring also helps in capacity planning by understanding the server’s system resource usage as well as capacity to handle requests.
Server monitoring software helps in automating the process of server monitoring. Monitoring server performance also helps in identifying other performance related issues like resource utilization, app downtime and response time.
Web server monitoring adds additional indicators such as those around requests, connections and average time taken to process a request.
Top Web Servers for your Business
Statistics show us that well over 80% of web applications and websites are powered by open source web servers. Listed below are the top solutions used by most.
- Apache HTTPD
- Lighttpd
- Microsoft IIS
- Nginx Web Server
- Node.JS
- Apache Tomcat
Any of the above can fulfill your need for a Web Server. But when you look at what most of the internet runs on it is Apache HTTPD and Nginx taking a commanding share of marketing going over 60%.
Why is Web Server Monitoring Important
Web servers are a critical part of the infrastructure of most companies. These critical pieces of software are not only used for internal operations and services but are also important for providing service to their customers.
Almost all businesses now have a presence on the internet. This raises the need for them to host their websites either on their own private servers or with external hosting service providers. Independent of which way they go the need to monitor the availability of websites and services becomes a critical need for them, as any downtime results in not only loss of potential revenue but also in credibility with their customers who rely on services provided to them.
“With so much competition in the market it is the customer who decide which business they want to give money to. They expect high performance and little to no downtime.”
The Guru
Web servers are thus a critical component of any company’s internal and external infrastructure needs. Ensuring that they are up and running means the business is able to serve the purpose they are created for, that is, serving their customers.
Over time by regularly monitoring the resources consumed by web servers and patterns of traffic and key performance indicators one can build a profile of their application and its components.
This includes not only the capability of resource and infrastructure planning but also helps identify usage patterns such as busy times of the year, month and week down to even the hour. This helps with ensuring that the right resources are available when needed and enough server processing is brought online during busy periods such as sales and shopping events.
Once you build performance and use profile for your web applications you can make better use of your resources while providing customers with the high-quality experience that they expect.
Key Web Server Metrics to Track
With webserver there are must have metrics to track such as those related to system variables like memory, cpu and disk usage. But there are also Key Performance Indicators (KPI) for applications being run on these servers. Let’s take a look at each category of KPIs.
System Metrics
CPU
High CPU usage indicates that there’s not enough CPU power to service all the requests coming in. But this may not be only because of poor Apache performance. Services running on the same box, such as an API server, and especially a database service, also require CPU and will compete for other resources, such as memory and network.
If you are experiencing high CPU usage, make sure to monitor the list of processes that are using a lot of CPU. If you see the Apache HTTP Server listed at the top of the list of processes, you might want to consider adding more resources to the box or moving other services (if there are any) to another server.
Disk
One major role of a web server is to serve content to its consumers thus storage capacity along with speed of access to the datais a critical factor to consider.
Memory
Web Servers can consume a lot of memory especially as you add functionality either though modules, built-in or custom. The memory usage can also go up significantly depending on number of users accessing the server concurrently.
When a web server get close to running out of RAM it will start disk swapping. Once a server starts using swap space the performance and response times significantly deteriorate.
Network
How a server communicates to other systems is through its networking interface. Even when using the fastest CPU, unlimited RAM and SSD for storage a slow or constrained network connection will impair the ability of the server to serve a large number of users.
Performance Metrics
Bytes Per Second
The number of bytes per second is the number of bytes or the amount of data that is being sent to and from your Apache web server every second on average. This indicates the amount of data transfer that your infrastructure is handling.
Requests Per Second
This metric represents the average number of requests per second, calculated over the entire uptime of the server. It can be a bit misleading when you’re trying to look for spikes over a period of time.
A sudden and unexpected rise in the number of requests could point to a denial-of-service (DOS) attack. It could also be that for some reason a large number of people are hitting your Web Server. In the former case you may need to look at security solutions for the protection of your resources and in the latter case you may need to look at (dynamic) scaling of your infrastructure.
Request Processing Time
Request processing time is the amount of time a Web Server requires to process one request. This includes the time it takes for all of the underlying services to process the request, as well as the web server itself.
Uptime
Uptime is the span of time since the Web Server was started and the current time. Frequent restarts may point to either poor management of the resource or an issue with the server itself requiring frequent restarts.
Application Monitoring Through Web Server Logs
Web Server logs provide extremely important information for you to track. The information available in the logs is completely different from what is provided through either Performance or System KPIs.
The information available in the logs provide details on the health of applications running on them. Logs also collect information on the usage patterns of your consumers as they may help provide Geo-location of visitors and what they are doing on your website based on their activity.
All activity of a visitor is available as a single line in the log file providing information such as their IP, time they accessed your server as well as the resource (URL) they visited.
Thus using the logs you can create an activity profile of users visiting your website. In addition, you further use threat analysis tools to secure your Web Server and Applications running on it.
Check out my post on Apache Web Server logging setup with basics of analysis and monitoring.
Web Server Monitoring Tools
Website monitoring is a catch-all term that describes any online service that checks a website for its availability, function, speed, and performance. Most often, these monitoring services are set up then run independently, only alerting you when a problem arises.
Server monitoring systems come in two main types, on premise and in the cloud (through SaaS).
On Premise
On premise system are built around software that is installed on your own, in-house infrastructure. This can be native modules bundled with your Web Server, Open Source software or commercial software through a third-party vendor. The cost in time and resources is usually higher and one may need in house expertise to maintain the software.
Cloud/Saas
SaaS solutions are monitoring systems that are installed and managed entirely via the web. There may be a need to add some software bits to your server on Web Application but mostly it is trivial work when compared to on premise solution. Options to customize a SaaS product may be limited. Although ease of use of such software and reduced management of additional infrastructure plays in its favor.
Free Open Source Web Server Monitoring Solutions
Listed above are some of the open source web server monitoring solution providers. In some cases the services offer only Uptime monitoring where as others provide additional services such as system and network monitoring.
Cabot is a self-hosted monitoring solution for your website and infrastructure. You can use it to monitor graphite metrics, Jenkins jobs, and web endpoints. Cabot also offers basic alerting features. You can use trigger alerts via phone, SMS, Hipchat, or email. Bear in mind that in order to use phone calls, you need to either opt for another solution. Cabot offers well-written documentation that will help you to start monitoring within minutes. Cabot is available on Github under MIT License.
Checkmk Raw has been an active open source project since 2007. It offers more than 1800 officially supported plug-ins, and both agent-based and agentless monitoring. You can also monitor your servers, network, applications, containers, Databases, Storage, Logs, Kubernetes, and more. Checkmk Raw also offers smart alerting, unlimited user accounts, elaborate user management, customizable GUI, customizable and real-time dashboards. You can benefit from its basic reporting features and automation. Checkmk Raw works thanks to Nagios monitoring kernel and is scalable for more than 100hosts. The shortest measurement interval available is one minute.
Nagios Core offers a complete URL monitoring solution for HTTP and HTTPS servers, protocols, and URL transactions. It offers complete IT infrastructure monitoring, visibility via a web interface, alerts delivered through email and SMS, event handlers for problem remediation, proactive planning features, basic reporting, user management, and API integrations. Advanced features, such as Website monitoring wizard, are available only for Nagios XI.
OpenNMS is an application performance management software designed to help businesses in the healthcare, retail, IT, energy, finance, education, government and other sectors monitor the performance of remote and local networks. The platform enables administrators to analyze network traffic, discover the topology and identify faults across applications.
Managers can receive notifications regarding asynchronous events, view resource graphs and track node status on a unified interface. OpenNMS allows teams to analyze the root cause and impact of hierarchy across business services. Additionally, operators can create custom graphs and group similar network alarms using AI-enabled tools.
It offers an application programming interface (API), which lets businesses integrate the system with several third-party applications including, Jira, Drools, Apache ActiveMQ, Elasticsearch and Apache Kafka. OpenNMS is available for free and on a subscription basis and support is extended via phone, email, knowledge base, chat, forum and an online contact form.
Prometheus is an open-source monitoring solution primarily fixated on data gathering and analysis based on time-series data. It enables users to set up monitoring capabilities by utilizing the in-built toolset. It is an ideal monitoring setup for containerized environments like kubernetes and the best open-source server monitoring tool.
It is able to assemble several metrics on servers, kubernetes, and various devices using SNMP pings and inspect network bandwidth usage from the device point of view, along with the other functions. The PromQL system analyzes data and permits the program to produce plots, tables, and other graphics on the systems it monitors.
The alert manager is another component of Prometheus. It handles alerting for all alerting rules configured in Prometheus.
Prometheus can collect system metrics, application metrics, and metrics from modern containerized applications. Also, it has very good integration with tools like Grafana for visualizing metrics. Grafana is one of the best open-source monitoring dashboards.
Sensu is Sumo Logic’s Website Monitoring Project available in both Commercial and Open-Source versions. Sensu OSS version is distributed under MIT-licensed code and enables you to monitor Servers, Containers, Real-Time Inventory, perform Health Checks and monitor custom metrics. You can also benefit from its automated remediation features and more than 200 community plugins. OSS version does not offer a lot of dashboarding, but you can work with a Grafana UI Datasource. You can also use custom plugins/script language support for the most popular language platforms. You also have basic user management available.
Upptime is a GitHub-powered open-source uptime monitor and status pages manager. Upptime uses GitHub actions, which allows a minimum interval of 5 minutes, which explains its monitoring frequency. Apart from uptime, it also measures the response time and commits it to git history. Using all this data, Upptime can generate graphs and long-term performance overviews of your website and publish them on a status page. In case of any downtime, Upptime automatically opens a new issue in your GitHub repo. You can edit this repo and add additional information about the outage, including the root-cause analysis. You can also set up notifications using integrations with services like Slack, or Telegram. Status websites are powered by GitHub Pages. It shows your live status, incident history, and response time graphs. Your Status Page shows data in real-time since it uses GitHub API to fetch data. Status pages offer customizability, such as custom logos, text, and more. Upptime is available on GitHub under the MIT License. It has more than 9.2k stars.
Uptime Kuma is a self-hosted monitoring tool with a UI that feels a lot like Uptime Robot. Using Uptime Kuma, you can monitor uptime for HTTP(s) and HTTP(s) keywords, TCP, Ping, DNS Record, Push, and Steam Game Server. It offers integration with more than 70 notification services, including Telegram, Discord, Slack, Email, and more. Uptime Kuma does not have a website. However, everything you need can be found on Github. Its developer LouisLam still works on the project and updates it. You can also try a limited, 10-minutes long demo hosted on a Japanese server Uptime Kuma is available on GitHub, it has more than 12 thousand stars, and it has an MIT License.
Zabbix is an enterprise-ready open-source monitoring solution allowing you to monitor Networks, Servers, Cloud, OS, Log Files, Databases, Apps, Websites, and much more. Using Zabbix, you can monitor average download speed per second, error messages, response times, response code, and other features of your web using custom web scenarios. Zabbix offers a widget-based multi-page web UI with dashboards. You can create and customize widgets using drag and drop functions, create custom refresh rates, set them as either public or private, visualize data using graphs, and much more. You can find everything you need on Zabbix’s website. Zabbix is one of the tools that require a lot of learning. They offer multiple training programs and certifications ranging from $50 all the way to thousands of dollars.
I have used many of these tools over the years, either in production systems or during evaluation and analysis. Check out my article on Top 3 Open Source Web Server Monitoring Tools.
Other Web Server Monitoring Solutions
In this section you will find companies providing both paid and free plans to monitor your web servers as well as the applications running on them.
- Better Uptime
- Cacti
- Checkmk RAW
- DataDog
- dotcom-monitor
- LibreNMS
- Montastic
- Netdata
- New Relic
- NtoPng
- Prometheus
- Observium
- Pandora FMS
- Riemann
- SemaText
- Site24x7
- SolarWinds Pingdom
- SolarWinds Web Performance Monitor
- StatusCake
- Updown.io
- UptimeRobot
- Uptrends
With such a huge list of software solutions to pick from it becomes difficult to evaluate what is the best monitoring solution. To save the readers some time I have tested these services and have narrowed it down to a manageable list of Top 5 Web Server Monitoring Solutions.
In the article you will find a thorough comparison of features and ratings on ease of use and setup. Hopefully, you will find it useful as a lot of time was spent testing all of these pieces of software.