Apache Performance and Resource Monitoring Using Mod_Status

Monitoring services for uptime and performance is a crucial part of any organization’s day-to-day operations. Essentially since its inception, Apache Web Server (Apache HTTPD) has been among the three most popular web servers on the internet. It has been providing critical hosting services to hundreds of millions of web sites commanding a share of over 31% of all installations.

Apache HTTPD is not only used for serving websites and static pages but is also used as a Caching Server but also as a Load Balancer distributing requests among multiple other web services.

At its hears Apache HTTPD is a core system whose functionality has been extended through pluggable modules which can easily be enabled through configuration file settings.

Monitoring Apache Natively Using MOD_STATUS

Apache provides a module MOD_STATUS that provides the ability to the server administrator to find out how well their serve is performing. This module provides details on the following variables:

• The number of workers serving requests
• The number of idle workers
• The status of each worker: Number of requests that worker has performed and the total number of bytes served by the worker.
• Total number of incoming requests.
• Count of data severed with “Bytes count served”.
• Server lifecycle info: The time the server was started/restarted and the time it has been running for
• Averages
  • number of requests/second
  • number of bytes served/second
  • average number of bytes/request
• The current percentage CPU used by each worker and in total by all workers combined.
• The current hosts and requests being processed.

Note: MOD_STATUS provides monitoring statistics on the web server itself. It does not provide any info on web applications running on the server.

Enable MOD_STATUS Configuration

In this section I will go over details of how to setup Apache HTTPD with MOD_STATUS, enabling collection of important server statistics. Since Apache HTTPD runs on multiple platforms, I will details steps to enable configuration on Linux and Windows. Listed below if my Test Environment setup.

Step 1: Edit httpd.conf File

Find the following entry in the config file

#LoadModule status_module modules/mod_status.so

Remove the leading # (which denotes that the module MOD_STATUS is disabled) and save the file.

Step 2: Edit httpd.conf or Appropriate Virtual Host Config File

If you server is setup serving multiple websites then you need to edit the appropriate config file.

For example if you are serving the domain example.com, then you need to look in the folder /etc/apache2/sites-enabled/example.conf and open it in your favorite editor.

If you are only serving one website in the default config then you will find the host settings in /etc/apache2/sites-enable/default.

Add the following to your config file if using default Apache Install. Just search for <Location> tag and change to the following:

<Location /server-status>
    SetHandler server-status
    Order allow,deny
    Deny from all
    Allow from all
</Location>

There are 2 things to note in this code:

• Location: This is the URL where you will find the status information for your Apache Web Server (HTTPD). If your server is running our sample domain example.com, then you will find the status page on https://example.com/server-status.
• SetHandler: This config variable tells Apache that when the user view the Location, then load the handler server-status (part of MOD_STATUS which we enable earlier in httpd.conf file).

Using the configuration below you can limit access to the /server-status URL only from localhost.

<Location /server-status>
    SetHandler server-status
    Allow from localhost
</Location>

You can also restrict access to authenticated users with the following configuration settings.

<Location /server-status>
    SetHandler server-status
    AuthUserFile /passwordlocation
    AuthType Basic
    AuthName <USER_NAME>
    Require user <USER_NAME>
</Location>

You can check all options available for <Location/> node in the Apache documentation.

If you are using virtual host then you need to add the <Location> code shown above under the <VirtualHost> tag as shown below:

<VirtualHost *:80>

    ServerAdmin [email protected]
    DocumentRoot /var/www/html/example # should be your actual path where the website files are kept
    ServerName example.com
    ErrorLog logs/example.com-error_log
    CustomLog logs/example.com-access_log common

    <Location /server-status>
       SetHandler server-status
       Order allow,deny
       Deny from all
       Allow from example.com 
    </Location>

</VirtualHost>

Regarding ExtendedStatus Setting

In the past certain monitoring values are only enabled if the ExtendedStatus was set to On in the httpd.conf file. Since version 2.3.6 of the Apache HTTPD, these extended settings have been set to On by default. If you are using a version previous to 2.3.6 then you will want to add the following to you httpd.conf file.

ExtendedStatus On

Step 3: Test and Restart Apache HTTPD

You can test the config file by type the following command in the shell.

user@host ~# sudo apachectl configtest

If there are no errors you can restart the Apache HTTPD.

user@host ~# sudo service apache2 restart

Viewing and Understanding Active MOD_STATUS web page

Once your server restarts, it will start collecting statistics for monitoring and performance of the server. Type the following URL in the browser to view available statistics.

http://example.com/server-status

Your screen will look similar to the screenshot shown below.

Note: You can use the URL http://example.com/server-status?refresh=N to refresh the page every N seconds

There is also an option to get the statistics in a machine readable format which allows easier integration with upstream monitoring tools. Use the URL http://your.server.name/server-status?auto to access statistics in this format. Below is the sample output.

Understanding Variables Shown on the Auto Web Page

Server Information Section

Hostname

Shows the name of the host/domain.

ServerVersion

This provides information on the Apache server version, including platform. In the image above you can see that Apache version 2.4.52 is running on 64bit version of Windows and has both OpenSSL and PHP enabled.

ServerMPM

Provides detail of the Multi-Processing Module(MPM) that is enabled on the server. Apache has the following three MPM modules available.

worker

The worker MPM causes Apache to fork into several processes so that a single crash does not kill the entire Apache process. Each forked process creates numerous threads and a listener thread that listens for connections. This enables the system to serve multiple requests concurrently.

event

Event MPM serves a single HTTP request across multiple threads. However, the event MPM uses a dedication system and dedicates a thread to handle all live connections. The system allocates a request to the child threads only when it receives a request. This allows the threads instant availability after the request finishes.

This MPM model is great We recommend this MPM for users that only make occasional concurrent requests, or when a user requires a long keep-alive timeout.

prefork

The prefork MPM causes Apache to fork into additional processes before the system receives requests. When the server receives requests, it responds quickly to each request because an Apache process already exists to handle it. If a problem occurs with an individual process, the system can kill the process with no effect on the other Apache processes.

This module is not multi-threaded therefore each process handles multiple requests synchronously. Consumes many resources and not the best option.

Suggestion: Use event MPM for generally the best performance.

Build Information

Provides information on the build. In my case, I am using Apache Lounge for the build.

CurrentTime

Current date and time on the machine running the Apache HTTPD Web Server. May be different from your local time depending on the machine where the server is running.

RestartTime

When the server was started/restarted.

ServerUptimeSeconds

Time in seconds since the server was started/restarted.

ServerUptime

Basically, this line converts the value from Line 9 to a more human readable string.

Server Information Section

Load1

Displays 1-minute load average.

Load5

Displays 5-minute load average.

Load15

Displays 15-minute load average.

Total Accesses

The total number of direct URL accesses on the server is monitored using this metric.

Total kBytes

Total kilo-bytes of data that has been sent over the network.

Total Duration

This shows the total time taken to process incoming requests. Since multiple threads may be serving incoming client requests, this number should be higher than the ServerUptime metric.

Uptime

The metric Uptime gets the total amount of time the server has been up and running. This should match the ServerUptimeSeconds.

ReqPerSec

Total number of client requests served across all threads/process of Apache HTTPD.

BytesPerSec

Total number of bytes sent over the network in response to incoming client requests across all threads/process of Apache HTTPD.

BytesPerRequest

Average bytes sent in response to all incoming requests. This is an actual average and not a mean.

BytesPerRequest

Average bytes sent in response to all incoming requests. This is an average and not a mean.

DurationPerRequest

Average time in seconds taken to process a single incoming request. This is an average and not a mean.

BusyWorkers

Show the total number of works actively serving incoming requests. The higher the number shows that your server is actively processing requests. The higher this number goes lower will be the IdleWorkers count.

IdleWorkers

Total number of workers available in the pool to serve incoming requests. If you have very few or no idle processes, it means your server is using up all the server resources and is likely to queue up incoming requests.

This results in a poor user experience for the visitors as the time taken for them to view results is going up due to a lack of resources available to process requests. You should evaluate your configuration settings and may look into increasing resources serving clients.

TLSSessionCacheStatus

Many of the metrics in the TLS section can be configured in the httpd-ssl.conf file. These are available only if you have MOD_SSL enabled for Apache HTTPD.

CacheType

I am using SHMCB. The other options available are DBM, DC, MEMCACHE, REDIS and SHMCB.

CacheSharedMemory

This metric shows the cache memory available to TLSSession. If you find that CacheIndexUsage is getting over 80% then you will want to increase the shared memory cache size.

There are other metrics available under the Caching category but I focus mostly on the CacheSharedMemory and CacheUsage.

Using MOD_STATUS to Troubleshoot Server Issues

To find out which requests or clients are causing unwanted behavior on the server you can use the /server-status page as a starting place for troubleshooting a situation where your server is consuming available resources such as CPU or memory.

Note: If using a version prior to 2.3.6, make you have ExtendedStatus set on. This way you can see the full request and client information for each child or thread.

Now using a command line tool such as top (or similar process viewing utility) identify processes hogging resources. Refresh /server-status page and you will be able to identify what request is being served by that process. This will allow you to isolate the client causing the issues. Since web server requests are transient you may need to refresh the status page a few time before you can identify the exact client causing the issue.

This process should give you some idea of what client, or what type of requests, are primarily responsible for your load problems. Often you will identify a particular web application that is misbehaving or a particular client that is attacking your site.

Conclusion

In this post, I went over details of how to use Apache Mod_Status to monitor internal Apache key performance indicators. I also showed how you can also get operating system resource consumption statistics as well.

There is a lot more to running an optimized installation of the Apache web server and I will be bringing you more information on that in the future.

For other guides and posts on Apache check out our main Apache web server page. New updates will also be posted on this page.