Monitoring performance with the TICK Stack
The TICK Stack was developed by InfluxData (InfluxDB) and is made of a series of integrated components that allow you to easily process time-series data generated by different services through time. TICK is an acronym that is composed of the first letters of each main product of the monitoring suite. T is for Telegraf, which collects the information we wish to obtain on our production server. I is for InfluxDB, which is a time-series database that contains the information collected by Telegraf or by any other application which is configured to do so. C is for Chronograf, a graph tool that allows us to easily understand the collected data. Finally, K is for Kapacitor, an alert automation tool.
Monitoring infrastructure performance is not only important to determine if applications and scripts are running as expected, but also allows for development of more advanced algorithms such as failure prediction and unexpected behavior pattern recognition, thus making it possible to automate many aspects of performance monitoring.
Of course, there are many other great performance monitoring tools, like Prometheus and Graphite, but we decided to use the TICK stack instead, because we are more interested in doing event logging than doing pure metrics. For more information on what the TICK stack is, how it works internally and what it is used for, please read this very informative article by Gianluca Arbezzano published on the Codeship website: https://blog.codeship.com/infrastructure-monitoring-with-tick-stack/.
Now, in order to see how useful our Blackfire.io supported analysis was and how much more efficient our code has become, we will run the two scripts again but, this time, while using copies of the official TICK Docker images so that we may monitor any improvement in the Web server's overall performance once the optimized PHP script is deployed on it. We will also be replacing Chronograf with Grafana, a highly customizable graph tool, and we will not be setting up Kapacitor, since configuring alerts is slightly beyond the scope of our current objectives.
Let's begin by activating mod_status on our Apache server. From our Linux for PHP's CLI, enter the following commands:
# sed -i 's/#Include \/etc\/httpd\/extra\/httpd-info.conf/Include \/etc\/httpd\/extra\/httpd-info.conf/' /etc/httpd/httpd.conf # sed -i 's/Require ip 127/Require ip 172/' /etc/httpd/extra/httpd-info.conf # /etc/init.d/httpd restart
Once you have done this, you should be able to see the server's status report by browsing with Chrome to the following URL: http://localhost:8181/server-status?auto.
The next step is to launch the TICK suite. Please open two new Terminal windows in order to do so.
In the first Terminal window, type this command:
# docker run -d --name influxdb -p 8086:8086 andrewscaya/influxdb
Then, in the second newly opened Terminal window, get the IP addresses of our two containers by issuing this command:
# docker network inspect bridge
Here is the result of this command on my computer:
The IP addresses of the two containers
Please retain these two addresses as they will be needed to configure Telegraf and Grafana.
We will now generate a sample configuration file for Telegraf with a simple command (this step is optional, as a sample file is already included in this book's repository).
Firstly, change the directory to our project's working directory (Git repository) and enter the following command:
# docker run --rm andrewscaya/telegraf -sample-config > telegraf.conf
Secondly, open the new file with your favorite editor and uncomment the following lines in the inputs.apache section. Do not forget to enter our Linux for PHP container's IP address on the urls line:
Configuring Telegraf in order to have it monitor the Apache server running in the other container
In the Terminal window, we can now launch Telegraf with this command (please make sure that you are in our project's working directory):
#
In the second newly spawned Terminal window, launch Grafana with the following command:
#
With Chrome, browse to http://localhost:3000/login. You will see Grafana's login page. Please authenticate with the User admin using the Password admin:
The Grafana login page is displayed
Then, add a new data source:
Connecting Grafana to a data source
Please choose a name for the InfluxDB data source. Select InfluxDB as the type. Enter the URL for the InfluxDB container instance, which includes the IP address that you obtained in one of our previous steps, followed by the default port number for InfluxDB, which is 8086. You can select direct access. The database's name is telegraf and the database's user and password are root:
Configuring Grafana's data source
Finally, click the Add button:
Adding the data source
Now that the data source has been added, let's add a couple of dashboards that we will import from the Grafana website. Start by clicking Import under the Dashboards menu entry:
Click on the import menu item to begin importing dashboards
The two dashboards that we will add are the following:
- Telegraf Host Metrics (https://grafana.com/dashboards/1443):
The homepage of the Telegraf Host Metrics dashboard
- Apache Overview (https://grafana.com/dashboards/331):
The homepage of the Apache Overview dashboard
On the import screen, simply enter the number of the dashboard and click Load:
Loading the Telegraf Host Metrics dashboard
Then, confirm the name of the new dashboard and select our Local InfluxDB connection:
Connecting the Telegraf Host Metrics dashboard to the InfluxDB data source
You should now see the new dashboard:
The Telegraf Host Metrics dashboard is displayed
We will now repeat the final two steps in order to import the Apache Overview dashboard. After clicking on the Import button under the Dashboards menu entry, enter the dashboard's identifier (331) and click the Load button:
Loading the Apache Overview dashboard
Then, confirm the name and select our Local InfluxDB data source:
Connecting the Apache Overview dashboard to the InfluxDB data source
You should now see the second dashboard in the browser:
The Apache Overview dashboard is displayed
All TICK suite dashboards allow for more advanced configuration and customization of graphs. It would therefore be possible to collect a custom set of time-series data points through the execution of custom cron scripts for example, and then configure the dashboards to display this data as you see fit.
In the case of our current example, the TICK suite is now installed and configured. Thus, we can begin testing and monitoring the PHP script that was optimized using Blackfire.io in the first part of this chapter in order to measure the changes in its performance. We will start by deploying, benchmarking and monitoring the old version. On the Linux for PHP's CLI, enter the following command in order to benchmark the old version of the script:
# siege -b -c 3000 -r 100 localhost/chap2pre.php
The benchmark test should yield something similar to the following result:
The results of the performance benchmark of the original script are displayed
Then, after waiting approximately ten minutes, start benchmarking the new version of the script by entering the following command:
# siege -b -c 3000 -r 100 localhost/chap2post.php
Here is the result of this latest benchmark on my computer:
The results of the performance benchmark of the optimized script are displayed
The results already reveal to us a considerable improvement in performance. Indeed, the new script allows for more than three times the number of transactions per second and more than three times fewer failed transactions.
Now, let's have a look at what data our TICK Stack collected concerning the performance of these two versions of our PHP script:
The gain in performance is clearly seen in the monitoring graphs
The graphs in our Grafana dashboard clearly show a performance boost of the same order of magnitude as the benchmark results themselves. The benchmark test launched after 08:00 against the new version of our script clearly generated two times less load on the server, caused two times less input (I/O) and was more than three times faster in general than the old version that was benchmarked previously around 7:40. Therefore, our Blackfire.io optimizations have, without a doubt, made the new version of our PHP script more efficient.