Monitoring SSL Certificates with Zabbix

Posted on 01/03/2015 · Posted in Development, Zabbix

To monitor SSL certificates expiry dates with Zabbix, there’s quite a few things you need to do. I’ve tried to make a comprehensive guide about it so you can make the thing yourself and learn how external scripts work. In the end, I’ve attached a Zabbix template for you that can easily be imported into Zabbix. This guide is for Zabbix 2.4, and the template will not work as is on older versions due to changes made on how triggers work. Hopefully you will be able to figure things out with the help of this post so if need be, it can be done on an older version as well.

First off, we need to make a script that will query for the SSL certificate, parse it and then return a value for Zabbix to store.

The External Script

#! /bin/sh
SERVER=$1
TIMEOUT=25
RETVAL=0
TIMESTAMP=`echo | date`
if [ -z "$2" ]
then
PORT=443;
else
PORT=$2;
fi
EXPIRE_DATE=`echo | openssl s_client -connect $SERVER:$PORT 2>/dev/null | openssl x509 -noout -dates 2>/dev/null | grep notAfter | cut -d'=' -f2`
EXPIRE_SECS=`date -d "${EXPIRE_DATE}" +%s`
EXPIRE_TIME=$(( ${EXPIRE_SECS} - `date +%s` ))
if test $EXPIRE_TIME -lt 0
then
RETVAL=0
else
RETVAL=$(( ${EXPIRE_TIME} / 24 / 3600 ))
fi

echo "$TIMESTAMP | $SERVER:$PORT expires in $RETVAL days" >> /etc/zabbix/externalscripts/ssl_check.log
echo ${RETVAL}

Lines 2-5:
Define values for our variables. We define the server to query as SERVER and assign the first input to it. For TIMESTAMP we store the current date so we can log things.
Lines 6-11:
If we have a second user passed variable (the port), store it but otherwise assume default HTTPS port of 443.
Line 12:
This is a little bit more complex. First off we query for the SSL certificate using openssl and pass the output to /dev/null as we don’t want to store it or print out the result. Instead we pipe the result back to openssl to extract the dates from this certificate. As the second openssl command returns two dates (notBefore and notAfter), and we are only interested in the expiry date we use grep to get the line that contains ‘notAfter. The last piped command select everything that’s after the “=” sign, to give us only the date in question. *phew* quite a mouthful wasn’t it? You can copy and paste the command piece by piece to see what the different outputs look like.
Line 13-14:
This calculates how many days there are between the certificates notAfter time and today. %s returns a value that corresponds to the total amount of seconds since epoch (seconds since 1970-01-01 00:00:00 UTC).
Lines 15-20:
We check if the returned value is negative, as the certificate might have already expired! Zabbix can’t handle negative values, and I wasn’t too worried about how long ago the certificate expired as long as we know it has or will expire. If it’s above zero, convert the seconds to days.
Line: 22:
Write the output to a log file, to keep track of what certificates have been checked. This can be commented out if found unnecessary.
line 23:
Return the value, either 0 (which means the query either failed for the certificate has already expired) or the amount of days left until the certificate expires.

 

You need to save the script as a .sh file, and make it executable with chmod u+x (at minimum). In addition, if the script file only has executable permissions for the user, then it needs to be owned by the user that Zabbix server is run with, which is by default zabbix. In the example below, i have changed the ownership of the script file to user:zabbix, group:zabbix with chown zabbix:zabbix ssl_check.sh. And I’ve give both the group, and the user read, write and execute permissions to the file. Other users have no permissions to the file.

total 12K
drwxr-xr-x. 2 zabbix zabbix 45 Feb 17 08:57 .
drwxr-xr-x. 7 root root 4.0K Feb 17 08:47 ..
-rw-rw-r--. 1 zabbix zabbix 1.9K Feb 28 08:44 ssl_check.log
-rwxrwx---. 1 zabbix zabbix 591 Feb 17 08:48 ssl_check.sh

 

Zabbix Server configuration

If you haven’t used external scripts before on your Zabbix Server, then you need to make changes to your Zabbix Server configuration file. This can be found by default in /etc/zabbix/zabbix_server.conf (CentOS) and in this file, you need to locate the following line(s):

### Option: ExternalScripts
#       Full path to location of external scripts.
#       Default depends on compilation options.
#
# Mandatory: no
# Default:
# ExternalScripts=${datadir}/zabbix/externalscripts
ExternalScripts=/etc/zabbix/externalscripts

I’ve changed ExternalScripts to /etc/zabbix/externalscripts (which i created) in the same folder as where the Zabbix server and Zabbix agent configuration files reside. Once you’ve changed this, Zabbix server needs to be restarted.

The Template

To make things easy, I created a template which can be applied on any host within Zabbix. This cuts down the amount of setting up required and can easily be applied to any webhost that’s being monitored by zabbix. Go to Configuration, Templates and press “Create Template“. I named mine “Template SSL Check“.

To make things easy to use, create two macros:

{$SSL_HOST} => {HOSTNAME}
{$SSL_PORT} => 443

Select Items, and press “Create Item

Give it a name, set the type to “External Check” which tells Zabbix to look in the External Scripts folder we defined earlier. The key must contain the name of the script file, in this case ssl_check.sh and the variables we want to pass to it being the host and port.

ssl_check.sh[{$SSL_HOST},{$SSL_PORT}]

The type of information should be “Numeric” and data type as “Decimal”. Update interval of 86400 seconds is enough, which equals to once a day. Store the value “As is” and show value “As is”. Lastly define a new application, as there isn’t one in this new template. I’ve named the new application as “Certificate Check“.

Hit save and now we have a new Item in this template that interacts with our ssl_check.sh script. At this point, you could try things out to see if any values are returned from the script. But without triggers, no alerts would be passed to you which is the ultimate goal.

The Triggers

Last thing to do is to create the triggers, which is pretty straight forward as we only have one item. You can either use the expression builder to create the trigger or write it yourself. Here’s one example of the triggers that can be created:
 

{Template SSL Check:ssl_check.sh[{$SSL_HOST},{$SSL_PORT}].last()}<64 and {Template SSL Check:ssl_check.sh[{$SSL_HOST},{$SSL_PORT}].last()}>=32

NameOfTemplate:Item.last() less than 64 days and NameOfTemplate:Item.last() bigger or equal to 32 days

 
You can change the days as you please, to make the alerts fit your need. In previous versions of Zabbix, at least in 2.0 it used to “&” sign instead of “and” to combine two trigger expressions together. Other than that, the trigger should work on the latest versions of Zabbix. You can download the trigger template below and import it to Zabbix version 2.4.