How to Enable Monit alert in Linux – Part 2

In last post we have seen how to setup monit, now we will see how to enable alert to various mandatory mediums. Without alert any monitoring won’t be much effective and it never fulfill any business or monitoring requirement. Also, alert helps business and system to running 100%. Today will see more in detail how to enable alert to E-Mail, Slack and Microsoft Teams, which are leading tools in current trend for any business.

Again, as we seen Monit is very lightweight tools and easy enable monitoring and even alerts.

This guide assumes you have already installed and configured Monit.

How to Configure Monit to Send Email Alerts

Enabling email alert required mail server details like SMTP hostname, port, username and password (if applicable). Once you have all the details, Open your Monit configuration

#  vim /etc/monit/monitrc

Replace smtp.mailserver.com with your Mail server domain and your-smtp-password with the postmaster’s password.

Set webmaster@foxutech.com to the email address you want to receive Monit email alerts from Mail server

set daemon 60 #check services ever 60 seconds
  set logfile /var/log/monit.log
  set idfile /var/lib/monit/id
  set statefile /var/lib/monit/state

#Event queue
  set eventqueue
      basedir /var/lib/monit/events # set the base directory where events will be stored
      slots 100                     # optionally limit the queue size
#Mail settings
set mail-format {
   from:    Monit Support monit@foxutech
   subject: [CRITICAL] Monit Alert --  $EVENT $SERVICE
   message: $EVENT Service $SERVICE
                 Date:        $DATE
                 Action:      $ACTION
                 Host:        $HOST
                 Description: $DESCRIPTION

            Your faithful employee,
            Monit

set mailserver smtp.mailserver.com port 465
  username postmaster@foxutech.com password "your-smtp-password"
  using TLSV1 with timeout 30 seconds
set alert webmaster@foxutech.com   #email-address which will receive monit alerts

Check the Monit syntax is OK.

#  monit -t

If you perform any change in monit configuration you should reload to get it reflect

#  monit reload

If the changes are not reflected, then restart the Monit service

#  service monit restart

Perhaps you are not a system admin at all; you are a web designer who works with many client sites on different hosts. Wouldn’t it be nice to proactively respond to site outages even before a client call? It is! You can configure Monit to check all your client sites’ statuses and alert you immediately if they are down

check host webserver with address www.example.com   
if failed port 80 protocol http with timeout 30 seconds then alert

Monit can test many protocols, not just HTTP:

check host mail-server with address mail.example.com
if failed port 143 protocol IMAP with timeout 30 seconds then alert
    if failed port 465 protocol SMTP with timeout 30 seconds then alert
    if failed port 22 protocol ssh with timeout 20 seconds then alert

Note that it is possible to change the alert recipient from the globally defined address in the set alert statement to another recipient using the noalert keyword.

check host webserver with address www.example.com
    if failed port 80 protocol http with timeout 30 seconds then alert
    alert someone.else@example.com
    noalert your.email@example.com

Setup Slack

As a first step you need to create a new Incoming WebHook. You can do that by going to https://my.slack.com/services/new/incoming-webhook, select or create a channel, and then click on Add incoming WebHooks Integration. Then you will see a Webhook URL that should be like this: https://hooks.slack.com/services/XXXXXXX/XXXXXXX/XXXXXXXXXXXXXX. You will need that URL in the next step.

The second part requires you to create a Bash script that will post Slack messages when you run it. This is an example of how this file should look like:

URL="https://hooks.slack.com/services/XXXXXXX/XXXXXXX/XXXXXXXXXXXXXX" 
# Slack Webhook URL
PAYLOAD="{
  \"attachments\": [
    {
      \"title\": \"$PROCESS was restarted\",
      \"color\": \"warning\",
      \"mrkdwn_in\": [\"text\"],
      \"fields\": [
        { \"title\": \"Date\", \"value\": \"$MONIT_DATE\", \"short\": true },
        { \"title\": \"Host\", \"value\": \"$MONIT_HOST\", \"short\": true }
      ]
    }
  ]
}"
curl -s -X POST --data-urlencode "payload=$PAYLOAD" $URL

If you want to see how to customize the Slack messages you can take a look at the official documentation.

Before running the script, we should add some permissions (I saved it as slack.sh in you preferable location)

# chmod +x /my/script/path/slack.sh

Now let’s run it to see if it’s working

# /my/script/path/slack.sh

If correct, you will see the message in your Slack channel.

Enable in Monit

You can see if you already have installed Monit by running

Now that you have Monit installed, you can add the configuration for your process on /etc/monit/conf.d

check process apache2 with pidfile /run/apache2/apache2.pid
     start program = "/bin/systemctl start apache2.service" with timeout 15 seconds
     stop program  = "/bin/systemctl stop apache2.service"
    restart program = "/bin/systemctl restart apache2.service"
if changed pid then exec "/bin/bash -c 'PROCESS=Apache /my/script/path/slack.sh'"
if 1 restart within 1 cycle then exec "/bin/bash -c 'PROCESS=Apache /my/script/path/slack.sh'"

This configuration will restart the process in case you stop it manually or if it gets stopped by itself. And when that happens it will call the slack.sh script to publish a message in the Slack channel. You can see a bunch of real-world configuration examples on this link.

Don’t forget to restart Monit after you make changes in the configuration.

# monit reload
Reinitializing monit daemon

You can easily test this by manually killing the process.

# ps aux | grep apache
www-data 26247   801  0 00:09 ?        00:00:00 /usr/sbin/apache2 -k start

# kill -9 26247

The process will be automatically restarted after a couple of seconds and you will receive a message in your Slack channel.

This is just one type of message that you can implement in your Slack team, but you can also do some cooler things with Monit, like sending reports or alerts. You just need to change the configuration files based on that.

Setup Teams

As same as slack, just replace slack.sh with following script follow the same setup and don’t forgot TEAM CHANNEL LINK