In last post we have seen how to setup monit, now we will see how to enable alert to various mandatory mediums. Without alert any monitoring won’t be much effective and it never fulfill any business or monitoring requirement. Also, alert helps business and system to running 100%. Today will see more in detail how to enable alert to E-Mail, Slack and Microsoft Teams, which are leading tools in current trend for any business.
Again, as we seen Monit is very lightweight tools and easy enable monitoring and even alerts.
This guide assumes you have already installed and configured Monit.
How to Configure Monit to Send Email Alerts
Enabling email alert required mail server details like SMTP hostname, port, username and password (if applicable). Once you have all the details, Open your Monit configuration
# vim /etc/monit/monitrc
Replace smtp.mailserver.com with your Mail server domain and your-smtp-password with the postmaster’s password.
Set webmaster@foxutech.com to the email address you want to receive Monit email alerts from Mail server
set daemon 60 #check services ever 60 seconds
set logfile /var/log/monit.log
set idfile /var/lib/monit/id
set statefile /var/lib/monit/state
#Event queue
set eventqueue
basedir /var/lib/monit/events # set the base directory where events will be stored
slots 100 # optionally limit the queue size
#Mail settings
set mail-format {
from: Monit Support monit@foxutech
subject: [CRITICAL] Monit Alert -- $EVENT $SERVICE
message: $EVENT Service $SERVICE
Date: $DATE
Action: $ACTION
Host: $HOST
Description: $DESCRIPTION
Your faithful employee,
Monit
set mailserver smtp.mailserver.com port 465
username postmaster@foxutech.com password "your-smtp-password"
using TLSV1 with timeout 30 seconds
set alert webmaster@foxutech.com #email-address which will receive monit alerts
Check the Monit syntax is OK.
# monit -t
If you perform any change in monit configuration you should reload to get it reflect
# monit reload
If the changes are not reflected, then restart the Monit service
# service monit restart
Perhaps you are not a system admin at all; you are a web designer who works with many client sites on different hosts. Wouldn’t it be nice to proactively respond to site outages even before a client call? It is! You can configure Monit to check all your client sites’ statuses and alert you immediately if they are down
check host webserver with address www.example.com
if failed port 80 protocol http with timeout 30 seconds then alert
Monit can test many protocols, not just HTTP:
check host mail-server with address mail.example.com
if failed port 143 protocol IMAP with timeout 30 seconds then alert
if failed port 465 protocol SMTP with timeout 30 seconds then alert
if failed port 22 protocol ssh with timeout 20 seconds then alertNote that it is possible to change the alert recipient from the globally
defined address in the set alert statement to another recipient using the noalert
keyword.
check host webserver with address www.example.com
if failed port 80 protocol http with timeout 30 seconds then alert
alert someone.else@example.com
noalert your.email@example.comSetup Slack
As a first step you need to create a new Incoming WebHook. You can do
that by going to https://my.slack.com/services/new/incoming-webhook,
select or create a channel, and then click on Add incoming WebHooks Integration. Then you will see a Webhook URL that should be like this: https://hooks.slack.com/services/XXXXXXX/XXXXXXX/XXXXXXXXXXXXXX.
You will need that URL in the next step.
The second part requires you to create a Bash script that will post Slack messages when you run it. This is an example of how this file should look like:
URL="https://hooks.slack.com/services/XXXXXXX/XXXXXXX/XXXXXXXXXXXXXX"
# Slack Webhook URL
PAYLOAD="{
\"attachments\": [
{
\"title\": \"$PROCESS was restarted\",
\"color\": \"warning\",
\"mrkdwn_in\": [\"text\"],
\"fields\": [
{ \"title\": \"Date\", \"value\": \"$MONIT_DATE\", \"short\": true },
{ \"title\": \"Host\", \"value\": \"$MONIT_HOST\", \"short\": true }
]
}
]
}"
curl -s -X POST --data-urlencode "payload=$PAYLOAD" $URLIf you want to see how to customize the Slack messages you can take a look at the official documentation.
Before running the script, we should add some permissions (I saved it as slack.sh in you preferable location)
# chmod +x /my/script/path/slack.shNow let’s run it to see if it’s working
# /my/script/path/slack.shIf correct, you will see the message in your Slack channel.
Enable in Monit
You can see if you already have installed Monit by running
Now that you have Monit installed, you can add the configuration for your
process on /etc/monit/conf.d
check process apache2 with pidfile /run/apache2/apache2.pid
start program = "/bin/systemctl start apache2.service" with timeout 15 seconds
stop program = "/bin/systemctl stop apache2.service"
restart program = "/bin/systemctl restart apache2.service"
if changed pid then exec "/bin/bash -c 'PROCESS=Apache /my/script/path/slack.sh'"
if 1 restart within 1 cycle then exec "/bin/bash -c 'PROCESS=Apache /my/script/path/slack.sh'"
This configuration will restart the process in case you stop it manually or
if it gets stopped by itself. And when that happens it will call the slack.sh script to publish a message in
the Slack channel. You can see a bunch of real-world configuration examples on this link.
Don’t forget to restart Monit after you make changes in the configuration.
# monit reload
Reinitializing monit daemonYou can easily test this by manually killing the process.
# ps aux | grep apache
www-data 26247 801 0 00:09 ? 00:00:00 /usr/sbin/apache2 -k start# kill -9 26247The process will be automatically restarted after a couple of seconds and you will receive a message in your Slack channel.
This is just one type of message that you can implement in your Slack team, but you can also do some cooler things with Monit, like sending reports or alerts. You just need to change the configuration files based on that.
Setup Teams
As same as slack, just replace slack.sh with following script follow the same setup and don’t forgot TEAM CHANNEL LINK