How do I push custom metrics from an EC2 Linux instance to CloudWatch?

Resolution

I want to monitor OS metrics and performance counters of my Amazon Elastic Compute Cloud (EC2) Linux instance. How can I do this using Amazon CloudWatch?

Short Description

You can create a custom CloudWatch metric for your EC2 Linux instance statistics by creating a script through the AWS Command Line Interface (AWS CLI). Then, you can monitor that metric by pushing it to CloudWatch.

Resolution

Before proceeding, be sure that you install and configure the AWS CLI for use with the instance that you want to monitor.

Create a custom CloudWatch metric

To create your custom metric:

1. Log in to the instance through the AWS CLI.

2. Copy the following bash script, and then save it to your instance (for example, mem.sh).

This example script shows the values that you can publish in CloudWatch. In this example, the put-metric-data API call is used to push the following values to CloudWatch:

• Percentage of used memory (USEDMEMORY)
• Number of total connections (TCP_CONN)
• Number of TCP connections on port 80 (TCP_CONN_PORT_80)
• Number of users currently logged in (USERS)
• Percentage of I/O wait time (IO_WAIT)

========Sample script======

#!/bin/bash

USEDMEMORY=$(free -m | awk 'NR==2{printf "%.2f\t", $3*100/$2 }')

TCP_CONN=$(netstat -an | wc -l)

TCP_CONN_PORT_80=$(netstat -an | grep 80 | wc -l)

USERS=$(uptime |awk '{ print $6 }')

IO_WAIT=$(iostat | awk 'NR==4 {print $5}')

aws cloudwatch put-metric-data --metric-name memory-usage --dimensions Instance=i-0c51f9f1213e63159 --namespace "Custom" --value $USEDMEMORY

aws cloudwatch put-metric-data --metric-name Tcp_connections --dimensions Instance=i-0c51f9f1213e63159 --namespace "Custom" --value $TCP_CONN

aws cloudwatch put-metric-data --metric-name TCP_connection_on_port_80 --dimensions Instance=i-0c51f9f1213e63159 --namespace "Custom" --value $TCP_CONN_PORT_80

aws cloudwatch put-metric-data --metric-name No_of_users --dimensions Instance=i-0c51f9f1213e63159 --namespace "Custom" --value $USERS

aws cloudwatch put-metric-data --metric-name IO_WAIT --dimensions Instance=i-0c51f9f1213e63159 --namespace "Custom" --value $IO_WAIT

===============================================

1. After creating the bash script, give execute permissions to the file.

$ chmod +x mem.sh

1. Run the bash script to check that it works.

Push your metric to CloudWatch

1. Create a cron job:

$ crontab -e

1. Add this line to execute your script every minute:

*/1 * * * * /home/ec2-user/mem.sh

1. Save and exit.

When the crontab is saved, crontab: installing new crontab appears.

Monitor your EC2 instance

Find your custom metric in the CloudWatch console:

1. Open the CloudWatch console.

2. Choose Metrics.

3. Choose the All Metrics tab.

4. Choose Custom.

5. Choose the dimension Instance.

6. Select your custom metric by its InstanceId and Metric Name.

7. View the graph of your metric.

Other uses

You can use this example to build your own logic to process multiple dimensions, and then push that metric data to CloudWatch.

For example, suppose that you benchmark your application. Then, you discover that when the I/O wait time and percentage memory usage reach a certain threshold, the system stops functioning properly. To address this problem, you can monitor both values simultaneously in a script. Store the logical AND of the values in a third variable that you push to CloudWatch.

c=0

if [[ $IO_WAIT > 70 && $USEDMEMORY > 80 ]]

then

c=1

fi

aws cloudwatch put-metric-data --metric-name danger --dimensions Instance=i-0c51f9f1213e63159 --namespace "Custom" --value $c

For normal conditions, this variable is 0 (zero). For situations when both conditions are met, the value is set to 1 (one). You can then build custom alarms around these parameters to warn of problematic situations for your system.