Synthetic Monitoring using AWS Canary & Cloudwatch

Introduction

Every time you look at an AWS service, you’ll discover a slew of hidden Easter eggs. AWS Cloudwatch synthetic monitoring with canaries is one of those Easter eggs.

Synthetic monitoring is a type of monitoring that involves the use of emulation or scripted recording of transactions. Script(s) are used to imitate an action or journey that a user would take on an application, website, or system. Synthetic monitoring is also known as active monitoring or proactive monitoring. These actions or journeys are then continuously monitored at specified intervals for performance, functionality, and availability measures.

AWS Canaries

Amazon CloudWatch now allows you to monitor your endpoints and APIs by creating canaries using Cloudwatch Synthetics. The canaries are programmable scripts that follow the same paths and do the same activities as customers, allowing you to validate your customer experience even when there is no client traffic on your apps. The canaries can be set to run on a set schedule.

Canaries’ current versions are scripts written in either Node.js or Python. In your account, they create Lambda functions that leverage Node.js or Python frameworks. They support both HTTP and HTTPS protocols.

Canaries check the availability and latency of the endpoints and can store load time data and screenshots of the UI. They monitor REST APIs, URLs, and website content, and they can check for unauthorized changes from phishing, code injection, and cross-site scripting.

AWS provides three options for creating canaries. The blueprint is the simplest. The other two are the inline editor and Import from S3 if you have your own synthetic monitoring scripts.

The remainder of this post will provide a quick introduction to the various types of canary blueprints as well as instructions on how to set up an API canary.

Canary Blueprints

Following canary blueprints are available for synthetic monitoring.

• Heartbeat Monitor — Heartbeat scripts load the specified URL and store a screenshot of the page and an HTTP archive file.
• API Canary — API canaries can test the basic Read and Write functions of a REST API.
• Broken Link Checker -The broken link checker collects all the links inside the URL that you are testing.
• Visual Monitoring — The visual monitoring blueprint includes code to compare screenshots taken during a canary run with screenshots taken during a baseline canary run.
• Canary Recorder — With the canary recorder blueprint, you can use the CloudWatch Synthetics Recorder to record your click and type actions on a website and automatically generate a Node.js script that can be used to create a canary that follows the same steps.
• GUI Workflow — The GUI Workflow Builder blueprint verifies that actions can be taken on your webpage.

Canary Blueprint Example

Step 1

Log into your AWS Console and head over to CloudWatch. You will see down the left-hand side, Canaries.

Step 2

Under Create canary, select the “Use a Blueprint” option and under Blueprint, select “Heatbeat monitoring”

Step 3

Give your canary a name. Let’s call it “myfirstcanary” for the sake of demonstration. Please keep in mind that no uppercase characters can be used in the name.

Step 4

Add the Application or endpoint, Method, Request Data, and Headers, to the HTTP request option. Give your step a name, and when finished, click the “Save” button.

Step 5

Leave the Script Editor alone. Use the Environment Variable option if you need to send in environment variables. For demonstration purposes, we’ll leave it blank.

Step 6

Select how frequently you want the canary to run under the Schedule box. You can choose between “Run Continuously,” “CRON expression,” and “Run Once.” We’ll choose “Run Continuously” and set the run frequency to every 30 minutes.

Step 7

Leave the Data Retention option at its default setting of 31 days.
After that, either leave the Data Storage option for the test artifacts at the default S3 location or adjust it to your preferred place. You can also instruct Canary to establish a new access authorization role or pick an existing one.

Step 8

You will need to rather have created or allow the canary to create an IAM role to allow the canary the correct permissions to run. Again for this, I shall leave this default.

Step 9

Next, arguably the most important part is notifying yourself and others that an issue is present. You have an option to set CloudWatch alarms, my preference is to set Failed as the Metric name, Greater/Equal to Threshold 1 (1 Failure) over the course of 15 minutes. Now as we left the default above of 5 minutes that the canary runs, it will check 3 times within 15 minutes and after 15 minutes it fails, then alerts. You can of course play around with these settings to get something that fits your situation better.

Click create a topic for the SNS or select pre-existing SNS topic you may have created.

There are more advanced options at the bottom of the canary for VPC and Tags but we will leave these out for this tutorial.

Click the orange button Create canary and give it some time to create, it will give you a warning of this. It will proceed to create, run and then will display Pass or Fail. If it fails please go back through your steps create a new canary and see what may have been missed.

If we go to the canaries page we will see our canary running.

At this point, you have successfully configured, created, and run your canary. You can see how you can create a clear customer pattern that you may wish to make sure completes as expected. This can alert you or a group of admins that the site is having difficulties and therefore react and act before more customers notice and in some cases before they notice at all.

You can run multiple canaries at once, carrying out various checks in all areas of your site.

I will update this with more checks and processes in the following days, please give claps and comments if you enjoyed this article and I will make more in the coming weeks.

Thank you 😁