Toolbox for AWS IoT streamlines the process of developing IoT applications. It provides a graphical user interface designed to support developing for IoT on AWS. The CDK stack in this repository deploys the frontend and all necessary infrastructure on AWS. The solution uses serverless services to provide the necessary backend infrastructure. Read the readme for each feature get more information.
Warning This example is for experimental purposes only and is not production ready. The deployment of this sample can incur costs. Please ensure to remove infrastructure via the provided commands when not needed anymore.
Video on the AWS IoT YouTube Channel explaining the Toolbox for AWS IoT
Toolbox for AWS IoT has two key features. The testing of SQL statements of Rules for AWS IoT and the recording and replaying of messages of AWS IoT Core.
- Custom message - provide a sample message to retrieve the output of the IoT SQL statement
- Topic message - subscribes to a topic and test the first incoming message.
The result of the IoT SQL statement is than displayed next to the provided input.
If you intend to use IoT functions which require to pass an ARN of an IAM role via the Toolbox please see the section on AWS IoT Rule functions which call other AWS services and if you intend to invoke a Lambda function as part of your IoT SQL statement see section AWS IoT Rule functions which invoke Lambda functions
After completing the Prerequisites and Getting started you can start using the Toolbox for AWS IoT. Open the frontend link of the CDK output. A password was sent to the provided email address during deployment.
Before installing this sample, make sure you have the following installed:
Note This architecture creates resources that have costs associated with them. Please see the AWS Pricing page for details and make sure to understand the costs before deploying this stack.
After completing the prerequisites run the following commands to deploy the full solution! A user with your specified
email address will be created in the Cognito User Pool and the temporary password will be sent to you. Deployment takes several minutes because it creates an optimized production build for the frontend and creates cloudfront distribution.
If you wish to deploy the solution without an initial user and instead create the user yourself in the Cognito User Pool,
deploy the solution without the -c initialUserEmail=<your email address> parameter.
cd cdk
npm install
npx cdk deploy --all -c initialUserEmail=<your email address> -c region=<region>
If you intend to test IoT functions which require to pass an ARN of an IAM role via the Toolbox please see the section on AWS IoT Rule functions which call other AWS services
In case you want to deploy the solution without the frontend (e.g. while customizing the CDK stack), run
npx cdk deploy --all -c initialUserEmail=<your email address> -c region=<region> -c deployFrontend=false
To run the integration tests
npm run integ-test
The frontend is a single page application hosted in a Amazon S3 bucket via Amazon Cloudfront.
You have two options to provide input to your SQL statement:
- Custom message - provide a sample message to retrieve the output of the IoT SQL statement.
- Topic message - subscribes to a topic and test the first incoming message.
All requests are synchronous and targeted towards Amazon API Gateway. The API Gateway invokes a AWS Lambda which invokes the correct AWS Step Function state machine. Details of each Step Function per feature are listed below.
The flow for the custom message is orchestrated via AWS Step Functions. The custom message including a task token is ingested to a temporary IoT rule with the SQL statement which sends the result to an AWS Lambda function. The Lambda returns the result back to the AWS Step Function and sends a task success.
The AWS Step Function creates the necessary infrastructure and deletes it afterwards. The flow is the following:
- Define name for temporary IoT Rule
- Create temporary IoT Rule which is created using the test SQL statement. The statement excludes the WHERE clause to avoid an unintended invocation.
- Catch SQL exception if the SQL statement is malformed
- Ingests a message via basic ingest to the rule. The message contains a task token as value. The result is forward to a receiving Lambda which forwards the token back to Step Function and reports a success.
- Report timeout if no output is received.
- Delete temporary IoT rule.
The flow for topic message is orchestrated via AWS Step Functions. An IoT Rule is created to get the first incoming message on the topic in the SQL statement. The rule adds a task token as value. If a message is received it is ingested into an IoT rule which contains the SQL statement which is to be tested. The rule forwards the output to a AWS Lambda function. The AWS Lambda function returns the result back to the AWS Step Function and sends a task success.
The AWS Step Function creates the necessary infrastructure and deletes it afterwards. The flow is the following:
- Define name for temporary IoT Rules
- Create temporary IoT Rules. The get message rule to fetch the first message incoming the to topic specified in the WHERE clause in the IoT SQL test statement. This rule includes the task token as user property.
- If no message is received for this topic the state triggers a timeout.
- Next, the ingest message rule is created. This rule contains the SQL statement excluding the WHERE clause and forwards the task token. The result message is forwarded to a Lambda which sends a task success back to the step function.
- Catch SQL exception if the SQL statement is malformed.
- Ingests a message via basic ingest to the rule. The message contains the task token as value. The result is forward to a receiving Lambda which sends the token back to Step Function and reports a task success.
- Report timeout if no output is received.
- Delete temporary IoT rules.
You can record MQTT messages and replay them. All requests are synchronous and targeted towards an Amazon API Gateway. The Amazon API Gateway invokes the corresponding Lambda.
The user sends a request to an Amazon API Gateway which invokes a AWS Lambda function. The AWS Lambda writes the metadata information (name, topic, ...) to a Amazon Dynamo DB table. Next, the AWS Lambda function creates the temporary IoT rule which stores all incoming messages on the specified topic in a Amazon Dynamo DB table.
With a separate request the user can stop the recording. This deletes the IoT rule and completes the metadata fields like start/stop time in the Amazon Dynamo DB table.
To start a replay the user defines the recording(s) and if desired a topic prefix. An AWS Lambda function starts an Amazon ECS Fargate task. This task gets all information about the recording(s) from the table, gets the messages which belong to the recording(s) and replays them in the same timely order to IoT core.
The replay task automatically stops if all messages are replayed, but can be manually cancelled from the user via an API call.
The record and replay functionality saves the information in three different Amazon Dynamo DB tables:
Metadata - stores the metadata information about a recording
| recordingId (Partition key) | createdAt | recordingName | status | topic |
|---|---|---|---|---|
| unique ID | timestamp | given name | PREPARING IN_PROGRESS FINISHED ABORTED FAILED DELETING |
recorded topic(s) |
Recording - contains the recorded messages
| recordingId (Partition key) | timestamp (Sort key) | message | mqttProperties | topic | userProperties |
|---|---|---|---|---|---|
| unique ID | timestamp | message content | message mqtt properties | message topic | message user properties |
Replay history - contains the replay history of records
| recordingId (Partition key) | replayId (Sort key) | recordingName | status | taskId | topicPrefix |
|---|---|---|---|---|---|
| unique ID | timestamp | given name | PREPARING IN_PROGRESS FINISHED ABORTED FAILED DELETING |
ID of ECS task | topic prefix for replay |
Warning This application was written for demonstration and educational purposes and not for production use. The Security Pillar of the AWS Well-Architected Framework can support you in further adopting the sample into a production deployment in addition to your own established processes. Take note of the following:
-
The application uses encryption in transit and at rest with AWS-managed keys where applicable. Optionally, use AWS KMS with DynamoDB, CloudWatch, and S3 for more control over encryption keys.
-
This application uses Powertools for AWS Lambda (Python) to log to inputs and outputs to CloudWatch Logs. Per default, this can include sensitive data contained in user input. Adjust the log level and remove log statements to fit your security requirements.
-
In order to simplify the setup of the demo, this solution uses AWS managed policies associated to IAM roles that contain wildcards on resources. Please consider to further scope down the policies as you see fit according to your needs.
-
If your security controls require inspecting network traffic, consider to attach the Lambda functions to a VPC via its
VpcConfig.
Note Even though we provide the Toolbox for AWS IoT as a sample which is not intended for production usage, it is deployed on public endpoints. Therefore, by default the CDK stack will deploy the following security-related configurations as well.
In case you already have controls in place (e.g. AWS WAF), your organizations policies restrict which types of resources you are allowed to create, or you want to reduce cost, each control can be disabled. To disable a control, either
- set the respective CDK context variable in cdk.json to
false - pass the context variable on the command line when deploying, e.g.
npx cdk deploy --all -c initialUserEmail=<your email address> -c enableApiLogging=false - set the respective environment variable to false before deploying the stack, e.g.
export TOOLBOX_ENABLE_API_LOGGING=false
NOTE: AWS WAF for CloudFront requires creation of the WebACL in the us-east-1 region. Many customers restrict which regions are usable. To enable a hassle-free deployment we therefore decided to not deploy AWS WAF for CloudFront by default but recommend to do so.
| Control | Description | Enabled by default | CDK context variable | Environment variable | Further information |
|---|---|---|---|---|---|
| Amazon API Gateway Access Logs | Logs all API Gateway Requests to CloudWatch | ✅ | enableApiLogging |
TOOLBOX_ENABLE_API_LOGGING |
API Gateway access logging |
| Amazon CloudFront Access Logs | Logs all Cloudfront Requests to an S3 bucket | ✅ | enableCloudFrontLogging |
TOOLBOX_ENABLE_CLOUDFRONT_LOGGING |
CloudFront access logging |
| Amazon S3 Access Logs | Logs all requests to the SPA and CloudFront access logs bucket to a separate bucket | ✅ | enableS3Logging |
TOOLBOX_ENABLE_S3_LOGGING |
S3 access logging |
| Amazon VPC Flow Logs | Logs all VPC traffic of the message replay cluster VPC to CloudWatch | ✅ | enableVpcLogging |
TOOLBOX_ENABLE_VPC_LOGGING |
VPC Flow logs |
| AWS WAF | Enables WAF with the Common Rule Set and the IP Reputation List on API Gateway and Cognito | ✅ | enableWAF |
TOOLBOX_ENABLE_WAF |
AWS WAF |
| AWS WAF - CloudFront | Enables WAF with the Common Rule Set and the IP Reputation List on CloudFront | ❌ | enableCloudFrontWAF |
TOOLBOX_ENABLE_CLOUDFRONT_WAF |
AWS WAF |
AWS IoT Rule SQL provides several functions which allow you to retrieve data from other AWS services, such as get_thing_shadow or get_dyanmodb.
These functions expect the ARN of an IAM role, which grants the rules engine permission to invoke the service, as a parameter. For example,
for get_dynamodb you need to provide a role which allows dynamodb:GetItem. For security reasons the Toolbox for AWS IoT cannot
create the those roles itself during deployment as the permission scope would be too broad.
If you want to use those functions you need to create the respective roles and provide their ARNs during deployment time of the Toolbox for AWS IoT. To do this,
- create an IAM role which can be assumed by AWS IoT and contains the required permissions (see the example below)
- (re-)deploy the Toolbox for AWS IoT with the role ARN(s) passed as parameter, separated by a comma:
npx cdk deploy --all -c initialUserEmail=<your email address> -c region=<region> -c ruleRoleArns=arn:aws:iam::<account-id>:role/<role1>,arn:aws:iam::<account-id>:role/<role2>
Here is an example policy and trust policy which allows the rules engine to query for thing shadows
Trust policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "iot.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}Permission policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Action": [
"iot:GetThingShadow"
],
"Resource": [
"*"
]
}
]
}AWS IoT can invoke a Lambda function on the MQTT message for advanced processing. If you want to invoke a Lambda function, you must grant AWS IoT lambda:InvokeFunction permissions to invoke the specified Lambda function. As part of each IoT SQL statements test run the Toolbox creates a temporary IoT rule with the name prefix iottoolbox_ingest_ followed by a unique id. The IoT rule needs to be referenced as source ARN as part of the permission policy of the Lambda function. The documentation how to call a Lambda function shows a more detailed overview.
Here is an example to how to grant the lambda:InvokeFunction permission using the AWS CLI:
aws lambda add-permission --function-name "function_name"
--region "region"
--principal iot.amazonaws.com
--source-arn arn:aws:iot:eu-central-1:account_id:rule/iottoolbox_ingest_*
--source-account "account_id"
--statement-id "unique_id"
--action "lambda:InvokeFunction"
- Empty the Amazon S3 bucket created as part of the CDK stack.
- Run
npx cdk destroyto cleanup all related resources in your account. - Delete DynamoDB tables manually
See CONTRIBUTING for more information.
This library is licensed under the Apache 2.0 License. See the LICENSE file.