Download FreeRTOS
 

Quality RTOS & Embedded Software

LIBRARIES
WHAT'S NEW
Simplifying Authenticated Cloud Connectivity for Any Device.
Designing an energy efficient and cloud-connected IoT solution with CoAP.
Introducing FreeRTOS Kernel version 11.0.0:
FreeRTOS Roadmap and Code Contribution process.
OPC-UA over TSN with FreeRTOS.

Running OTA over HTTP
With coreMQTT Agent and Code Signing


Preamble

While this demo uses the AWS IoT OTA update service, FreeRTOS is generic MIT licensed open source software, and can be used with any OTA mechanism that works for you. We do however recommend you digitally sign your firmware whichever OTA method you choose. That way the devices that receive the new executable image can verify it came from an authorized source and has not been been modified. You can use Code Signing for AWS IoT to sign your firmware or you can sign with your own code-signing tools.

Introduction

This example demonstrates the OTA update process using MQTT protocol for communicating with AWS IoT OTA update service and HTTP protocol for downloading the file from S3 pre-signed url. This example uses the coreMQTT agent to receive notification of a pending OTA Update Job along with the S3 pre-signed url for the firmware image. Use of the coreMQTT agent enables the OTA update functionality to run in the background, simultaneously with other tasks that are sharing the same MQTT connection. The download from S3 pre-signed url is implemented using coreHTTP. The example also demonstrates the use of code signing to verify that the downloaded firmware image has not been tampered with and comes from a trusted sender.

See the comments at the top of each C file in the Source Directory or the Functionality section for additional information.

Instructions

Getting started

Start by connecting one of the simple MQTT agent examples described on the main MQTT agent documentation page to the AWS IoT MQTT broker. That will ensure connectivity to AWS IoT is working correctly before moving on to set up the OTA service.

Once connectivity is working with a non OTA demo the following instructions show you how to:

  • Set up the cloud services to be able to store the firmware image and send it to the device.
  • Configure the device and run the OTA client to receive the update.
  • Prepare and create the OTA Update Job to send the firmware to the device.

Setting up the cloud services

Get started with the AWS console

This demo application uses the AWS console to create an IoT Thing, store a firmware image, and schedule an OTA job. An AWS account is required for you to be authorized to perform these operations. Click here to get started with creating and configuring an account.

Prerequisites for using the OTA Update Manager Service

Create a Thing

An AWS IoT Thing, or simply "Thing", is the cloud representation of a device. By registering your physical device as a Thing, you are able to store security credentials for it, assign permissions to your device (what it can and cannot do or access in the cloud), and select it when sending an OTA update. For this demo, the FreeRTOS Windows Simulator “device” will be represented by an IoT Thing and will be sent an OTA job from the AWS Console.

You will have created a Thing when connecting one of the non OTA demos to AWS IoT, so there are no additional steps here.


Setting up the device (the OTA client)

If, as recommended above, you have already used one of the other MQTT agent examples to connect with AWS IoT then you already have the source files, and have configured the network and AWS IoT connection.

Application Version

For a client to accept an OTA update, the version number of the update it receives must be higher than the version of the firmware that it’s currently running.

The application version of the device software is set in the “demo_config.h” source file with the “APP_VERSION_MAJOR”, “APP_VERSION_MINOR”, and the “APP_VERSION_BUILD” macros. The default settings of these macros are 0, 9, and 2 respectively. These do not need to be changed for now, but they will be modified in a future step.

OTA Control Protocol

The OTA Control Protocol setting manages the protocol used for AWS IoT Service control operations such as job notifications, updating the job status, and reporting progress. For control operations, only MQTT is supported at this time.

This setting is controlled by the “configENABLED_CONTROL_PROTOCOL” macro that can be found in the “ota_config.h” header file. The default value for this macro is “OTA_DATA_OVER_HTTP” and does not need to be modified for this demo.

OTA Data Protocol

The OTA Data Protocol defines the format used to transfer data over-the-air to and from the device. Currently MQTT and HTTPS are supported.

This option is controlled by setting the “configENABLED_DATA_PROTOCOLS” macro that can be found in the "ota_config.h" header file. The default setting for this macro is "OTA_DATA_OVER_HTTP" and does not need to be changed for this demo.

Code Signing

Code signing is used to confirm the author and guarantee that the code was not tampered with or corrupted since it was signed. As part of the OTA process, the signature of the file is verified after being completely downloaded.

The certificate used to verify the file needs to be set before you perform an over-the-air update. If you want to use a certificate stored on the device to verify the file, you must enter the path to the certificate in a later step. Otherwise, to configure the code signing certificate, set the "signingcredentialSIGNING_CERTIFICATE_PEM" variable in the "aws_ota_codesigner_certificate.h" header file.

Other OTA Library Configurations

Please refer to other OTA library configurations. These configuration values do not need to be changed for this demo.

Verify that the project builds and runs successfully

Before you continue, verify that you are able to build and run the project. You can do this by pressing F5 in the Visual Studio demo project, or by navigating to the “Debug” tab and clicking “Start Debugging”.


Prepare for creating the OTA update job

To send an OTA job, there needs to be an updated firmware image stored in an S3 bucket. The AWS IoT OTA Manager service will read the image out of this bucket and send it to the device. For the device to accept the image the received image must have a higher version number than the image already being executed on the device. The FreeRTOS Windows Simulator is being used for both building and running the demo, which cannot be done at the same time. Due to this, we need the following workflow (documented below the bullet list):

  1. Set the application version number and build the project executable.
  2. Verify that the executable can build and run correctly.
  3. Upload the executable /build/VisualStudio/Demo/RTOSDemo.exe to S3.
  4. Set the application version number to something lower than that used in step 1.
  5. Build the demo with the lowered version number.
  6. Run the updated demo and let it continue to run while waiting for an OTA Job.
Note: An example of a production workflow would be:
  1. Write the MCU firmware, integrating the OTA Client library source code.
  2. Program the device hardware (the MCU) with the initial firmware.
  3. Make changes to and test the firmware locally.
  4. Generate the binary for the new version of the firmware.
  5. Upload the new version to S3 and send it to the MCU with the an OTA Job.
Set the application version

To simulate having a “new” firmware image, increment the version number. For this demo, update the following macro values that can be found in the “demo.config” source file as shown here:

#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 2

Build the “new” firmware image

Generate the firmware image by building the project. To do this, press “Ctrl+Shift+b” or navigate to the build tab and press build within the Visual Studio project. This will generate the following executable:

/build/VisualStudio/Demo/RTOSDemo.exe

Because the demo runs on the FreeRTOS Windows Simulator, the “firmware image” in this case is the Windows executable.

Verify that the project builds and runs successfully

It’s best practice to verify firmware locally before you send it via an OTA update. See the “Verify that the project builds and runs successfully” section.

Upload the firmware image to the S3 bucket

  1. Sign in to the Amazon S3 console at https://console.aws.amazon.com/s3/.
  2. Click on the bucket created in the previous steps.
  3. Click on the “Upload” button that is under the “Overview” tab.
  4. Drag and drop “RTOSDemo.exe” to the bucket.
  5. Click “Upload” to add the executable to the bucket.

Lower the application version

Lower the application version. For this demo, update the following macro values that can be found in the “demo_config.h” source file as shown here:

#define APP_VERSION_MAJOR 0
#define APP_VERSION_MINOR 9
#define APP_VERSION_BUILD 1

Build and run the OTA client

Press the “Local Windows Debugger” button to build and run the demo. Allow the client to continue to run as it waits to receive an OTA Job from the AWS IoT OTA Manager service.


Creating the OTA update job using the AWS IoT console

At this point, you should have:

  • Created an AWS IoT Thing with the AWS IoT Service.
  • Setup the S3 bucket and managed permissions for the various services.
  • Uploaded a “newer” firmware image to the S3 bucket.
  • Completed the setup required for code signing.
  • Configured the OTA client running on your device.

Create the OTA update job

With the OTA Client running and the cloud services set up, the next step is to send the device a new firmware image by creating an OTA job. Start by going to the AWS IoT console.

  1. In the navigation pane of the AWS IoT console, choose Manage, and then choose Jobs. Then press “Create a Job”.


    Create a Job - click to enlarge

  2. Under Create a FreeRTOS OTA update job, choose Create OTA update job.


    Create OTA Update Job - click to enlarge

  3. On the OTA job properties page, enter a Job name for the FreeRTOS OTA update job (for example, "win_sim_update"). You can optionally enter a Description and add Tags to your job. Then press Next to continue.


    OTA Job Properties - click to enlarge

  4. You can deploy an OTA update to a single device or a group of devices. On the OTA file configuration page, under Devices to update, choose the things or thing groups associated with the devices you want to update. Under Select the protocol for file transfer, select the check box next to HTTP.


    Select a Device to update - click to enlarge

  5. Under Sign and choose your file, keep the default option of Sign a new file for me selected. Under Code signing profile, press the Create new profile button which is next to Existing code signing profile.


    Select the Files to OTA - click to enlarge

  6. On the Create a code signing profile page, under Profile name, enter "winsim_codesigning". Under Device hardware platform, select "Windows Simulator". Under Code signing certificate, change the default and select the Select an existing certificate checkbox. Under Certificates, select the certificate and certificate private keys that you generated earlier. If you followed the suggestion, these will be named "ecdsasigner.crt" and "ecdsasigner.key". Then press the Import button.

    Under Path name of code signing certificate on device, enter the path to the "ecdsasigner.crt" certificate that you just imported. However, if you set the variable "signingcredentialSIGNING_CERTIFICATE_PEM" during a previous step, you should enter "/" for the path here.

    Finally, press the Create button to make the code signing profile.


    Create a Code Signing Profile - click to enlarge

  7. Back on the OTA file configuration page, under File, change the default and select the checkbox for Select an existing file, then press the Browse S3 button and choose the executable you uploaded to S3 during a previous step. Under Path name of file on device, enter "./NewRTOSDemo.exe". This path is where the file that is downloaded during the OTA update will be saved.

    Note: The File type feature is supported in OTA library version v2.0.0 or higher.


    Choose S3 file and location on device - click to enlarge

  8. Under IAM role, select the IAM role created for the OTA process. Then press Next to continue.


    Choose IAM role - click to enlarge

  9. Under OTA job configuration, keep the default settings selected for Job run type (snapshot), Job start rollout configuration (constant rate), and Job run timeout configuration (No timeout). Then press the Create job button to finish creating the OTA Update Job.


    OTA Job Configuration - click to enlarge

  10. You can monitor that status of the job by pressing the “View Job” pop-up or by navigating to Manage > Jobs in the AWS IoT console. The job will be shown as “IN PROGRESS” until the device has successfully rebooted. 

    OTA Job Status - click to enlarge



Receive and activate the update

After the job has been created, the device should begin downloading the update. The progress of the download can be monitored by viewing the Visual Studio console. Below is an example of a progress update:

Current State=[WaitingForFileBlock], Event=[RequestFileBlock], New state=[WaitingForFileBlock]
Received valid file block: Block index=96, Size=2048
Number of blocks remaining: 3

The following message will print to the monitor after receiving the final block of the download:

Received final block of the update.

At this point, the demo will verify the code signing signature of the download. If it's successful, the following message will be printed to the console:

sig-sha256-ecdsa signature verification passed.

The demo will attempt to activate the new image after successfully downloading the update. For this demo, the update needs to be activated by manually running the downloaded executable. This is indicated by the following message:

Failed to activate new image: activateNewImage returned error: Manual reset required: OtaPalStatus_t=OtaPalSuccess

The demo will exit with the following message after successfully validating the update.

OTA agent task stopped. Exiting OTA demo.

If the image was successfully downloaded then it is ready to be activated. Stop the debugging session and close the window. Check the directory that you specified earlier for the new executable that was downloaded over-the-air. If you typed in “./NewRTOSDemo.exe” when you created the job, then the new executable will be located in the same directory as the Visual Studio solution. Run this executable. The device will begin to validate the image after the new executable has started. If this process is successful, then the following message will be printed to the console:

New image validation succeeded in self test mode.

When the job process is complete the state of the job will change from IN PROGRESS to COMPLETED in the IoT console. If the device has successfully passed the self-test process, the status of the job will change to Succeeded.


OTA Job Status - click to enlarge


Functionality

This demo illustrates how to build an application with the OTA library which uses the coreMQTT agent and coreHTTP libraries. Utilizing the coreMQTT agent enables the OTA agent to share the same MQTT broker connection with other tasks. The coreHTTP library is used for downloading the file using pre-signed S3 url.

Demo setup

This example application includes the resources and interface implementations required by the OTA library. See the OTA library design documentation for more additional information about these interfaces.

OTA PAL implementation

The OTA PAL interface is a set of APIs that are used to store, manage, and authenticate downloads. This demo provides an example implementation for the Windows Simulator platform.

Details of the port can be found in the Win32 repository on GitHub.

OTA OS Functional interface implementation

The OTA library calls the OS Functional interface implementations to perform actions that are typically provided by an operating system. This includes managing events, timers, and memory allocation. This demo uses FreeRTOS implementations of these interfaces.

Details of the port can be found in the ota_os_freertos.h file on GitHub.

OTA HTTP interface implementation

The OTA HTTP interface is used by the OTA library to connect to request file blocks from the S3 service. The OTA library uses the HTTP protocol to download the firmware image from the S3 pre-signed received from the AWS IoT OTA service.

Details of the port can be found in the OtaOverHttpDemoExample.c file on GitHub.

OTA application callback implementation

The OTA library provides a way to call a user defined function after receiving a job. This function is called when the OTA agent has completed an update job and when the device reboots after activating the image and is performing a self-test. Typically if the update job was successful, then the device will want to reset to boot into the new image. If the job was rejected, this callback will typically return without doing anything and wait for another job.

Details of the implementation can be found in the OtaOverHttpDemoExample.c file on GitHub.

Memory allocation setup

The OTA agent requires memory to download, decode, and store the job and file. The OTA agent will use the buffers provided by the user unless they are insufficient. In this case, it will dynamically allocate the memory required with the memory abstraction functions defined by the OTA OS Functional interface implementations.

User configuration

The demo requires some configuration that is specific to the device running it. The details of this can be seen in the earlier demo steps.

Entry point of the demo

The OTA Demo source code can be found in the OtaOverHttpDemoExample.c file on GitHub. This function is the primary thread for managing the over-the-air download process. It performs the following steps:

Initialize the OTA agent

In order to use the OTA agent, you'll need to provide interface implementations, buffers, and an app callback function to the "OTA_Init" API before starting the event processing task.

You can see an example of this in the OtaOverHttpDemoExample.c file on GitHub.

OTA callback

Create the OTA agent event processing task

The OTA agent operates by managing a state machine that tracks the current status of the download process. The state machine is influenced by receiving events that are sourced by either internal calls or the main application. The OTA agent requires a loop to be running to receive and process these incoming events before it can start. The demo runs this processing loop in a separate thread that calls the “OTA_EventProcessingTask” API that is defined by the OTA library.

You can see an example of this in the OtaOverHttpDemoExample.c file on GitHub.

Start the OTA agent

Once the OTA agent has been configured and the event processing task has been started, the OTA agent can begin the download process. This is done by sending an event with the "OtaAgentEventStart” ID to the OTA agent with the “OTA_SignalEvent" API.

You can see an example of this in OtaOverHttpDemoExample.c file on GitHub.

Monitor the OTA agent statistics

After the OTA agent has been started, it will continue to operate the download process in the background until it is shutdown. The OTA library provides the "OTA_GetStatistics" API for receiving statistics related to the download progress. These statistics include how many packets have been received, queued, processed, and dropped.

You can see an example of polling and printing these statistics in the OtaOverHttpDemoExample.c file on GitHub.

Shutting down the OTA agent

The OTA agent will continue to run in the background until it receives an event to shutdown. The OTA library provides an API for sending this event called “OTA_Shutdown”. The demo calls this API after receiving and activating a job to shutdown the OTA Agent.

You can see an example of this in the OtaOverHttpDemoExample.c file on Github.


Troubleshooting

The following sections contain information to help you troubleshoot issues with OTA updates.  

Topics

Copyright (C) Amazon Web Services, Inc. or its affiliates. All rights reserved.