Step function and async lambda

Example project with async lambda and dynamo db

  1. Step function invoke lambda with the wiatforTaskToken method
  2. The async lambda can get the task token from event
  3. When async process finishes, the last step will be calling step function api to sendTaskSccess/fail/heartbeat with token from step 1
  4. Step function will continue to the next state
  5. A good idea will be configure a heartbeat/timeout method so it will not wait for function forever.

AWS Step Functions can coordinate asynchronous Lambda functions by using the Wait for Task Token API. When you invoke a Lambda function asynchronously, Step Functions returns a task token. You can then use the Wait for Task Token API to wait for the Lambda function to complete.

The following is an example of how to coordinate asynchronous Lambda functions using Step Functions:

  1. Create a Step Functions state machine that has two states:
    • Start Lambda state that invokes the Lambda function asynchronously.
    • Wait for Task Token state that waits for the Lambda function to complete.
  2. In the Start Lambda state, specify the Lambda function that you want to invoke and the task token that you want to use.
  3. In the Wait for Task Token state, specify the task token that you received from the Start Lambda state.

When the Step Functions execution reaches the Start Lambda state, it will invoke the Lambda function asynchronously. The Lambda function will then execute and return a task token. The Step Functions execution will then move to the Wait for Task Token state. The Wait for Task Token state will wait for the Lambda function to complete and return the task token. When the task token is returned, the Wait for Task Token state will complete and the Step Functions execution will continue.

Here is an example of the Amazon States Language (ASL) code for the Step Functions state machine:

AWS Document


When you invoke a Lambda function, the execution will wait for the function to complete. If you invoke the Lambda function with a callback task, the heartbeat timeout doesn’t start counting until after the Lambda function has completed executing and returned a result. As long as the Lambda function executes, the heartbeat timeout is not enforced.

The “Wait for callback” pattern lets you pause your workflow at this state until some external process is complete. To use this pattern, you must pass a unique task token along with the API parameters for [Service][API]. You retrieve the task token from the Context Object , and put it into a key value pair, for example:
"MyTaskToken.$": "$$.Task.Token"

Later, after your external process completes, you will pass the task token to the Step Functions SendTaskSuccess  or SendTaskFailure  APIs to report the outcome and resume the execution. Note: Only APIs whose parameters support arbitrary data can be used with the “Wait for Callback” pattern.