« back to list

A sample usecase of AWS Lambda, API Gateway, DynamoDB and Cognito

Store data in AWS DynamoDB using a serverless AWS Lambda function, (accessible via AWS API Gateway) and secure the process with AWS Cognito.


The purpose of this tutorial is the following:

  1. Build a AWS Lambda function (running with Python 3) that stores message in AWS DynamoDB
  2. Expose this Lambda through AWS API Gateway
  3. Build a client for testing the freshly built stack
  4. Once this works, secure the connection with AWS Cognito

Please, note that this code and stack are only a hello-world-kind-of-app to familiarize yourself with the process of reaching DynamoDB via Lambda and API Gateway, and to authenticate your users with Cognito. It can contain some security issues that doesn't make it suitable for a production environment!


Build the Lambda function

AWS Lambda is a way of executing backend tasks without having to worry about the servers it's running on. This could be helpul especially for building apps that scale easily, within few clics. You only pay for the amount of processing time, calculated to the nearest hundreth of second. More information here.

The first step consist of  building a simple Lambda function that stores messages in DynamoDb. The data stored along with messages will be the user UUID, the date/time of storing, and a message UUID.

Let's begin by writing the Python code for this simple task:


import boto3
import json
import uuid
from datetime import datetime
from boto3.dynamodb.conditions import Key


def respond(err, response=None):
	return {
		'statusCode': '400' if err else '200',
		'body': err if err else json.dumps(response),
		'headers': {
			'Content-Type': 'application/json',
		},
	}

def get_user_id(username):
	'''If the username exists, get its UUID, else create a new one'''
	userTable = boto3.resource('dynamodb').Table('User')
	
	user = userTable.get_item(
		Key={
			'Username': username
		}
	)
	try:
		userId = user['Item']['UUID']
	except KeyError: # New user
		userId = str(uuid.uuid4())
		userData = {
			'Username': username,
			'UUID': userId,
		}
		userTable.put_item(Item=userData)
	
	return userId

def lambda_handler(event, context):

	method = event['httpMethod']

	if method not in ['POST']:
		return respond('Lambda function only supports POST request for the moment.')

	try:
		body = json.loads(event['body'])
		message = body['Message']
		username = body['Username']
		msgID = uuid.uuid4()
	except KeyError:
		return respond('Request is not properly formated.')
    
	userId = get_user_id(username)

	messageTable = boto3.resource('dynamodb').Table('Message')
	
	#create a new db entry
	messageData = {
		'EntityID': str(entityID),
		'SentOn': str(datetime.utcnow()),
		'SentBy': userId,
		'Value': message,
	}

	messageTable.put_item(Item=messageData)

	return respond(None, messageData)

The "event" passed as argument in the handler is the HTTP request made by client. The rest is pretty easy to understand. Just note the package "boto3" imported, which contains all the objects needed to interact with any AWS product.

Note Since this function communicates with DynamoDb, you'll need to create a new IAM role for execution. See here to follow the steps of the role creation. You will also need to build a new DynamoDB table, paying attention to name it precisely as you did in the Lambda code. During the table creation process, I chose to set up the EntityID of the message as the Primary partition key, and UserID as the Primary sort key. I also created a table for Users as you saw in the code, that simply contains Username as the primary key, and a related UUID for each user.

Log in to your AWS Lambda console, and follow the instruction to build a new Lambda. Select Python3.6 as the runtime, and once it's set up, paste your code inside the appropriate area.


Create a new API for interfacing with Lambda

In the AWS Console, go to the API Gateway section, and create a new API, giving a name and a description. Once it's created, create a new POST method. You'll be asked about 'Integration type', select "Lambda Function", and check 'Lambda Proxy Integration'. Select the region where your Lambda is located, then select the Lambda you've just created earlier.

Important You have to enable CORS for this API, as the request will come from a client located on a different host (go to 'Action' > 'Enable CORS'). Then deploy. DO NOT deploy the API before enabling CORS! (ie, don't do the same error that I did )

When you're done, you can deploy the API ('Action' > 'Deploy'). You'll be prompted to choose a deployment stage (eg "prod", or "staging") and to give a description about this stage. Once you have validated, your API is deployed, and you'll get the public URL to which this API responds.

Build a web client for storing message

Here we'll set up a supra-basic webpage for interacting with our Lambda function. Prepare to be amazed...

basic-webclient-screenshot

Let's have a look at the Javascript code behind, quite simple too, but we'll improve it later:


function ajax(method, url, data, callback) {
    
    var req = new XMLHttpRequest();
    req.open(method, url);

    req.addEventListener("load", function () {
        if (req.status >= 200 && req.status < 400) {
            callback();
        } else {
            console.error(req.status + " " + req.statusText + " " + url);
        }
    });

    req.addEventListener("error", function () {
        console.error("URL " + url + " unreachable.");
    });

    req.setRequestHeader("Content-Type", "application/json");
    
    data = JSON.stringify(data);

    req.send(data);
}

function apiClient(httpMethod){
    var formData = new FormData(form);

            var data = {
             'Username': formData.get('username'),
             'Message': formData.get('message'),
             };
            
            ajax(
                httpMethod,
                "https://XXXXX-XXXXX.execute-api.us-east-1.amazonaws.com/prod",
                data,
                function(){
                    alert("Message sent!");
                }
            )
}  

var form = document.querySelector("form");

form.addEventListener("submit", function(e){
    e.preventDefault();

    apiClient('POST');
})

Basically, we just connect to the API (via the URL given on the web interface of your API's deployment stage) using an AJAX request.

Test your service by sending a sample message. Check the console of your brower to catch any errors... If your client is not able to connect to the API, you'll probably need to re-check if CORS is enabled. The thing is: you need to re-deploy your API AFTER you enabled CORS, otherwise the change won't be taken into account, and you won't have any clue... I lost a significant amount of time figuring out this issue.

Log in your DynamoDB console, and check the table... Now, you should see your message stored (Yay )

Secure the connection to Lambda with AWS Cognito

Ok, now the stack works as it should. The only problem is that anyone can access your database, as long as he/she gets the public URL of the API... I'm sure you don't want this in your app, and luckily AWS also has the right product to offer, called Cognito

The goal here is to authenticate a user before allowing his client (web app or mobile app) to reach your API. You first need to create new User Pool in AWS Cognito, which is a pretty straight-forward task. For a detailed guide, see this tutorial. Set up your pool as you wish, either allowing or not users to sign up themselves. For the sake of this little workshop, just create one new user for yourself (note that you'll probably need to change the password you set up at the creation when you first sign in).

Now that you have a user-pool (containing a single user...) you must be able to authenticate the user who wants to reach the API: if he is in the one already registered in the user-pool, he'll be granted access. Otherwise, either he'll have to sign up first to the pool (if you allowed this feature when creating the pool) or he will be rejected. The last thing you need is to register your app ('General Settings' > 'App clients' > 'Add new app') with this user pool, so both can communicate during the authentication process. Once it's done, copy your App Client ID, you'll need it when enhancing the Javascript client.

The last step for securizing your API against unknown users is to authorize its calls only to people who has successfully pass the authentication process on Cognito. Go back to your API Gateway settings, and in the menu click on "Authorizers". Create a new Cognito User Pool Authorizer. Select the region where your pool is stored, choose the pool you've just created and the leave the rest as is. Note that now, the request has to carry a valid token in the "Authorization" header, otherwise it won't respond.

Now, we should have everything we need for upgrading the web client. Go the "General Settings" of your Cognito User Pool to copy two important information, the Pool ID and the Pool ARN Number. I've found this step-by-step tutorial useful for upgrading the Javascript client, have a look at it if you need more explanations of the following.

Here is the new JS client code for the function APIClient, the rest it the same as earlier (no need to say that you need to add a password field to your form! ).

Note You'll also need to use the Cognito Javascript SDK to access required JS objects that we use for reaching AWS. You can download them from Github, and simply add them at the end of your HTML code


    <!-- end of your HTML form file -->
    <script src="[PATH]/aws-cognito-sdk.min.js"></script>
    <script src="[PATH]/amazon-cognito-identity.min.js"></script>
    <script src="[PATH]/aws-sdk.min.js"></script>
    <script src="[PATH]/[YOUR-FILE].js"></script>

function apiClient(httpMethod){
    var formData = new FormData(form);

    var authData = {
        Username : formData.get('username'),
        Password : formData.get('password'),
    };
    var authDetails = new AWSCognito.CognitoIdentityServiceProvider.AuthenticationDetails(authData);

    var poolData = {
        UserPoolId : 'us-east-1_rXXXXX',
        ClientId : 'YYtuXXXXdkhc85XXXXXXX'
    };
    var userPool = new AWSCognito.CognitoIdentityServiceProvider.CognitoUserPool(poolData);

    var userData = {
        Username : formData.get('username'),
        Pool : userPool,
    };
    var cognitoUser = new AWSCognito.CognitoIdentityServiceProvider.CognitoUser(userData);

    cognitoUser.authenticateUser(authDetails, {
        onSuccess: function(result) {

            var idToken = result.getIdToken().getJwtToken();

            var data = {
             'Username': formData.get('username'),
             'Message': formData.get('message'),
             'Id-token': idToken,
             };

            AWS.config.credentials = new AWS.CognitoIdentityCredentials({
                IdentityPoolId : 'us-east-1_XXXX',
                Logins : {
                    'cognito-idp.us-east-1.amazonaws.com/us-east-1_XXXX' : idToken
                }
            });
            
            ajax(
                httpMethod,
                "https://XXXXX-XXXXX.execute-api.us-east-1.amazonaws.com/prod",
                data,
                function(){
                    alert("Message sent!");
                }
            )

        },

        onFailure: function(err) {
            alert(err);
        },

        newPasswordRequired: function(userAttributes, requiredAttributes) {
            cognitoUser.completeNewPasswordChallenge(
                prompt("First connexion. Please, choose new password: "))
        }
    }); 
} 

Basically, what we do here is the following:

  1. Gather the user credentials, and build a new authDetails object based on a AWS API class
  2. Gather the app and pool ID, and build a new userPool object
  3. Create a new object cognitoUser based on the user credentials and the pool object
  4. Call the method "authenticateUser" of this object. This may return different things:
    • onSuccess your user has been successfully authenticate. In this case, you need to get the ID-Token provided by the authentication, as you'll need to use it in your AJAX call.
    • onFailure the user has failed to authenticate
    • newPasswordRequired when he first signs in, he'll need to set up a new password before proceeding

If succeeded, the Cognito API will provide you an ID-Token that you'll have to insert in the header of your request before sending it to the API (remember? The name of this token in the request must match the one you set up in the Authorizer of your API):


    // [....]
    // Add the following to your ajax function
    req.setRequestHeader("Authorization", data["id-token"]);

    data = JSON.stringify(data);

    req.send(data);
}

And it's done!

You now call the AJAX function only if the user is authenticate via AWS Cognito..

  • If he is, the client will send the access token granted by Cognito to reach the API. The API will first verify if this token is valid, and then proceed to transmit the request to Lambda.
  • If he's not, he won't be able to call the API, thus to store anything in the database table nor to interact with the Lambda function.
Any suggestion or criticism? Please, do not hesitate to contact me, feedbacks are welcome!

A sample usecase of AWS Lambda, API Gateway, DynamoDB and Cognito



« back to list