Build a Serverless URL Shortener With Go

gowebing

This blog post covers how to build a Serverless URL shortener application using Go. It leverages AWS Lambda for business logic, DynamoDB for persistence, and API Gateway to provide the HTTP endpoints to access and use the application. The sample application presented in this blog is a trimmed-down version of bit.ly or other solutions you may have used or encountered.

It's structured as follows:

In this blog, you will learn:

Once you deploy the application, you will be able to create shortcodes for URLs using the endpoint exposed by the API Gateway and also access them.

# create short code for a URL (e.g. https://abhirockzz.github.io/)
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

# access URL via short code
curl -i $URL_SHORTENER_APP_URL/<short-code>


Let’s Get Started: Deploy the Serverless Application

Before you proceed, make sure you have the Go programming language (v1.16 or higher) and AWS CDK installed.

Clone the project and change it to the right directory:

git clone https://github.com/abhirockzz/serverless-url-shortener-golang
cd cdk


Start the Deployment

To start the deployment, all you will do is run a single command (cdk deploy), and wait for a bit. You will see a (long) list of resources that will be created and will need to provide your confirmation to proceed.

Don't worry: in the next section, I will explain what's happening.

cdk deploy

# output

Bundling asset ServerlessURLShortenerStack1/create-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/access-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/update-url-status-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/delete-url-function/Code/Stage...

  Synthesis time: 10.28s

This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
.......

Do you wish to deploy these changes (y/n)?


This will start creating the AWS resources required for our application.

If you want to see the AWS CloudFormation template which will be used behind the scenes, run cdk synth and check the cdk.out folder.

You can keep track of the progress in the terminal or navigate to the AWS console: CloudFormation > Stacks > ServerlessURLShortenerStack

AWS CloudFormation Stack

Once all the resources are created, you can try out the application. You should have:

Before you proceed, get the API Gateway endpoint that you will need to use. It's available in the stack output (in the terminal or the Outputs tab in the AWS CloudFormation console for your Stack):

AWS CDK output

Shorten Some URLs!

Start by generating shortcodes for a few URLs:

# export the API Gateway endpoint
export URL_SHORTENER_APP_URL=<replace with apigw endpoint above>

# for example:
export URL_SHORTENER_APP_URL=https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/

# invoke the endpoint to create short codes
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

curl -i -X POST -d 'https://dzone.com/users/456870/abhirockzz.html' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

curl -i -X POST -d 'https://abhishek1987.medium.com/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL


To generate a short code, you need to pass the original URL in the payload body as part of an HTTP POST request (e.g., https://abhishek1987.medium.com/).

'Content-Type: text/plain' is important. Otherwise, API Gateway will do base64 encoding of your payload.

If all goes well, you should get a HTTP 201 along with the shortcode in the HTTP response (as a JSON payload).

HTTP/2 201 
date: Fri, 15 Jul 2022 13:03:20 GMT
content-type: text/plain; charset=utf-8
content-length: 25
apigw-requestid: VTzPsgmSoAMESdA=

{"short_code":"1ee3ad1b"}


To confirm, check the DynamoDB table.

DynamoDB table records

Notice an active attribute there? More on this soon.

Access the URL Using the Shortcode

With services like bit.ly, etc., you typically create short links for your URLs and share them with the world. We will do something similar. Now that you have the shortcode generated, you can share the link with others (it's not really a short link like bit.ly, but that's ok for now!), and once they access it, they would see the original URL.

The access link will have the following format:

<URL_SHORTENER_APP_URL>/<generated short code> (e.g., https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b)

If you navigate to the link using a browser, you will be automatically redirected to the original URL that you had specified. To see what's going on, try the same with curl:

curl -i $URL_SHORTENER_APP_URL/<short code>

# example
curl -i https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/0e1785b1


This is simply an HTTP GET request. If all goes well, you should get an HTTP 302 response (StatusFound) and the URL re-direction happens due to the Location HTTP header which contains the original URL.

HTTP/2 302 
date: Fri, 15 Jul 2022 13:08:54 GMT
content-length: 0
location: https://abhirockzz.github.io/
apigw-requestid: VT0D1hNLIAMES8w=


How about using a shortcode that does not exist?

Set the Status

You can enable and disable the shortcodes. The original URL will only be accessible if the association is in an active state.

To disable a short code:

curl -i -X PUT -d '{"active": false}'  -H 'Content-Type: application/json' $URL_SHORTENER_APP_URL/<short code>

# example
curl -i -X PUT -d '{"active": false}'  -H 'Content-Type: application/json' https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b


This is an HTTP PUT request with a JSON payload that specifies the status (false, in this case, refers to disable) along with the shortcode, which is a path parameter to the API Gateway endpoint. If all works well, you should see an HTTP 204 (No Content) response:

HTTP/2 204 
date: Fri, 15 Jul 2022 13:15:41 GMT
apigw-requestid: VT1Digy8IAMEVHw=


Check the DynamoDB record: the active attribute must have switched to false.

As an exercise, try the following:

  1. Access the URL via the same short code now and check the response.
  2. Access an invalid short code, i.e., that does not exist.
  3. Enable a disabled URL (use {"active": true}).

Ok, so far we have covered all operations, except delete. Let's try that and wrap up the CRUD!

Delete

curl -i -X DELETE $URL_SHORTENER_APP_URL/<short code>

# example
curl -i -X DELETE https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b


Nothing too surprising here. We use an HTTP DELETE along with the shortcode. Just like in the case of an update, you should get a HTTP 204 response:

HTTP/2 204 
date: Fri, 15 Jul 2022 13:23:36 GMT
apigw-requestid: VT2NzgjnIAMEVKA=


But this time, of course, the DynamoDB record should have been deleted. Confirm the same.

What happens when you try to delete a short code that does not exist?

Don’t Forget To Clean Up!

Once you're done, to delete all the services, simply use:

cdk destroy


Alright, now that you've actually seen "what" the application does, let's move on to the "how."
We will start with the AWS CDK code and explore how it does all the heavy lifting behind setting up the infrastructure for our serverless URL shortener service.

With AWS CDK, Infrastructure-IS-Code!

You can check out the code in this GitHub repo. I will walk you through the keys parts of the NewServerlessURLShortenerStack function, which defines the workhorse of our CDK application.

I have omitted some of the code for brevity.

We start by creating a DynamoDB table. A primary key is all that's required in order to do that: in this case, shortcode (we don't have a range/sort key in this example).

    dynamoDBTable := awsdynamodb.NewTable(stack, jsii.String("url-shortener-dynamodb-table"),
        &awsdynamodb.TableProps{
            PartitionKey: &awsdynamodb.Attribute{
                Name: jsii.String(shortCodeDynamoDBAttributeName),
                Type: awsdynamodb.AttributeType_STRING}})


Then, we create an API Gateway (HTTP API) with just one line of code!

urlShortenerAPI := awscdkapigatewayv2alpha.NewHttpApi(stack, jsii.String("url-shortner-http-api"), nil)


We move on to the first Lambda function that creates shortcodes for URLs. Notice that we use an experimental module awscdklambdagoalpha (here is the stable version at the time of writing). If your Go project is structured in a specific way (details here) and you specify its path using Entry, it will automatically take care of building, packaging, and deploying your Lambda function! Not bad at all!

In addition to Local bundling (as used in this example), Docker-based builds are also supported.

    createURLFunction := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("create-url-function"),
        &awscdklambdagoalpha.GoFunctionProps{
            Runtime:     awslambda.Runtime_GO_1_X(),
            Environment: funcEnvVar,
            Entry:       jsii.String(createShortURLFunctionDirectory)})

    dynamoDBTable.GrantWriteData(createURLFunction)


Finally, we add the last bit of plumbing by creating a Lambda-HTTP API integration (notice how the Lambda function variable createURLFunction is referenced) and adding a route to the HTTP API we had created. This, in turn, refers to the Lambda integration.

    createFunctionIntg := awscdkapigatewayv2integrationsalpha.NewHttpLambdaIntegration(jsii.String("create-function-integration"), createURLFunction, nil)

    urlShortenerAPI.AddRoutes(&awscdkapigatewayv2alpha.AddRoutesOptions{
        Path:        jsii.String("/"),
        Methods:     &[]awscdkapigatewayv2alpha.HttpMethod{awscdkapigatewayv2alpha.HttpMethod_POST},
        Integration: createFunctionIntg})


This was just for one function. We have three more remaining. The good part is that the template for all these is similar. For example:

  1. Create the function.
  2. Grant permission for DynamoDB.
  3. Wire it up with API Gateway (with the correct HTTP method i.e. POST, PUT, DELETE).

I will not repeat it over here. Feel free to grok through the rest of the code.

Now that you understand the magic behind the "one-click" infrastructure setup, let's move on to the core logic of the application.

URL Shortener Lambda Function and DynamoDB Logic

There are four different functions, all of which are in their respective folders, and all of them have a few things in common in the way they operate:

  1. They do some initial processing: process the payload, or extract the path parameter from the URL, etc.
  2. Invoke a common database layer to execute the CRUD functionality (more on this soon).
  3. Handle errors as appropriate and return response.

With that knowledge, it should be easy to follow the code.

As before, some parts of the code have been omitted for brevity.

Create Function

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
    url := req.Body

    shortCode, err := db.SaveURL(url)
    if err != nil {//..handle error}

    response := Response{ShortCode: shortCode}
    respBytes, err := json.Marshal(response)
    if err != nil {//..handle error}

    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusCreated, Body: string(respBytes)}, nil
}


This function starts by reading the payload of the HTTP request body. This is a string which has the URL for which the shortcode is being created. It invokes the database layer to try and save this record to DynamoDB and handles errors. Finally, it returns a JSON response with the shortcode.

Here is the function that actually interacts with DynamoDB to get the job done.

func SaveURL(longurl string) (string, error) {
    shortCode := uuid.New().String()[:8]

    item := make(map[string]types.AttributeValue)

    item[longURLDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: longurl}
    item[shortCodeDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: shortCode}
    item[activeDynamoDBAttributeName] = &types.AttributeValueMemberBOOL{Value: true}

    _, err := client.PutItem(context.Background(), &dynamodb.PutItemInput{
        TableName: aws.String(table),
        Item:      item})

    if err != nil {//..handle error}

    return shortCode, nil
}


For the purposes of this sample app, the shortcode is created by generating a UUID and trimming out the last 8 digits. It's easy to replace this with another technique: all that matters is that you generate a unique string that can work as a short code. Then, it is all about calling the PutItem API with the required data.

Access the URL

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

    shortCode := req.PathParameters[pathParameterName]
    longurl, err := db.GetLongURL(shortCode)

    if err != nil {//..handle error}

    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusFound, Headers: map[string]string{locationHeader: longurl}}, nil
}


When someone accesses the short link (as demonstrated in the earlier section), the shortcode is passed in as a path parameter e.g. http://<api gw url>/<short code>. The database layer is invoked to get the corresponding URL from DynamoDB table (errors are handled as needed). Finally, the response is returned to the user wherein the status code is 302 and the URL is passed in the Location header. This is what re-directs you to the original URL when you enter the short code (in the browser).

Here is the DynamoDB call:

func GetLongURL(shortCode string) (string, error) {

    op, err := client.GetItem(context.Background(), &dynamodb.GetItemInput{
        TableName: aws.String(table),
        Key: map[string]types.AttributeValue{
            shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}}})

    if err != nil {//..handle error}

    if op.Item == nil {
        return "", ErrUrlNotFound
    }

    activeAV := op.Item[activeDynamoDBAttributeName]
    active := activeAV.(*types.AttributeValueMemberBOOL).Value

    if !active {
        return "", ErrUrlNotActive
    }

    longurlAV := op.Item[longURLDynamoDBAttributeName]
    longurl := longurlAV.(*types.AttributeValueMemberS).Value

    return longurl, nil
}


The first step is to use GetItem API to get the DynamoDB record containing URL and status corresponding to the shortcode. If the item object in the response is nil, we can be sure that a record with that shortcode does not exist: we return a custom error, which can be helpful for our function which can then return an appropriate response to the caller of the API (e.g., a HTTP 404). We also check the status (active or not) and return an error if active is set to false. If all is well, the URL is returned to the caller.

Update Status

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

    var payload Payload
    reqBody := req.Body

    err := json.Unmarshal([]byte(reqBody), &payload)
    if err != nil {//..handle error}

    shortCode := req.PathParameters[pathParameterName]

    err = db.Update(shortCode, payload.Active)
    if err != nil {//..handle error}

    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}


The first step is to marshal the HTTP request payload, which is a JSON (e.g., {"active": false}), and then get the shortcode from the path parameter. The database layer is invoked to update the status and handle errors.

func Update(shortCode string, status bool) error {

    update := expression.Set(expression.Name(activeDynamoDBAttributeName), expression.Value(status))
    updateExpression, _ := expression.NewBuilder().WithUpdate(update).Build()

    condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
    conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()

    _, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
        TableName: aws.String(table),
        Key: map[string]types.AttributeValue{
            shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
        UpdateExpression:          updateExpression.Update(),
        ExpressionAttributeNames:  updateExpression.Names(),
        ExpressionAttributeValues: updateExpression.Values(),
        ConditionExpression:       conditionExpression.Condition(),
    })

    if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
        return ErrUrlNotFound
    }

    return err
}


The UpdateItem API call takes care of changing the status. It's fairly simple except for all these expressions that you need, especially if you're new to the concept. The first one (mandatory) is the update expression where you specify the attribute you need to set (active in this case) and its value. The second one makes sure that you are updating the status for a short code that actually exists in the table. This is important since otherwise, the UpdateItem API call will insert a new item (we don't want that!). Instead of rolling out the expressions by hand, we use the expressions package.

Delete Shortcode

func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

    shortCode := req.PathParameters[pathParameterName]

    err := db.Delete(shortCode)
    if err != nil {//..handle error}
    return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}


The delete handler is no different. After the shortcode to be deleted is extracted from the path parameter, the database layer is invoked to remove it from the DynamoDB table. The result returned to the user is either an HTTP 204 (on success) or the error.

func Delete(shortCode string) error {

    condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
    conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()

    _, err := client.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
        TableName: aws.String(table),
        Key: map[string]types.AttributeValue{
            shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
        ConditionExpression:       conditionExpression.Condition(),
        ExpressionAttributeNames:  conditionExpression.Names(),
        ExpressionAttributeValues: conditionExpression.Values()})

    if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
        return ErrUrlNotFound
    }

    return err
}


Just like UpdateItem API, the DeleteItem API also takes in a condition expression. If there is no record in the DynamoDB table with the given short code, an error is returned. Otherwise, the record is deleted.

That completes the code walk-through!

Wrap Up

In this blog post, you learned how to use DynamoDB Go SDK using a URL shortener sample application. You also integrated it with AWS Lambda and API Gateway to build a serverless solution whose infrastructure was also defined using actual code (as opposed to yaml, JSON, etc.), thanks to the Go support in AWS CDK.

 

 

 

 

Top