Creating a serverless API and hosting a frontend with S3 (Detecting Paris’ locked bicycle stations 4/5)
This series of articles is about me spending way too much time trying to solve a niche problem (detecting locked bicycle stations in Paris, see Part 1) while learning how to use the AWS Serverless stack. To find the other articles, skip to the bottom of the page.
UX and API Design
After having implemented a detection algorithm using DynamoDb Streams and Lambda functions (see part 3), I need a frontend to display the results. A simple HTML table listing the status of each station will be enough, but where is the fun in that? So I’ll create a basic but functional app giving the ability to anyone to find the closest station and see its current content and state. I will also use this application to display some of the statistics my pipeline is computing.
But before spending hours on development, I’ll first work on a simple mockup. And for each page, I’ll try to define which API will be required.
In the end, I settled on five main pages:
- The home page contains a few metrics about today’s network usage (with a link to a more detailed statistics page) and a list of the last stations’ change of state (the stations that went from ‘OK’ to ‘Locked’, and vice-versa).
- The statistics page displays today’s metrics and a chart comparing the predicted activity with the real one. I’ll use it myself to quickly monitor how good my predictions are.
- The map page locates the user and the nearby stations and displays them on a map, with custom pins to quickly see locked or unavailable stations. Clicking on a pin will redirect to the station’s details page.
- The stations' list page allows a user to find a specific station by name, with a search box.
- The station details page represents all the information about a specific station, with a small map, details about the station’s content, and a chart displaying the station’s own activity for today.
All the data required to populate those pages can be summarized by the following endpoints.
Note: ideally, I would have relied on path parameter to more easily get the activity or prediction of a single station (for example : GET /activity/{station-code}. But, cost and technical constraints regarding caching prevented me from doing that. More about that in part 5.
Serverless API with API Gateway
Let’s start by implementing the API. Surely, to expose an API on the web you need some kind of always available server, right?
Well, no. Or more exactly, AWS provides a solution that manages that for you, and you just need to focus on what happens when an endpoint is actually called. It’s the API Gateway.
In a nutshell, API Gateway acts as a proxy between the web and your code. It exposes endpoints and when an endpoint is called, it triggers the service of your choice, forwarding the call’s parameters and returning whatever the service decides. In our case, we simply have to trigger a Lambda function. And from a Lambda function’s perspective, it’s just another event, with a specific event payload.
So let’s create the first “GET /stations” endpoint:
In the SAM/CloudFormation template, we need to create a new function, and declare an “HttpAPI” event, with the wanted path and method, and a reference to the API Gateway. Then we have to create the API Gateway. SAM provides a simpler declaration with the AWS::Serverless::HttpApi type, which mostly needs the CORS configuration if needed and a reference to the domain name and related parameters. Here I’m using my own velinfo.fr (Velinfo, Velib’+info, got it?) domain name. More exactly I created a specific api.velinfo.fr subdomain just for the API.
This requires setting up a domain name through AWS Route 53 and creating the corresponding hosted zones and certificates, but I won’t dive into this topic.
Note: in fact there are two SAM types for API Gateway: AWS::Serverless::Api and AWS::Serverless::HttpApi. They represents respectively the RestApi and HttpApi Gateways. The first one is the most feature rich, with its own caching for example, while the second one is the most barebone but also the most performant and the cheapest (by about x3). I’m using the last one.
The Lambda function is quite similar to the ones we’ve already seen. The main differences are that it takes a specific event type as an argument and that it will return a structured object that will contain the status code, body, and potential headers needed that represent the REST response.
Here, I’m not using the event argument, but I could use it to find the contextual information regarding this specific API call, the path and query parameters used, and so on.
Here is what the event would look like:
Let’s deploy the new endpoint (using sam deploy as usual) and test it.
Works well!
Most of my endpoints are simply exposing what’s inside a specific DynamoDb table, so I’ll skip their implementation.
Frontend hosting with S3
With the API created, I can start implementing the frontend. As Angular development is not the topic of this article, I’ll fast forward a bit.
For those interested, I mostly relied on the Angular Material component library for basic styling and structure. I used agm for the map (maybe not the most logical choice in the long run, as Google Maps’ free plan is quite restrictive) and ngx-charts for the custom charts.
I can build my frontend app, and generate the corresponding static files using the Angular CLI: ng build --prod
But, how am I supposed to host that? Obviously, I could rent a server at any web hosting company and host my application there, but that would not be very “serverless” 😉
In Part 1, I quickly talked about S3, AWS’s “Simple Storage Service”.
S3 is an extremely resilient object storage service. An object can be anything, a .html or a .css file for example. And what is a frontend application, if not a bunch of static files served to a web browser?
Even if web hosting was not initially a core functionality of S3 (it was really intended as a storage service, where objects/files would be manipulated through an SDK or an API), hosting static websites quickly became an obvious use case for S3 and a web hosting feature was introduced.
To use it, we first need to define a new S3 “bucket” (a bucket is a unit of storage in S3, see it as a shared folder) and define a WebSiteConfiguration to specify where is the index document (in my case, the index.html file at the root of the bucket). We also need to define an access policy allowing everyone to read the content of the bucket. Finally, we need to know what is the public URL that has been generated for this bucket so I’ll add an Output for that.
To create the bucket, I simply need to deploy the configuration with SAM: sam deploy . At the end of the operation, I’ll get the generated bucket’s URL.
But if I try to call that now, it returns nothing. That’s normal, as I created the bucket, but I’ve put nothing in it. I need to push my static site content to the bucket, using the AWS CLI. I’ll use the AWS CLI for that.
#build using angular CLI
ng build --prod#clear the bucket content
aws s3 rm s3://velinfo-frontend --recursive#push the new content
aws s3 cp dist/velinfo s3://velinfo-frontend --recursive --cache-control max-age=31536000
Calling the bucket’s URL in the browser will display the website, but the API calls fail. That’s because of our good friend CORS, which is blocking my API call as my site (or bucket)’s domain is not https://www.velinfo.fr so it’s not in the accepted domains list.
To properly host the site on www.velinfo.fr, I need to create a DNS record to point to the bucket’s URL. To do that, I can simply use AWS's own DNS service (Route53) and add a CNAME record.
Now, it’s working!
Architecture summary
So, let’s recap what happens when I’m accessing my website:
- the www.velinfo.fr domain name points to the velinfo-frontend S3 bucket, using a DNS record managed by Route53
- the velinfo-frontend bucket is configured to serve its content as a website
- the frontend application is built using the Angular CLI and is pushed to the bucket using the AWS CLI
- the frontend application fetches data from the api.velinfo.fr REST API
- api.velinfo.fr points to the API Gateway using a DNS record managed in Route 53
- the API Gateway triggers the Lambda function corresponding to the endpoint called and gives it all the information about the call
- the Lambda function calls the corresponding DynamoDb table
- the Lambda function returns a specific output representing the status code, headers, and body. And all this gets forwarded back by the API Gateway to the caller.
- the whole infrastructure is defined in a SAM/CloudFormation template
The API and the web app are now finished. But before closing the series, I want to tackle one last thing: performance.
See you in part 5!
- Part 1: Choosing the AWS serverless stack for a prototype
- Part 2: The backbone of a serverless app: Lambda functions and DynamoDb tables
- Part 3: Implementing a real-time detection algorithm with Lambda functions and DynamoDb streams
- Part 4: Creating a serverless API and hosting a frontend with S3
- Part 5: Performance tuning for a Lambda-based API
