How to set up a Reverse Proxy with CloudFront, Lambda@Edge and Phoenix

My team at jobvalley and I ran into the problem that we had multiple services, which needed to be served from the same URL. We had one service running the Elixir + Phoenix stack that we considered our “main service”. This service should be accessible under an URL like https://example.com. We also had another service serving our content, which we wanted to integrate into the navigation bar of our main service. We wanted the content service to be accessible under e.g. https://example.com/content.

Setting up CloudFront and Lambda@Edge

We decided on using AWS CloudFront to route the requests to the appropriate services. In order to do that, we first needed to create two Origins, one forwarding any request to our main service and one to the content service. These were our settings for the Origin of our main service. The settings for the content service where exactly the same except for the Origin Domain Name and Origin ID.

In order to route the requests to the appropriate services, we set up two Behaviors. One behavior would forward any requests containing the /content path to the content service and the other requests would be forwarded to our main service by default. This were our settings for the /content behavior:

Next, we wanted to remove the /content-path from the request since the routes of our content service didn’t include the /contentpart of the path. For example, the post-route of our content service was available at /post/my-first-post and not /content/post/my-first-post. Therefore, we had to remove the /content part of the request URL. We decided on using a Lambda@Edge function for this.

First, we created a Node.js 12.x Lambda-Function “from scratch”. We needed to make sure that the function had all the right permissions in order to be triggered by the CloudFront-Behavior. Therefore, we used the Basic Lambda@Edge permissions (for CloudFront Trigger) Policy Template, which predefines all the necessary permissions.

We based the code for the function on this informative StackOverflow post. It uses the JavaScript replace() function to remove the /content-path.

'use strict'; exports.handler = (event, context, callback) => { const request = event.Records[0].cf.request; request.uri = request.uri.replace(/\/content/,'/'); return callback(null, request); };

This function retrieves the request object from the event, removes the /content part of the request uri and returns the updated request to CloudFront for further handling. Once we saved the code, we deployed the function Lambda@Edge. Under the menu “Actions”, we chose “Deploy to Lambda@Edge” and entered the following information:

After deploying the Lambda-function, CloudFront would roll out the new distribution to all instances within 5–10min. Once the roll-out succeeded, our services were accessible under https://example.com and https://example.com/content respectively.

Setting up Phoenix

After we accessed our content service, we had the problem that our static content including the .css and .js files wasn’t loaded properly anymore. The problem was that the URLs generated by Phoenix did not include the /content path. So, the URLs to our static files were e.g. https://example.com/js/app.js instead of https://example.com/content/js/app.js. Being the well-written framework that Phoenix is, changing how routes were generated took a single configuration in the Plug.Endpoint. We added the path: “/content” setting to our AppWeb.Endpoint configuration.

# config/prod.exs config :app, AppWeb.Endpoint, url: [ scheme: "https", host: "example.com", port: 443, path: "/content" ]

Once deployed, Phoenix would generate routes prefixed with the /content-path. Since the static_url setting falls back to the url setting, our static files were now also accessible under e.g. https://example.com/content/js/app.js . One caveat was though that we make sure to always use the conn instead of AppWeb.Endpoint when generating routes, e.g. Routes.index_path(@conn, :index) since otherwise the /content-prefix would not be included in the URL. Also, the Plug.Static setting in the endpoint.ex should not be changed since the static files must still be served under “/” and not “/content” .

Conclusion

Setting up a reverse proxy with CloudFront, Lambda@Edge and a Elixir + Phoenix stack was not straight forward and a lot of trial-and-error if you do it for the first time (and don’t feel like reading 1000 pages of AWS documentation). But once you’ve managed, you can offer all your services through a single domain and specify to which service traffic should be forwarded based on a path. Since we plan on adding more services in the future, we are positive that this solution will help us integrate the new services seamlessly.

(Photo by Caspar Camille Rubin on Unsplash)