Mixmax is a rich communications client integrated directly into Gmail using iframes. As such, we need to load all our resources over https - both for the security of our users, and to comply with Gmail's strict security policies. This creates issues during our normal development workflow, where we configure the app to refer to local resources.
Our platform is hosted across a set of microservices, each having a unique role. We have a microservice for serving contact autocomplete, rendering out the compose modal, displaying the dashboard, and more. Every service sits on its own fully-qualified domain name (FQDN) -
app.mixmax.com, respective to the aforementioned services. For development, however, we need to host our microservices locally. To maintain consistency and functional parity with our production environment, we put a proxy server in front of our microservices, which handles requests to specific domains, such as
app.mixmax.com, and forwards them to the appropriate server. This parity means we don't need to hard-code a microservice's port to make a request to it. Up until this point, unlike our production environment, our proxy only handled http connections.
A content security policy determines how the browser handles requests to origins different from the page's origin. A strict content security policy like Gmail's is intended to keep the user safe, in particular disallowing insecure requests on a secure page. When those resource requests are rejected, our app fails to load, which makes it impossible to test during development. Read more about content security policy on HTML5 ROCKS. Gmail's content security policy makes it so that if you inject content that uses insecure resources, the requests for those resources are rejected. From the Mozilla Developer Network:
When a user visits a page served over HTTPS, their connection with the web server is encrypted with TLS and is therefore safeguarded from sniffers and man-in-the-middle attacks. If the HTTPS page includes content retrieved through regular, cleartext HTTP, then the connection is only partially encrypted; the unencrypted content is accessible to sniffers and can be modified by man-in-the-middle attackers, so the connection is not safeguarded. When a web page exhibits this behavior, it is called a mixed content page.
As a result, we've had to run Chrome in an insecure mode when testing our changes. This introduces developer friction, and makes results from testing locally inconsistent with results post-deploy, so we'd like to mitigate this problem.
We'd like to share our findings while reducing friction in our development workflow, specifically regarding changes to our proxy server. We'll also discuss the changes we made to support our existing livereload mechanism. The straightforward solution entails enabling our local proxy server to serve content over https so that the content security policy allows our app to function.
Generating the Certificates
Any https connection requires a set of certificates, which serve to prove the identity of the organization or entity on the other end of the connection, and to encrypt the data between the two parties.
Actual certificates should not be used during this process - they are not necessary, and distributing them opens them up to unnecessary risk. Were any developer's computer compromised, the attacker could use the certificates to intercept legitimate traffic to the real servers, and cause real damage to both users and the company. Since we don't want to use real certificates, we'll need to generate our own. In our case, we have a number of different domains we'd like to handle, including those on two separate second-level domains:
To support arbitrary servers and minimize the number of changes we'll need to make if we add a new subdomain, we'll use wildcard certificates for
*.mixmaxusercontent.com. We'll also create a self-signed certificate authority with which to sign the certificates. Finally, we'll add the certificate as trusted to the root certificate store, which instructs Chrome to trust the certificate. Here's the script:
We run this script once for each developer, rather than create one set of self-signed wildcard certificates and check them in to git. This mitigates the admittedly small risk of the pre-shared certificates leaking, and keeps our developers marginally safer. To make installing the certificates easier, the above script also prompts the developer to add them as trusted certificates to the keychain:
It seems that Chrome doesn't acknowledge the certificate unless it's in the admin cert store, hence the
Configuring the Proxy
Our existing proxy server uses
http-proxy to actually proxy the individual requests to the microservices, and includes a short mapping of fully-qualified domain names (FQDNs) to local ports. This mapping mirrors our production setup with different FQDNs for each app server, and means we don't need to hardcode the ports for each microservice into our codebase.
The original proxy server:
It may seem as simple as replacing
https, but it's not. Node uses and maintains its own set of trusted certificates. As such, we can't simply make all server-to-server requests run over https without also making changes to every server-to-service request location. Moreover, we want to give our developers a chance to migrate their code - the parts that don't directly integrate with Gmail - so we'll keep the http proxy code.
The https proxy code will, similar to the http proxy code, need to map individual requests to the corresponding FQDN. Given the requirement that we support both
mixmaxusercontent.com, and given that we don't want to specify a port with each request, we'll need to use a TLS extension called Server Name Indication (SNI). With SNI, before the server responds with its certificate, the client signals the name of the server it wants to communicate with, and the server uses this to select the appropriate certificate. Node supports SNI out of the box in both the tls and https modules, which require a little effort to configure correctly.
Our app now loads fine, but our livereload doesn't work anymore. Livereload is super useful during development, so let's fix it. Chrome is rejecting the websocket connections due to Gmail's content security policy, which is causing mixed content issues. We're using
connect-livereload, and the gulp plugin provides its own development certificates. We haven't instructed Chrome to trust those certificates, so we'll need to feed the livereload connections through our proxy as well.
Each of our servers uses a different port for livereload. In the name of reducing the use of hardcoded ports, we'll use
<server>-livereload.mixmax.com to connect the proxy server to the upstream livereload server.
At the Proxy
On the proxy side, we already have a mechanism to handle different domains - namely, the
domainsToPorts map - so it makes sense to handle the livereload subdomains there:
That's all the proxy needs, because the existing code supports both loading the livereload script, and upgrading requests into websockets for livereload.
On the Client
On the client side, we can override the port by specifying the
port option to
connect-livereload, but we'd like to change the domain as well, which means we'll need to use the
src option in
There's a catch - by default, livereload will use the domain and protocol of the page, but uses the
port option and falls back to the port
35729. We'd like the
script to be served over
https on port
443, but we'd also like the subsequent connections to be over port
443. When reading the
src attribute of
script tags, Chrome omits the port if it corresponds to the protocol. Therefore, we can't just specify the port in the URL. The undocumented query parameter
port solves the problem by overriding the default port:
https://app-livereload.mixmax.com/livereload.js?snipver=1&port=443. Now the livereload script makes the correct request to the correct protocol/hostname/port tuple.
Now our developers can launch Chrome as normal, without needing to include flags to disable web security. We hope you've gained some insight into setting up your own secure proxy for development.
Interested in a low-friction development workflow? Come join us at Mixmax.