SSL Tracking Links on Mailgun with Ghost Blog
Last week I blogged about Ghost v4 and the upgrade to it. It has put front and center the notion of independent professional publishing. With that release came the ability to send email subscriptions of blog posts for those subscribed.
Behind the scenes it uses Mailgun to add some enhanced features like:
- Approximate location of subscriber
- Detect whether email was opened
- Detect whether email was interacted with (clicked on)
How does Mailgun even do this? Lets stop with a quick lesson on email tracking.
1px images
<img width="1px" height="1px" alt="" src="http://email.mg.connortumbleson.com/o/[redacted]">The first trick is embedding a specific query string image into the email. It will probably be a 1 pixel by 1 pixel transparent image and you won't even know its there. Your email browser behind the scene will download that image as instructed.
If you are using Gmail they actually pre-download images from their servers and show them to you. This has the benefit of preventing the email from knowing if you interacted with the email as they only see Google viewing the image.
So if you pop open the email on your desktop and your computer reaches out to Mailgun to download that tiny image. Mailgun sees your web request and that specific image being downloaded. It can then obtain quite a bit of information.
{
"geolocation": {
"city": "Tampa",
"region": "FL",
"country": "US"
},
"tags": [
"bulk-email"
],
"timestamp": 1616411499.964054,
"log-level": "info",
"event": "opened",
"campaigns": [],
"user-variables": {
"email-id": "[redacted]"
},
"recipient-domain": "gmail.com",
"ip": "47.xxx.xxx.xxx",
"client-info": {
"client-name": "Thunderbird",
"client-type": "email client",
"user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0",
"device-type": "desktop",
"client-os": "Linux"
},
"message": {
"headers": {
"message-id": "20210322023002.1.2A977408ED7C81D8@mg.connortumbleson.com"
}
},
"recipient": "[redacted]@gmail.com",
"id": "[redacted]"
}So you can see it knows my Thunderbird client downloaded this image and approximated the location to Tampa Florida. So Ghost can easily API this information out to see where your subscribers approximate location is.
So now how does Ghost know if you clicked on a link in the email? Well that is pretty easy in the scheme of things. Mailgun simply rewrites all links in the email to Mailgun specific links.
So if I had a link to my blog, Mailgun would translate it to:
http://email.mg.connortumbleson.com/c/{randomchars}This subdomain is a CNAME to mailgun.org as instructed by Mailgun during sign up. So when that request hits Mailgun, much like the image trick, the string of characters at the end of URL makes perfect sense to Mailgun to preform a redirect.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="https://connortumbleson.com">https://connortumbleson.com</a>. If not click the link.If clicked that link that will redirect back to my intended location probably without myself even noticing, but that middle jump resulted in Mailgun being able to track that click.
{
"geolocation": {
"city": "Tampa",
"region": "FL",
"country": "US"
},
"tags": [
"bulk-email"
],
"url": "https://connortumbleson.com",
"timestamp": 1616582042.542381,
"log-level": "info",
"event": "clicked",
"campaigns": [],
"user-variables": {},
"recipient-domain": "gmail.com",
"ip": "47.xxx.xxx.xxx",
"client-info": {
"client-name": "cURL",
"client-type": "library",
"user-agent": "curl/7.47.0",
"device-type": "unknown",
"client-os": "unknown"
},
"message": {
"headers": {
"message-id": "[redacted]@mg.connortumbleson.com"
}
},
"recipient": "[redacted]@gmail.com",
"id": "[redacted]"
}So boom - Ghost blog can see who opens and clicks on their emails which leads to all the cool graphics/images they show in this help post.
However, when I enabled this originally. This is what subscribers saw when they clicked on a link.
This wasn't good as subscribers getting an invalid link hurts the experience so I disabled the open/link tracking while I investigated.
It didn't take long to realize that HSTS was the culprit. I saw this header in the response while debugging.
Strict-Transport-Security: max-age=63072000; includeSubdomains; preloadWhat this means to browsers is my domain should never be accessible in a non-secure manner as well as any subdomain I have and this should last 2 years at a minimum. This is a protection against downgrade attacks and just ensuring that a secure transport mechanism is always used.
So email.mg.connortumbleson.com is a CNAME for mailgun.org, which is just an alias for a domain. This means that the SSL would have to be setup by Mailgun and I don't think Mailgun is in the market to setup SSL certifications for their clients.
Mailgun has a help guide for this and instructs you to use Cloudflare to set this up. However, we don't use Cloudflare here and not going to use it for one subdomain. So lets do this ourselves.
First off, lets remove the CNAME we previously had to Mailgun. We can't mix these type of records as we will be pointing our own A record to our server.
We know we basically have to setup that domain to proxy requests to Mailgun, so the request can remain secure to the end user, but we are communicating as the server behind the scenes insecurely to Mailgun.
So we will setup a simple nginx block for this.
server {
listen 80;
server_name email.mg.connortumbleson.com;
location ^~ /.well-known/acme-challenge/ {
allow all;
}
location / {
resolver 1.1.1.1 8.8.8.8 valid=300s;
set $mailgun "http://mailgun.org";
proxy_pass $mailgun;
port_in_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
We are basically doing two things here. We are only setting up an insecure block first as we will leverage LetsEncrypt for securing it. We want to exclude the verification block that LetsEncrypt uses so we don't redirect the verification calls to Mailgun.
Now the proxy to Mailgun required a bit of a trick. The IP for Mailgun seemed to change quite heavily and I didn't know if there was a constant IP to hit. So we will query for the IP using a 5 minute cache from the popular DNS resolver of Google and Cloudflare.
Now anytime we request a URL to email.mg.connortumbleson.com that will be passed behind the scenes to mailgun.org, much like the previous CNAME did. So now we just have to secure this domain with a certificate.
root@dune:/etc/nginx# certbot -d email.mg.connortumbleson.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for email.mg.connortumbleson.com
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/email.mg.connortumbleson.com.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Congratulations! You have successfully enabled
https://email.mg.connortumbleson.comThis server already had certbot configured for renewals and crons so this is now supported. All we had left to do was wait for a few hours for all these DNS changes to propagate.
So now we are in business - a secure tracking link through Mailgun ourselves. Just one support ticket to Mailgun to enable SSL for us and we are done.
Technically, we don't need to ask Mailgun since the HSTS automatically upgrades non-secure to SSL, but I don't want to rely on that.
➜ curl https://email.mg.connortumbleson.com/c/[redacted] -i
HTTP/1.1 302 FOUND