Blog Devlog: Newsletter Feature

If you scroll down to the bottom of this page, you may have noticed a new addition to the post in the form of a box that allows you to submit your email to get notified of new posts in this blog. I see this often in blogs that I personally read, and felt that it would be a good experience to learn how to integrate such a feature into my own DIY blog.

This post outlines the process of creating this feature for the blog in terms of design and implementation.

Foreword

First, it must be highlighted that this is not a serious attempt at starting some kind of marketing campaign of sorts, and the primary goal of creating this feature is meant as a learning exercise, and if there happens to be any readers or peers who wish to keep up with this blog, it would be a nice way for them to get notified. As such, there are 2 guidelines/principles I set for this feature:

1. Privacy: Any information (email) obtained from this newsletter is solely for the purpose of notification of new posts, and nothing more. It is not to be sold, misused or misappropriated in any way, shape or form.
2. Respect: The newsletter feature should not in any way impose on the user in such a way that degrades their experience on the blog (e.g. dark patterns such as modal pop-ups). Users can also unsubscribe to the newsletter at any time.

While it may seem obvious or simple, I believe laying out such guidelines explicitly are essential in setting boundaries and that we do not go down a slippery slope or end up unintentionally developing something that breaks our guidelines.

Lastly, this feature is not intended to handle scale; the traffic on this blog is generally low, and attempting to optimize for scale would take more time and resources ($). As such, I tried to keep to simple workflows that get the job done.

Planning

The first step in the process is to plan out what are the functionalities we will need. The high-level end-to-end flow of a newsletter feature can be summed up roughly like this:

• Users submit their emails to subscribe to the newsletter (Subscribe phase)
• When a new post is made, send out email to all subscribed users about the post (Notification phase)

The first part is fairly simple, but the second requires a bit more involvement. In addition, there is also frontend work involved in terms of the Newsletter component to be embedded onto the site. I decided to develop the functionalities in the following order: Frontend mockup -> Email storage -> Notification.

Frontend & Subscribe mechanism

For the frontend, there isn't a lot of technical exposition to be made, but it is important that the design follows the guidelines we highlighted above.

In terms of UX, it was paramount that we do not interrupt the user's experience in any way, such as forced modals or forced scrolling. As such, I have tried to make it as simple and non-imposing as possible through a simple box, text and input which will be placed at the end of every post. Other details include a loading animation when users are subscribing to provide feedback and weight that their actions have triggered the subscription process.

In order to send emails to subscribers, we have to know who they are, which means that we have to store the values somewhere. For this purpose, I decided to employ AWS DynamoDB which allows us to store key-values in database tables. In this particular case, we do not even need a value, we use the emails as the key and the sole column of the table, which will be called subscribers. This way, it prevents duplicate submissions of emails. When we need to send emails later, this database of emails will be read.

To access this DB from the frontend blog application, we create an API endpoint /api/subscribe which receives an HTTP request from the frontend with the user's email. The application's server then sends a request to DynamoDB to add a new entry to the subscribers table.

Notification Mechanism

In order for us to inform subscribers whenever there is a new post, we need to think about how we are going to implement the trigger. In this case, the trigger is that a new post is published, but how does that work? This blog uses Vercel's Git integration for deployment, so that whenever a new commit is made on the main branch, a new production deployment (www.kkmin.net) is made by Vercel.

All posts are stored in a directory in the form of .mdx files which are then converted into HTML pages using Nextjs's MDX Plugin. You can check out the post I made in the past to see what MDX can do.

My workflow for creating a new post consists of creating and editing a new .mdx on a separate branch, and merging it into main once I am done with the post. Therefore, our trigger for notifying subscribers have to be when new commits are made to main, and there is a new post in the posts directory. (Note that we do not want to notify subscribers when there are commits but with no new posts, e.g. modifying posts or adding other features)

The flow of this Github action looks like this: Check if new post is made -> Get subscriber emails from DynamoDB -> Send notifications to emails. This whole process is implemented inside a script (Publish script) which the Github action runs.

To check whether a new post has been made or not, we first create a DynamoDB table posts_metadata that stores two items: post_count and latest_post which stores the number of posts and the title of the latest post that is live in production currently. We then create a Github action that runs on commits to main which checks the post_count and latest_post inside the directory against the DB values. If the codebase's post_count is 1 more than the DB's post_count and hte latest_post values are different from the codebase and the DB, then we can take it that a new post has been made in this commit, and we need to send notifications.

To send notifications, SMTP2Go was used to link our domain (kkmin.net) with their API, which allows us to send emails via API requests. Without going too much into the details, I created an email template in their webpage, and call thier API to send emails to a user with that template.

Closing Thoughts

This was a really engaging process which challenged me in terms of using different services/platforms to create 1 coherent feature; while it looks simple in retrospect, trying to make sense of what systems/components to use and synergize when there is only a blank slate in front of you was not a trivial task.

Based on my past experience, enterprise systems are also similar in this regard that there's many different systems and services that need to come together, albeit more complex than this; working through the entire end-to-end flow has given me a better appreciaton of the work that happens inside realworld enterprise systems for sure.