This post follows up on the previous post: Migrating My Personal Website to Confluence Cloud, Transformed by AWS. As mentioned in the previous post, the current implementation did have a few limitations:
The next step would be to address these limitations.
Confluence Cloud offers a range of useful APIs https://developer.atlassian.com/cloud/confluence/rest/, but of particular interest to me would be the “Search Content by CQL”, as well as “Get Content”. With these APIs, it would be possible to extract published content from Confluence itself, and apply a much more customised presentation.
A two step approach would be adopted:
It would be worthwhile to explain the content strategy I had in mind. At the moment, I had identified three content types:
Confluence supports page and blog posts. While it would be possible to leverage on these content types, it would still be lacking a way to differentiate between embedded content and full fledge pages.
A much more flexible approach was to leverage on Confluence labels. There would be three pre-defined labels, each mapping to a type of content (content_type_blog, content_type_page, content_type_embed). This gives me flexibility in the future to add new content types as well.
On top of content type, I also needed some mechanism to tag metadata to each content. Confluence supports excerpt and page properties macro out of the box, and it would be most logical to leverage on these two.
I defined the following additional page properties:
With the content type template defined, the next step would be to figure out how to style them. These would be the guidelines and principles I had established:
My choice of presentation technology was ReactJS, quite simply because I wanted to learn more about it, with no other particular reason. For the current iteration, only ReactJS on the browser would be explored and implemented.
My thoughts after implementing ReactJS were:
At the moment, the following content type templates / pages were identified:
This would be a high-level flow of how a page of this website would be loaded. All other pages would follow a similar flow.
The final version had the API Gateway performed the content merging with the content template. This had not been the original plan, which was to leverage on S3 events and Lambda triggers to transform downloaded content into another bucket to improve website efficiency. The drawback of that was template changes required re-generation of all impacted pages. That proved to be too huge an operational overhead, and so the idea was dropped.
Note: Take note that in order for CloudFront to front an API Gateway, CORS must be implemented! API Gateway custom domains were not necessary.
I was handling the error page rendering initially within the Lambda functions, until I realised that CloudFront could associate HTTP error codes with a custom error page from my S3 Bucket. This was a much cleaner way to manage error page rendering.
All content served were extracted from Confluence and cached into the following:
It was not trivial trying to use DynamoDB, especially if one had came from a traditional SQL background. Figuring out queries, scans, index and projections took a while. Updating existing data was a pain as well due to the asynchronous nature of using the API. A single content update required me to delete data from 3 tables and then adding it back. It did not give us the a ability to do cascading deletes. Definitely not using DynamoDB in the right way.
In additional, each new DynamoDB tables/indexes and Lambda added some CloudWatch metrics and alarms, which quickly went beyond the free tier allowance. After going back to review the default metrics, I made a conscious decision to simply eliminate all of them. They might be useful in a real world production operations, but not for a personal website.
The current setup would be such that the extraction happens via a nodejs script from my local laptop. In the future, this could be migrated to AWS Lambda + API Gateway + Cognito for a more online or maybe even automated content publishing experience.
Putting all the pieces together, this was the final architecture.
The final architecture leveraged heavily on AWS free tiers, specifically:
At the moment, the only AWS bill that came was a 0.12 USD bill on AWS Lambda@Edge (there was no free tier for this!!!!). All else were free. So on top of the monthly Confluence Cloud subscription, my website cost did went down significantly (could be lower if I tried to host my own Confluence, either in my home and publish to AWS, or spin up a t2.micro EC2 instance and run Confluence when I needed to).
The website was far from finished. There were additional features I would like to put in place, but for now I had explored whatever I could on AWS with this particular ‘project’. It offers an extremely low cost option to hosting a website, and I would definitely be coming up with other projects to learn more AWS technologies. The focus would of course be for free tiers, specifically:
But for now, it had been a fun learning experiment and experience.