I just shut down
www.bookfetch.co.nz after the maintenance burden became too big.
Bookfetch was always supposed to run with minimal intervention, so reducing maintenance cost was a big guiding principle in the development I did. But despite my best intentions, so many things still needed fixing and updating along the way.
Here’s the maintenance log for my zero-maintenance webapp.
It’s a price comparison website: you tell it a book, and it tells you how much that book costs at different retailers. The site get book details and prices from a combination of scraping and official APIs.
The site is hosted on Google App Engine, a hosting service that I don’t have to administer. The hope is that the managed App Engine platform will mean I don’t have to do any server maintenance.
Mere days after I launch Google deprecates the datastore library I’m using. I dutifully rewrite Bookfetch’s models to use the new
ndb library. A sign of things to come.
…breaking my scraper. A few months later I give up on trying to keep scrapers up to date, instead just removing the retailer if its scraper fails. There are enough retailers with stable APIs to keep the site useful.
This affects the book details API, price API, and affiliate link format.
Over a couple of years, the types of websites that need encryption has gone from “just banks” to “all of them”.
In 2014 there are are two options for free SSL certificates: CloudFlare and StartSSL. I
randomly wisely choose CloudFlare, which is just as well because in 2016 all major browsers stop trusting StartSSL certificates after they are found guilty of shenanigans.
After treading unknowingly on thin ice for two years, I learn why you aren’t supposed to keep a git repository in Dropbox. I can’t remember exactly what happened or how I fixed it. I do know there was a lot of swearing in my git commit log around that time.
The App Engine SDK – used for uploading code and running a development server – is retired in favour of the Google Cloud SDK. The new SDK has a command for uploading code but not for running a development server.
The documentation for running a dev sever appears a few week after the old SDK download link is removed.
…causing rampant numerical issues.
I don’t notice for months due to strong caching and weak logging. Move to new provider.
Amazon limits usage of its Product API based on the number of sales driven by your account. I don’t refer enough sales so I get errors on the API, which is used for the Bookfetch book search.
With the Amazon book details API failing, I only now realise that the Google book details API changed years ago to requiring authentication, and the Book Depository API had been offline since the UK merger in 2014. I didn’t notice at the time because I’d designed book details fetching to be Zero-Maintenance: if one API failed I just tried the next one. Well now all three were broken.
Book search no longer works.
Of the 30 retailers I set up in the beginning, only 5 are left. All but 2 pulled their affiliate program over the last 8 years.
It was sad to finally pull the plug. Bookfetch taught me webdev. It landed me my first internship and my first job. It had long ceased to be profitable on an hour-spent basis, but the Amazon API change was the last straw.
I’ll miss you Bookfetch.