« Scott Nonnenberg


The great gatsby upgrade

2021 Oct 17

[Fourth in a series of posts about GatsbyJS. All three initial posts have been updated: 1. Introduction and The SPA Calculation. 2. Tips for productionizing. 3. First notes on this blog’s source code.]

In the past year I’ve upgraded this blog’s version of Gatsby not just once, but twice. And wow, it has gotten so much better. And so has my blog. Fork it and play around! Or, if you’re not yet convinced, read on…

A powerful architecture

I could talk about any number of compelling Gatsby features, but the real game-changer for Gatsby is its deep extensibility. In the abstract, it is a system that surfaces data via self-documenting, type-safe GraphQL to consuming code, with many built-in primitives for assembling the resultant high-performance web application.

It really is that abstract, because you need to add ‘transformers’ to Gatsby project to supply the data - there’s nothing to start! For example, this blog uses gatsby-transformer-filesystem to pull files from disk, gatsby-transformer-remark to turn any markdown files found into HTML, and gatsby-transformer-sharp to process large images into various responsive sizes.

With that data, my code then generates pages and other assets like RSS files. But not before tweaking the GraphQL a little as an optimization - a perfect sample of the deep extensibility available in the system.

The ecosystem

A common pitfall of highly-customizable systems is that they are not accessible to the beginner. The most basic ‘hello world’ requires too many decisions, and the hunt through all the settings can be very lengthy.

Enter gatsby plugins. Experts can tackle that comprehensive extensibility and package it up for everyone else to use! Some plugins are very simple, and some are quite complex indeed.

The gatsby site reports over 2800 searchable plugins as of this writing. It was plenty for me! When updating my blog, things I had to do manually before became as simple as a search, then adding the plugin. No need to manually add and integrate catch-links - just add gatsby-plugin-catch-links to your gatsby-config.js!

The struggle

I won’t lie to you. Despite these improvements, or perhaps because of them, it was not an easy process upgrading from 0.12.7 to 3.13.0. Gatsby had changed a whole lot! I kept it simple: my goal for the upgrade was to keep the user experience exactly the same. No redesign rabbit holes. But it was still a major effort.

I had big plans in May 2020 of using quarantine time to upgrade to Gatsby 2. I got pretty far, but then stalled out due to Storybook/Gatsby incompatibilities. And, of course, the other challenges inherent in trying to establish new routines in the face of sweeping societal change. I bought a rowing machine and played a lot of Magic The Gathering Arena (more about both of these in future posts).

I came back in November of that year, started gaining momentum during the course of that month, and finished it all off the week of Thanksgiving. I made almost 60 commits that month! My first successful Vercel deployment was November 23rd 2020. Then there was another burst of activity in December to improve performance and turn on Javascript in a branch.

Now, as fall starts, almost a year later, I just upgraded to Gatsby 3.13.0.

Why Gatsby?

I first started using Gatsby in January 2016, and here we are in 2021. This is the longest I’ve stayed with one blogging platform. It had value for me as a React developer back then, but now it has a large collection of value propositions:

But I couldn’t stop here. I had to go deeper to really make it mine.

My customizations

Things have changed a lot since late 2016, the last time I made a large change to the technology of my blog. The industry has changed. I’ve changed. I now have a new set of requirements for code I’ll be living with for a while.

You can see the full set of recent changes in the project’s change log. These are just the highlights!

A contentious customization

I’ve previously spoken about my decision to disable Javascript on my blog. And this time around I did a good amount of work to maintain that decision in the face of a substantial amount of Gatsby change.

But it’s not just about the theory - let’s allow for the visceral sense of actually navigating with Gatsby’s Javascript and without!

This blog has no Javascript (probably where you’re reading this): https://blog.scottnonnenberg.com

And you can try out the Javascript-enabled version of this blog here, right now: https://blog-js.scottnonnenberg.com

What do you think?

Giving Javascript a chance

The Javascript-enabled site is pretty fast, and most people tell me that they can’t tell the difference between Javascript enabled and not. Funny enough, that says to me that downloading all that extra Javascript probably isn’t worth it!

I really did give it a good-faith effort, though. I did a bunch of work to take the download size down, both the bundle sizes and the page-data.json files.

The very first thing I did was install the excellent gatsby-plugin-webpack-bundle-analyser-v2 gatsby plugin to help me zero in on problem areas. The obvious next step was to remove lodash, moment, and underscore.string from the bundle, implementing myself the few functions I needed.

At that point it was time to look at what the browser was downloading. Spending time in the Network tab of the Chrome Dev Tools, I noticed that the page-data.json files for pages like index, popular and tags were quite large. Again, pretty obvious when I dug in - the React components shouldn’t be interacting with HTML at all, because that means all the HTML, for all the pages, has to be shipped to the client!

Now it started getting complex. I needed to think like I was writing a gatsby-transformer-xxx plugin - how might I surface just the information needed for each of my components? And the answer, like everything in Gatsby, is “expose it via GraphQL!” This is where the createResolvers Gatsby API comes in:

  createResolvers: ({ createResolvers }: CreateResolversArgs): any => {
    const resolvers = {
      MarkdownRemark: {
        htmlPreview: {
          type: 'String',
          resolve: async (source: PostType, args: any, context: any, info: any) => {
            const htmlField = info.schema.getType('MarkdownRemark').getFields()['html'];
            const html = await htmlField.resolve(source, args, context, info);

            const slug = source?.frontmatter?.path;
            if (!slug) {
              throw new Error(`source was missing path: ${JSON.stringify(source)}`);
            }
            return getHTMLPreview(html, slug);
          },
        },

This Typescript adds a new GraphQL field htmlPreview alongside the html field gatsby-transformer-remark already provides for our markdown files. The key tricky bit is that info.schema.getType and ultimate resolve() call - with those tortured lines we’re querying that sibling html field in the GraphQL to get what we need to construct our custom result. The full change adds another textPreview field and updates the consuming GraphQL queries.

It was a really big step forward, and I was finally starting to get comfortable in this environment!

I made a couple more commits to further slim down my React components and I was set. A few principles:

It’s a far cry from the Gatsby of old, putting all of your posts in that single bundle. But it’s still not good enough for my use case. I think the exercise was useful, despite that. And just maybe you’ll find my blog’s /js branch a better starting point!

Turning off Javascript

My very first steps towards the Gatsby 2.0 upgrade included gatsby-plugin-no-javascript, because I wasn’t about to have a core behavior of my blog go away with this so-called upgrade! It was quick and easy to install. At least the spirit of my original noProductionJavascript option was still around!

Once I had everything basically working post-upgrade, I started looking at the site more closely. I quickly discovered an errant 85kb Javascript polyfill file still included in my built pages. Disappointed that my plugin missed this, I pulled out one of my favorite quick-and-dirty tools: patch-package. It shortens the fork/fix feedback loop into ‘change the node module on disk and run patch-package’ - no git URLs in your package.json, no new fork in your GitHub profile, no fuss. I eliminated the unnecessary download with a quick patch.

From there I continued to find things that my original no-Javascript plugin wasn’t eliminating. I continued to patch, but I also found another highly-useful plugin: gatsby-plugin-no-javascript-utils. It allowed me to drop inline styles for a separate CSS file I could include from all of my pages, caching aggressively. Now I was really in business!

Later, I decided to source the author image at the bottom of my pages from local assets instead of a separate CDN domain, and in so doing give it the same treatment all other images had on the site: a base64-encoded thumbnail, different sizes available for different screens, etc. Enter gatsby-image. It was the recommended way to make images responsive, so I jumped in. But then I realized that it didn’t work without Javascript! It put some code into a <noscript> tag, so it was ready for browsers with Javascript disabled, but not site builders excluding it entirely! I was shocked, because my images in my markdown-generated posts had worked so well, Javascript or no. Exit gatsby-image.

But gatsby-image didn’t go far, really. As you do in this kind of situation, I replicated much of it for my author image. This is probably where I felt the most like an outsider in the Gatsby ecosystem, doing something the system wasn’t really meant to do. But it works!

The High and the Low

I definitely had my high points and low points during the process of refreshing my Gatsby knowledge and modernizing my blog.

The good:

  • The ecosystem is great - you can find plugins that do most things, or at least good inspirations.
  • GraphQL is powerful, typed, and self-documenting.
  • The modern Gatsby API gives you a lot of extension points. I love that I can easily generate RSS/Atom in the build step without extra scripts.

The bad:

  • It was a little difficult to turn off Javascript - but at least this time I didn’t have to pull request Gatsby itself!
  • Storybook was quite difficult to set up when I first tried!
  • Typescript still isn’t supported perfectly - I want a gatsby-config.ts!
  • Major version upgrades are still painful, even from 2.x to 3.x.
  • There were some unfortunate breaking changes:
    • With a new markdown parser came new sensitivities to certain nested formatting operators.
    • With a new plugin providing header anchor links, special characters were processed differently. Some deep links into my posts were broken.
    • My blog previously used Typography.js, but it hasn’t been updated lately and its API had changed too radically. So I ejected, using its generated CSS but none of its functionality.

Fork away!

I’ve done a bunch of work to modernize my blog, and I like working with it much more than I used to. It’s been a rewarding effort.

But it’s not just about me. With my readme and inline comments, it could be useful for you too! Fork it and add new storybook stories and React components! Start with the /js branch and add interactivity to the pages! Use it to express yourself, then deploy it to the world!

I won't share your email with anyone. See previous emails.

NEXT:

Better web development and deployment 2021 Oct 31

In the last four years I’ve really improved how I develop and deploy web applications. There’s a new set of tools I don’t leave home without! Let’s talk about what’s changed, and more importantly... Read more »

PREVIOUS:

A holistic health checkin 2017 May 31

It’s been a while since I last talked about nutrition, fitness or health. I think it’s time. Where before my articles about this have been focused on one aspect of health, this article will cover... Read more »


Hi, I'm Scott. I've written both server and client code in many languages for many employers and clients. I've also got a bit of an unusual perspective, since I've spent time in roles outside the pure 'software developer.' You can find me on Mastodon.