Migrating from nextjs 10 to 11 @smallcase

Migrating from nextjs 10 to 11 @smallcase

Featured on Hashnode

We had recently upgraded one of our codebases which uses Next.js with a custom webpack configuration from Next 10 to 11 @smallcase and as part of this upgrade, we also updated all the webpack related plugins and loaders to their latest version to make them compatible with webpack 5 which nextJS 11 comes with out of the box. So in this blog, I will be sharing some of the problems we faced while doing this upgrade and the solutions we used to solve them.

Next.js at smallcase

Before diving into the upgrade let's first talk about what our next.js setup looks like. A lot of our products at smallcase use next.js but in this blog, I will be specifically talking about smallcase.com. The next.js setup that we have is a little different from the traditional next.js apps because of the following reasons:

1. Support for dynamic routing

This codebase was started when the latest available version for next.js was v7. At this time next.js didn't have a lot of features that are currently available and one of the major features lacking at that time was dynamic routing (Eg: A dynamic route looks like: /blog/:id) which later then got introduced in next.js v9.

So at that time to support dynamic routing in our web app we used a combination of the next JS custom server and the next-routes package.

Note: The next-routes package has stopped receiving any updates from 2018 and is not recommended to use with the latest version of next.js. Instead, next.js provides this feature out of the box now (from version > 9) so one should use that. We also have plans to migrate away from this setup in the future.

2. transpiling modules from node_modules

Currently, no official support for transpiling modules from node_modules is provided by next.js. So to support this use case, we use the next-transpile-modules package.

To explain why we need this let's go through the diagram below to understand how we use components in our smallcase ecosystem.

local-test.png

  1. smallcase.com: Our B2C web platform
  2. Broker platform: Our multi-tenant B2B web platform for different brokers like Zerodha, HDFC, etc.
  3. Component library: A monorepo of shared components maintained using lerna.

We currently have two codebases that use a lot of common components so instead of writing the same component again and again and then maintaining them at two different places we instead create the common components in a different repo of its own. This repo is a monorepo where we create and maintain components in different packages with the help of lerna. So overall we have 2 repos that use common components from a 3rd repo.

We use two different ways for consuming the packages from the component library:

  1. local testing using symlinks: Before publishing the packages we sometimes want to test the package locally on the consumer codebases (i.e smallcase.com and Broker platform) to test their working and for this, we symlink the packages.
  2. Publishing on npm: When the package/component is completed we publish it to our private npm registry and then consume them like any other npm package available.

But before publishing the packages currently, we do not transpile them in our component library as we have a few packages which depend on getting some information from the consumers before transpiling (some variables which are required as input at build time to modify a few packages according to the consumer needs) and because of this reason currently we need to transpile this component library on the consumer codebases and due to this reason we need to use the next-transpile-modules package in our smallcase.com codebase.

3. CSS modules usage

For CSS we use css modules through the babel-plugin-react-css-modules npm package. This package makes it easier to use CSS modules by introducing a new prop called styleName which can be used on all components and it provides the following features:

  1. With CSS modules, we are forced to use the camelCase naming convention for naming our CSS classes which is not required here.

     /* css modules */
     .btnPrimary
    
     /* babel-plugin-react-css-modules */
     .btn-primary
    
  2. We don't need to refer to the styles object every time we use a CSS Module.

     /* css modules */
     import styles from './Button.css'
    
     <button className={styles.btnPrimary}>Click me</button>
    
     /* babel-plugin-react-css-modules */
     import './Button.css'
    
     <button styleName="btn-primary">Click me</button>
    
  3. A clear distinction between global CSS and CSS modules, e.g.

    <button className='global-css' styleName='local-module'>Click me</button>
    

Now to use this babel plugin for all our CSS files we need to:

  1. configure css-loader for which we have to use a custom webpack config through which we enable the use of CSS modules.
  2. The styleName prop handling is done by babel-plugin-react-css-modules for which we also have a custom babel config.

4. Static assets support

For supporting static assets like images and custom fonts we currently use file-loader and url-loader which are configured through a rule in our custom webpack config.

Why did we decide to upgrade?

As part of our engineering practices at smallcase, once every 2 quarters or so we analyze the different dependencies present in our project using tools like npm audit and npm outdated and figure out which dependencies have got new updates or any security problems and accordingly try to upgrade them whenever possible. We like to keep our dependencies upgraded to the latest version so that we can use the latest available features and also mitigate any possible security issues. It had also been quite some time since we had not updated a few of our old dependencies related to our custom webpack setup in this codebase, so this was a good opportunity for upgrading them.

next-11.jpeg

Also next.js 11 brings quite a lot of performance improvements and new features like:

  • Improved startup time: They have optimized their internal babel setup to reduce the startup time which will give us a faster developer experience which is definitely something every dev would love to have.
  • Webpack 5: With next.js 11, webpack 5 is now enabled by default which brings quite a lot of performance improvements with it.
  • next/script: A new component from next.js for automatically prioritizing the loading of third-party scripts to improve performance.
  • Conformance: Conformance is a system that has been open-sourced from Google's Web Platforms team. It is a system that provides a set of rules to support optimal loading and Core Web Vitals (in the future they also plan to add support for other quality aspects like security and accessibility). They provide these rules to next.js in form of an eslint-plugin and next.js 11 now comes with out-of-the-box support for eslint making it easier to catch common framework-specific issues during development and at build time.
  • next/image: Further improvements to the existing <Image /> component next.js provides which includes features like automatic image size detection and support for blur-up placeholders.

The Upgrade: Process, Problems faced, and their solutions

After going through the official migration guide from next JS for upgrading to nextJS 11, we first found out the dependencies we will need to upgrade:

dep.jpeg

1. React

  • react
  • react-dom

With next 11 the minimum required version from react is changed to v17.0.2. We didn't face any problems when upgrading react and react-dom from v16 to v17. No new features were included in this upgrade of react but there were a few breaking changes which can be checked out here.

With next 11, webpack 5 comes out of the box but as we are using a custom webpack configuration we needed to upgrade few more dependencies:

  • html-loader
  • mini-css-extract-plugin
  • css-loader
  • terser-webpack-plugin

Note: NextJS gives an option to opt-out of the default webpack 5 upgrade too and stick to webpack v4 but it is recommended to use webpack 5 as it offers these benefits and also with nextJS v12 webpack 4 support is dropped.

html-loader

No problems were faced when upgrading html-loader. There were quite a few breaking changes in v1 and v2 but we didn't have to make any changes in our codebase for this upgrade.

mini-css-extract-plugin

As part of the breaking changes in v1 and v2 of this plugin, a few options were removed but as we were not using those options there was no direct impact on our codebase.

One error that we saw when upgrading to v2.0.0 of this plugin was:

(0, _identifier.getUndoPath) is not a function

which was eventually found out that it was a problem within the plugin when more people started reporting the same error. This got fixed in version v2.4.1 of this plugin.

terser-webpack-plugin

For this plugin, a lot of plugin options were either deprecated or removed out of which the only thing that impacted us was the config for removing comments in the final build. For this we did the following change in our config:

// old config options
    new TerserPlugin({
        terserOptions: {
            comments: false,
            output: {
                comments: false,
            },
        },
    })

// new config options
    new TerserPlugin({
        extractComments: false,
        terserOptions: {
            format: {
                comments: false,
            },
        },
    }),

These options are how the terser plugin now recommends using if we want to remove all comments from the build.

css-loader

We were using a pretty old version of css-loader i.e v0.28.8 before the upgrade. v1 and v2 had a lot of breaking changes in it, where a lot of loader options were removed or replaced but we were not using them so that didn't affect us.

But when upgrading to v3, localIdentName option in the loader was removed in favor of modules.localIdentName so we had to do the following change in our loader config:

// old config options
{
    loader: 'css-loader',
    options: {
      ...
      // old syntax
      modules: true,
      localIdentName: '[name]__[local]__[hash:base64:5]',
    },
  }


// new config options
{
    loader: 'css-loader',
    options: {
      ...
      // new syntax
      modules: {
        localIdentName: '[name]__[local]__[hash:base64:5]',
      },
    },
  },

After that when we tried to upgrade the loader to v4 we started facing a few issues in our setup. This is what the home page of smallcase.com looked like after the upgrade:

Screenshot 2022-02-03 at 12.58.09 PM.png

After debugging the problem it was found out that the babel-plugin-react-css-modules plugin we are using is not compatible with css-loader which is causing this issue. The hash generated for the class names by css-loader and babel-plugin-react-css-modules plugin needs to be the same for the plugin to work but this was not happening after upgrading css-loader to v4 because of which the appropriate CSS was not getting applied to the components.

<!-- hash generated by babel-plugin-react-css-modules plugin --> 
<div class="HeaderContainer__container__2dEPY"> .... </div>
/* hash generated by css loader */ 
.HeaderContainer__container__ND0QF {
  flex-direction: column;
  position: fixed;
  left: 0;
  right: 0;
  top: 0;
}

As you can see from the code above different hash are getting generated for the same CSS class name because of which the CSS is not getting applied to the components.

It was then found out that the problem is with the babel-plugin-react-css-modules plugin:

css-loader made some changes in their hash generating algorithm so now babel-plugin-react-css-modules plugin also needs to update the same in their plugin. But turns out the last update this plugin received was on 11 May 2011 so it does not seem like this going to be fixed.

So for now we decided to stick with v3.6.0 of css-loader for the short term but in the long term, we will have to figure out a better solution here.


Few loaders and plugins that were replaced as part of the upgrade:

DependencyReplaced with
optimize-css-assets-wepback-plugincss-minimizer-webpack-plugin
file-loaderasset module provided by webpack 5
url-loaderasset module provided by webpack 5

optimize-css-assets-wepback-plugin

For webpack 5 and above it was recommended by the plugin to use css-mimizer-webpack-plugin instead of we replaced this plugin with that.

filer-loader and url-loader

With webpack 5, asset modules were introduced which come out of the box with webpack 5 and help to use assets files (images, fonts, etc) without installing any other additional loaders.

asset.jpeg

So, this was a good sign to let go of file-loader and url-loader and to welcome assets modules in our setup.

// before asset modules
{
      test: /\.(ttf|svg|png)$/,
      loader: 'url-loader?limit=50000',
}

// after asset modules
{
    test: /\.(ttf|svg|png)$/,
      type: 'asset',
      parser: {
        dataUrlCondition: {
         maxSize: 50000,
        },
     },
}

As it was recommended in the nextJS migration guide to upgrade all your webpack-related plugins and loaders to their latest version, we also upgraded postcss and its related plugins from v7 to v8 as part of this upgrade.

  • postcss
  • postcss-custom-media
  • postcss-custom-properties
  • postcss-flexbugs-fixes
  • postcss-import
  • postcss-loader
  • postcss-mixins
  • postcss-nested
  • postcss-simple-vars
  • autoprefixer
  • postcss-hexrgba

The only problem we faced was while upgrading the postcss-hexrgba dependency so I will be only talking about that in this section.

postcss-hexrgba

As the name of the plugin suggests, this was used to convert hex color values to rgb. We used this plugin as most of the global colors were defined in the hex format and stored as a CSS variable. Using this plugin we could use those hex color variables as rgba values too.

Example:

:root {
  --color-black: #00000;
}

/* In a component */
.card {
  /* here hexrgba plugin coverts hex to rgb */
  color: rgba(var(--color-black), 0.5);
}

But it was found out that this particular plugin was not actively maintained and due to that it was also incompatible with postcss version 8. Due to this reason, we had to remove this plugin from our setup.

With the removal of this plugin we also had to manually change the way we were using hex values inside rgba() throughout our codebase:

:root {
  --color-black: #00000;
}

/* rgb version of the same color */
:root {
 --color-black-rgb: 0, 0, 0;
}

/* In a component */
.card {
  color: rgba(var(--color-black-rgb), 0.5);
}
  • next
  • @next/bundle-analyzer
  • next-transpile-modules

The only problem we faced was while upgrading the next-transpile-modules dependency here so I will be only talking about that in this section.

next-transpile-modules

Before the upgrade, we were using a pretty old version of this package i.e v2.0.0 so when upgrading this package to v8.0.0 we had to go through a lot of breaking changes for which we had to make a few changes in the codebase.

  • In v3: transpileModule property was removed from the global config object in next.config.js. next-transpile-modules now exposes a function that takes an array of strings (the modules you want to transpile) as its first parameter. So for this, we had to change the way we pass module name to next transpile inside next.config.js.
// Inside next.config.js
// prev usage
const withTM = require('next-transpile-modules');

module.exports = withBundleAnalyzer(
  withTM({
    transpileModules: ['@smallcase'],
    // other config options
    ...
    }),
);

// new usage
const withTM = require('next-transpile-modules')(['@smallcase']);

module.exports = withBundleAnalyzer(
  withTM({
    // other config options
    ...
    }),
);
  • v5: A new module resolution strategy was introduced. Previously regex was used to find if a file should be transpiled or not. Now it has changed to use require.resolve instead. This now requires the module we want to transpile to have a compulsory main field in package.json. So now just mentioning ['@smallcase'] as module name will not work. To fix this we are now reading all scoped package names from package.json inside next.config.js and passing them to next-transpile-module.

The config from the above example was further modified to:

const packageJson = require('./package.json');

const { dependencies } = packageJson;
const smallcaseScopedPackages = Object.keys(dependencies).filter((str) =>
  str.startsWith('@smallcase'),
);

const withTM = require('next-transpile-modules')(smallcaseScopedPackages);

module.exports = withBundleAnalyzer(
  withTM({
    // other config options
    ...
    }),
);
  • In v4.1 a new option param was introduced called resolveSymlinks. Its value is set to true now by default. This causes a problem when we try to symlink a package from our component library to this codebase for local testing of the package:

  • So to solve this we need to make config.resolve.symlink = false in the next.config.js when trying to use a package locally i.e symlinking a package from our component library for local testing of the package.

The Result

After the upgrade, a few immediate benefits that we noticed were:

  1. The final build size got reduced

    We used to @next/bundle-analyzer plugin to check the before and after upgrade bundle size and we observed the following:

    • Client bundle: Around 18% drop in the bundle size
    • Server bundle: 54% drop in the bundle size
  2. The Cold start up time of the development sever reduced

    A significant drop (around 50%) in the cold start-up time was seen when trying to use the codebase in dev mode.

  3. Time to load a new page in dev mode also got reduced.

Overall we got what next.js promised us i.e a faster development experience.

Future plans

  1. Our major focus is to first move away from packages that are no longer receiving any updates:
    • Starting with next-routes first as next.js now already have supports dynamic routing out of the box.
    • We also need to think about how we are going to maintain using babel-plugin-react-css-modules with css-loader or maybe move away from it as css-loader keeps getting new updates whereas babel-plugin-react-css-modules has stopped receiving any updates, making them more and more incompatible.
  2. Start experimenting and using new features provided by next.js. With next.js 11 we got a lot of new features like conformance, next-script, and improvements in next-image which we would definitely like to try out and see what benefits we can get out of them.

References

  1. The official migration guide from next.js.
  2. Webpack 5 benefits with next.js 11
  3. Conformance for Next.js
  4. Webpack 4 to 5 migration guide