BenchSci’s flagship product is AI-Assisted Reagent Selection, a web application that allows scientists to search the body of knowledge concerning various types of reagents and model systems (i.e. the ingredients used in their pharmaceutical experiments). The first version of this application was written in 2015 and, like most pieces of software, has gone through various iterations by various developers over the years. This app in particular was a JavaScript Web Application that was built at a time when React was relatively new and TypeScript was just gaining popularity. To give you an idea of what we were dealing with, here were some of the major features:
Regardless of its slightly older, convoluted software, the app has served the scientists using BenchSci well. Regular updates are deployed to over 50,000 users from 16 of the top 20 pharmaceutical companies who routinely log in to search and select reagent and model system products that will work best for them. Yet, our company’s goal is to bring novel medicines to patients 50% faster, so we are building new applications to further aid scientists in their quest.
When we began developing new apps, we had a new crop of developers with new ideas and a more modern arsenal of technology to draw from, like Next.js and TypeScript. This left us with two very different frontend applications: our legacy Reagent Selection app and our shiny new Next.js-supported one. Users would unknowingly flip back and forth between the two apps as they needed. The experience was seamless because we used a custom Express server to determine which application bundle was served to the user, based on the URL of their initial request. Links buried inside one React app were clever enough to know when to reach out and boot up the other application if the user navigated from one app to the other. Still, a loading screen was sometimes seen between link clicks as each application needed time to boot up.
That’s why toward the end of 2021 we decided it was time to update our legacy app to make use of the latest breakthroughs in the JavaScript ecosystem. We sought to combine our old and new applications into one platform. Namely, we were drawn to these upgrades:
And so at the start of 2022, we began the great move of our flagship JavaScript application over to Next.js and TypeScript.
Any developer at BenchSci will tell you that the impact your changes make to the end-user is paramount. To that effect, we pondered whether we should repeat taking one vertical slice of the Reagent Selection app and moving it over to Next.js, or should we move the entire application over in one giant move? The former would give our users a taste of the new world sooner but it also posed some risks.
Our old JavaScript application was complex. It had years of Redux Sagas built into it, most of which call other Redux Sagas and Selectors, making the code quite tangled and hard to reason about. It is challenging to pull a single page out of the application, taking all its sagas and selectors with it, without affecting some of the pages left behind.
In addition, we still had two frontend teams that were actively working on features for the legacy JavaScript application. If we pulled out a single page at a time, they might be left with feature plans that straddle two different applications. Even worse, our users would likely experience application loading screens in the middle of their workflows if some of their pages had been migrated while others had not.
Given these truths, we elected to first migrate all our legacy JavaScript code into the Next.js application in one move, then update it to TypeScript and functional components. In the process, we would remove the Redux Sagas and Selectors in favor of a leaner store that made use of the React Query library for its API calls. This would give us the advantage of all our developers working in the same codebase, so we could leverage their efforts in helping with the migration. It would also remove any “Frankenstein” experience for our users—no one would have to straddle their workflows across two application loads and the experience would be smoother overall.
Before making the big move, we had to prepare both the old code for its trip and the new destination framework for its impending new occupant. Our goal in this first step was to make as few changes to the JavaScript codebase as possible. Any changes that needed to be made were hopefully done in our main code branch and deployed to Production before the move began.
But some of the changes we needed to make to the old codebase only made sense once it was living inside the Next.js framework. So we began to make lists of what those would be, including:
Some of our new developers, who worked on the Next.js application, raised concerns that we were going to add unoptimized JavaScript code to their clean TypeScript ecosystem. The Next.js project had stricter ES Linting rules and despite trying to auto-fix our old code base with new ES Linting rules, there were too many places where the auto-fix couldn’t help us. We also knew that the moving of code was simply Phase 1 of our project. After we had merged the applications, we would begin migrating JavaScript to TypeScript.
To help us measure our efforts here and separate the inferior JavaScript code from the pristine TypeScript code, we elected to move our entire old application into a rather obtuse folder inside Next.js: /src/reagent-legacy/src/. This ugly folder path (with its offending dash and double mention of “src”) serves to remind us that this is just temporary housing for this code. It was also a folder designated as an exemption from the strict ES Lint rules of Next.js. Eventually, in Phase 2, the contents of this folder will be converted to TypeScript, properly ES Linted, and moved to its final resting place among our other components. We can measure our success in Phase 2 by reducing the number of files in this ugly folder to zero, after which we promised ourselves a party!
We practiced our move in a separate branch. We started by making bash scripts that did the move (and patched up the code) for us. We’d perform the move, run our tests, see what failed, and strategize how to best fix things. Then we’d go back to patching things up in the main branch and redo our scripted move. Eventually, once we had made all the changes to the main branch that we could, we realized we’d need a long-running git branch to contain our moved app and all our Next.js updates until all our tests were passing.
As we still had developers both in Next.js and the legacy JavaScript application adding and editing features, we had to incorporate their changes into our long-running merge branch. But how would we know about changes in a legacy application if we had moved those files over to Next.js in our application? The life-saving command here was “git mv.” Instead of naively copy-pasting code from our legacy application into our Next.js, we used “git mv” to bring the code over. This allows git to know that changes in a file in one location are meant to be changes applied to a file in a different location. But there’s an important “gotcha” when using this tool. You cannot make any changes to the code in the commit that does the “git mv” operation. It’s imperative that git labels your “git mv” as a file rename and not one file being deleted while another is added. The latter happens if you make changes to the code in the same commit. In the end, we squashed the commits in our long-running branch down to:
To keep our long-running branch up-to-date with our main branch, we would rebase it off our main branch, thus incorporating any changes that our other developers had made to either application. We aimed to do this every day, but often neglected this step, making each rebase more difficult as the change logs grew larger. Rebases would often contain conflicts, thankfully mostly in ES6 import statements pointing to the old folder structure. If there were any logical conflicts, we almost always chose the changes from the main branch as our goal was to not create logical changes in our migration efforts. We used the Git Graph plugin inside VS Code to help ease the burden of rebasing and to visualize where each branch was in our source control ecosystem.
Eventually, we arrived at a point where all the tests in our long-running branch were passing, and our manual QA efforts found no bugs. The week before our official move day, we held an informal meeting to work through any concerns our developers might have after the move. We knew that the open work branches (those that had not yet been merged back into our main branch) would be confused about the state of the world after the move. We found that the easiest solution was to start a new branch once the merge had happened and “git cherry-pick” changes from an old working branch to a new branch. Our team preferred this over attempting to rebase a working branch on top of the newly merged app.
On the day of our move, we only did a code freeze for about an hour, the length of time it took us to merge our long-running branch into our main branch and do a sanity check on our QA server to see our application running under a single framework.
After a successful move, we’re settling into our new Next.js digs
Today we have everything under the banner of Next.js. Although our users cannot visually tell the difference, they’re no longer downloading a giant app bundle and instead are consuming only the code they need to complete their task. After running some tests, we discovered that our time-to-interactive metric, a measure of the app’s load time, has gone from over seven seconds to four seconds—a 41.6% improvement in our app’s loading speed. Our codebase still has lots of old JavaScript in it, but it’s sectioned off and our legacy frontend teams are excited to begin the job of converting their old components into a fresh set that is ready for the modern TypeScript landscape. We’re excited to migrate to React Query, to lose the complexity of Redux Saga, and gain all that Next.js has to offer.
It’s going to be a bold and wonderful new world for BenchSci's engineers. We’re not at the completion of our journey but we can see the light at the end of the tunnel and we’re excited to reap its rewards!
--
We’re hiring! If you’re interested in building a great career in engineering, check out our Careers page. And subscribe to our blog to stay up to date with all things BenchSci.