Profiles of Innovation: Why Typescript?
Some changes in software engineering happen quickly that you expect to take a long time. Our mongo to postgres migration is one example where we thought it would take a quarterand it took just a few weeks despite the ginormous codebase. Our transition from ES6 to Typescript was a similar endeavor which produced tremendous value with a small initial investment.
Evolution
Let’s talk about our process of moving from ES6 for both our react.js frontend and node.js backend with Google’s Protobuf.
Our company has several complex stacks, with about 45% of the codebase being scala. We use Scala and the Akka framework for our real-time-bidding (RTB) stack for digital video ads which answers up to 1.3M requests / second, decisioning across several billion users, and answering in ~30ms. It’s a beast. We also do streaming ingest of about 5Pb of data / month via Kafka to Spark Streaming, which is made available to Spark SQL. We compile our own Spark and have committed features in SparkSQL in 2.1.
So it’s safe to say a good amount of our engineers come from strictly-typed language backgrounds. The other main stack is our node.js / react.js stack which serves composite API services for the frontend. Historically these have always been Javascript and therefore loosely typed.
Disclaimer
Before I get into singing the praises of Typescript, you have to understand that I’ve been a card-carrying Linux user for about 20 years. Except for Internet Explorer testing with Selenium, I have not touched Microsoft technology in about 18 years. I ran Linux on the Desktop as my primary environment going back 20+ years.
Now, the Kool Aid… I must say that they really got it right with Typescript, as it adds so many useful and productive features to Javascript, and they involved the broader open-source community in its growth.
The problem with Loosey Goosey Typing…
Scripting languages are a double-edged sword. On the one hand, they are very easy to learn for beginner programmers, on the other hand they allow you enough rope to hang yourself. Javascript is not immune.
Case-in-point: back in the dinosaur days of my career writing php, when I wore a bear skin and lived in a cave, the most horrendous bug occured due to loose typing. I was in charge of a high-volume ecommerce checkout which was a SAAS platform for print-to-web publications for job postings that was used by 100’s of newspapers in the US.
The checkout was sufficiently complex and allowed our customers to create products of arbitrary shapes and sizes to sell to their customers. There were easily 2,500 differently configured packages and options, therefore it was necessary to design the products in a way that allowed for flexibility.
My main responsibilities were twofold: 1) create an API which developers could use to deal with all of the phases of the online checkout, including an object-model which encapsulated all of this configurability, and 2) build a testing / validation framework which recorded live transactions, scrubbing the PII and credit card data, which could then be run as test fixtures prior to any deployments to ensure we were not going to crash this high-volume order ingestion endpoint.
As 2) became more and more mature, it was easier to push new software builds with confidence, and it evolved into this blissful state of running fast with the scissors.
There was just one problem. Back then, you could tack on any properties to objects you wished, and there wasn’t the notion of public / private, nor strict typing. Type-casting was done left-and-right in order to make your life easier. Right? WRONG!!!
Long story short, these loose “affordances” of php allowed another developer to tack on a text-based property which collided with another decimal-based variable that was used via reflection deep within the abstraction of the API in order to sum up the order total. They had no idea they were “doing it wrong”, because nobody or nothing told them so. Layer on an implicit type cast, and KABOOM!
As a result, there was this insidious bug which caused summing errors, and which dealt with the daily settlements with customers being off. You just can’t screw with people’s money, and this bug cost countless engineering time to track down.
Granted this was more about loose use of objects adding “transient properties”, but the object interface is definitely a characteristic of type. I was able to grab the creator of php Rasmus Lerdorf after a session at Oscon for a discussion about this. He was first entirely dismissive of the problem, but came around by the end of the convo that php needed a strict Object paradigm whereby you could “lock” down objects so this could not happen.
Older Javascript suffers from the same problem, eg:
var obj = { foo: “bar”};
//-= Far away, in another part of the code,
//-= this could potentially be totally naughty
obj.baz = “Ohayou Gozaimasu”;
//-= And potentially, so could this
obj.foo = 3.1415;
Prior to ECMA-262, there was no reliable way to enforce this. Then they came up with Object.preventExtensions to combat these effects. That’s a great start, but most people never use it to lock objects down. The type system available in Typescript takes it much further so that you can “say what you mean, mean what you say”.
A Solution to Loose Types
Our full-stack engineering team brought in Typescript because they wanted more type safety and guardrails. Allowing us to move even faster. Traditional JS is written for ease of development, and it’s not possible to hit all the use cases all the time. Even with strict unit test cases, we end up using our own APIs improperly; similar to that php case study.
What we like about Typescript is that it’s gradually typed, meaning that we can layer on more and more strict rules and compiler features, while still maintaining working apps. This was an important aspect as we migrated our code to be compliant.
People can understand and use the code more freely and it greatly aids in the readability. Newly onboarded engineers can grok existing code quicker because it’s self documenting.
Between Scala and Typescript there is a very close comparison, with some nuances in type differences. It’s very easy to reason about them in the same way, and there is less context switching when working between them. Similar to both, you can tell if it compiles and it brings another level of confidence to the code.
One downside with TS early on was that there were libraries which didn’t have full typings. However, that has evolved over time, and one of most popular Github site last year was the “definitelytyped” @types repository bringing in type libraries for all of the most popular Javascript packages.
In the frontend we use protobuf.js to generate TS and we use those types for our data structures everywhere. We also use @types for the state management.
The integration into protobuf, we use across our stack, one helpful thing is that it auto-generated stubs in TS for server and client-side. Whenever we get something from the DB, it’s able to narrow that @type declarations. From DB to the API to the Frontend, this ensures typesafe throughout. In short, the same contract between front and back.
IDE Efficiencies
Across the board, we found that the IDE support is not quite all that great, especially when you mix in GRPC and other types, however VSCode was the best we found. We couldn’t get Webstorm to work for our purposes. It is still green, and one must install many plugins to get it really usable. There are rough edges, but VSCode is best tool for the job, and that makes sense since it’s written in Typescript.
Some of the interesting effects of the IDE are code completion, flow-of-execution scope checking, and dynamic code correction as you type. If you try to pass an improper type or argument to a function call, the editor warns you immediately, and when you are unsure of an interface, dynamic documentation pops up to aid you.
Playing around with our main Typescript project, I was even able to customize my sacred vim dotfiles to allow for all of these same features with the help of youcompleteme.
Saving Ourselves from Ourselves
As our codebase has gotten sufficiently massive, it became more and more difficult for any one person to understand it. Different people and teams are building large components that other people and teams use.
We are now leveraging type safety in all of the code where it matters, but this did not come without taxes:
- When we first started adding in Typescript, it was hard to deal with all the compiler errors. We ended up adding “any-types” and needing to layer in tslint annotations to ignore certain issues. As this code matured, we locked down “any-types” into more specific types, and it became less ambiguous about the intentions of the engineer who developed it.
- We tried to parallelize our front-end typings without finished protobuf types, and had to backtrack our typing and front-end logic that we created with assumptions and this did not line up with our centralized type repo. Finishing the shared types should have been our first priority.
It’s agreed that typing is preferred in some form when managing state in React. Default props are a bare-bones way to check state and prop types, and Flow is a common alternative solution in the React ecosystem that has bigger React community and support. However, because of our use of typescript in the rest of our stack, it was a logical choice to use this on the front end as well.
State management can be tricky if you don’t practice immutability or a 100% functional programming approach. This means that it can be often to track down where a mutation might have occurred because of React’s abstraction and lifecycle methods that can make it trickier to see when and where an object is being transformed incorrectly. Typescript nips this problem in the bud, forcing us to explicitly state transformations and makes it very easy for another engineer to read you code and follow your typings and props as you navigate through all the front-end components.
Typescript makes reading React code pleasant, the metaphorical “guide rails” allow us to navigate other peoples’ code with confidence and comfort. It has helped us work faster and collaborate together more smoothly.
The compile-time errors are wonderful as well, because they drastically reduce the time to debug and they tell us how our reasonings about types are incorrect.
Conclusion
Adding Typescript to your frontend and backend Javascript takes some time to adjust, but it can reduce confusion as your projects become sufficiently large. The shortage of available Typescript experts which coincides with an noted uptick of adoption by companies. We feel it’s worth a look because the @types support of most modern and popular Javascript projects is now pretty much there.
This video presentation is a great overview of the migration strategies for Javascript projects to become Typescript-compliant, along with new features in Typescript.
Go check it out!
