A Cognitive Steeple Chase, and the Genius of the MongoDB Interface
“I ain’t here to argue about His facial features
Or here to convert atheists into believers” — Kanye West, Jesus Walks
What follows are my opinions alone and do not represent the views of MongoDB. Most of the people that read this post will work in computer software or operations because of the title. This post is short, semi-technical, for anyone designing something that is difficult to simplify, and mostly a public reminder to myself to remain on my quest to simplify.
MongoDB came to the world in 2009 at a moment when there were far more jobs and needs for software developers than such human capital existed. In turn, the “invisible hand[s]” of market forces went to keyboards all over the world — with open source software and search engines paving the road for what was to follow. Though always present and influential from the beginning of computing, a stronger wave of autodidactic developers hit the scene, without the bias of the predominant academic point of view. They wanted simplicity out the gate.
REST easy. One day I will delineate the history I know more in depth and articulate why the biases of academia at the time did not impede MongoDB’s long-term adoption. This short essay will explain the genius of MongoDB succinctly by familiarizing users with a scenario that readers will understand as context switching. Next—and don’t worry too much what these words mean if you are unfamiliar—readers will see two once competing data formats developers have used for representing data, JSON and XML. The essay wraps up with a discussion on of how most application programming today, thanks to many abstractions outlined here and elsewhere, is a very human endeavor and my bold claim that the best computer interfaces are the most human ones.
Think about a time when you were very focused on a task that could be difficult to resume if you were interrupted. The middle of a one-problem problem set in Physics, a passage of dense Kant in Philosophy, or the book of Numbers at church. Or just cooking some noodle dish where you need to boil almost all the water. If somebody interrupts you when you’re in the middle of doing something that requires focus you might come back to some burned noodles.
For many software developers, when they are interrupted for thirty seconds, it can take a long time to pick up where they left off. The context switch from the memory they were managing or the threads they were parallelizing may never switch back! Sometimes developers need to start from the beginning of the task at hand. While it’s not always a bad thing to come back to a problem later after your brain has solved it thanks to percolation, usually creators need to step away own volition. Uninitiated interruption happens to backend or full-stack software developers dozens of times in the work week because of the discontinuity of relational databases from APIs and business logic, the throes of tables and rows.
Software developers building application backends work with modern APIs that return JSON (JavaScript Object Notation). They build their business logic using an object-oriented paradigm. And then, when they get to the step of data persistence or querying, they have to contort the structure of the data to conform to the requirements of a relational databases. It’s like being interrupted when you’re focused on something important. It happens to them dozens of times in a work week, amounting to ten minutes of productivity lost here, and fifteen minutes there, and so on.
A workplace consultant speaking with the Washington Post on the topic of interruption in the workplace estimated that the cost to productivity is about 6 hours per week. Let’s assume for many developers stuck on relational databases, this interruption sums up to about 7 unproductive weeks in a 48 week work year. If my assumption holds, and it probably is exaggerated for some users, an additional seven weeks of many engineers being frustrated because their boss was really comfortable with SQL and hand’t re-evaluated the market.
Before JSON became popular and MongoDB was one the fastest growing companies in history, the relational database companies had a friend in the format department. XML. It’s much simpler to map XML to tables and rows but people still use JSON whenever XML used to be used because it’s a pain to read.
Many of the fastest growing companies in the world, across every industry, are using MongoDB. The cognitive continuity is a big reason why. There are many more people who know SQL, teach SQL, and preach SQL. But almost all of them are getting challenged by application developers that can innovate faster.
It doesn’t have to be MongoDB. I’m crediting them with popularizing the object-native paradigm. Today, there are many widely used NoSQL data stores for different purposes like Elasticsearch for logs and observability, DynamoDB for—admittedly solid—vendor lock-in, and Redis for caching. Even before MongoDB, there was SOLR, a once widely used NoSQL data store for search, and the incredible Apache Lucene (there’s finally a managed version after nearly 20 years).
There are many pictures on the internet juxtaposing JSON and XML but I chose a small one so you have to do some more work. Squint. Pinch. Reach for your glasses. That’s how it feels to work with relational databases to build applications, most of the time. I acknowledge that some ORMs and some frameworks make the context switching problem much better but unsolved. The cool kids want to use JavaScript/Typescript, Python/Data Frames, Go (kinda), Elixir, and Rust. Some companies must de-risk and evolve one step at a time, and still use the powerful Java language. You should feel good with whatever language you are using and whatever problem you solve daily.
I can remember a French and a Spanish dreams long ago. I cannot remember the first time I dreamt in JSON. It’s not Turing complete and is only a format we borrow from the JavaScript language. I have totally imagined objects in JSON. If you try to imagine objects in a relational database management system, you end up with a bunch of file cabinets, excel sheets, and arrows pointing in every direction.
MongoDB stores the data in BSON (Binary JSON), which is the discount version of JSON. It’s compressed so that you can store the JSON for less money. The continuity of the API (REST APIs returning JSON), application logic (and an object oriented paradigm), and the persistence layer (BSON) take a little bit of the edge off. As an added bonus, the MongoDB query language is also JSON.
I already know I won’t convince everyone. Many developers have a religious devotion to relational database management systems. In the past 15 years, some teams like Snowflake and DataBricks have built incredible SQL-powered offerings that serve a specific purpose. Ex-Googlers have hoodwinked uninformed buyers of credits into thinking they’ve built a SQL-speaking Oracle alternative that will facilitate a buttered lift and shift. And other people have specialized needs that I would not in good faith recommend MongoDB for today.
There’s so much data, and so much more coming, there will be many awesome database companies that emerge in the future. I predict 100 in the fortune 2000 by 2042, and many do not exist or are not known by anyone. I’m enamored by the fundamental design choices of the MongoDB interface.
There are many other reasons companies choose MongoDB that I didn’t mention because I’m not trying to advertise MongoDB in this post. I am arguing for cognitive continuity in the user experience, as much as possible, whoever your user may be.
