One of the most common takes you see around, being it a joke or serious, is that the software development industry sucks. And I’m not talking about it being hard, or overpaid, or any sort of work/life balance (whatever that may be, I’ll let you define). Usually the topic on-hand is about the life of the developer being miserable as the standard, which in return has as symptom the many videos on youtube about burnout, career changes, and overall mental life crisis that a lot of developers share.

There are many theories on why this is the case, but I would like to talk Jonathan Blow’s theory, at least on a first moment, which is based on the idea that we, the human race, keep reliving the cycle of forgetting important pieces of knowledge, which causes the collapse of the entire civilization eventually. In summary, the thesis puts great emphasis on blaming abstraction as the root cause of the entire process. Although I agree with his main thesis, I would like to expose related ideas to and why, in my opinion, abstraction is the wrong target to solve the problem.

We should go Down

Blow’s point is about overcomplication and levels of abstraction. The talk starts by him showing historical evidence that great discoveries were made in the past, then something made such civilization decline, resulting on those inventions being lost to time, just for later to be rediscovered in the future. The pattern is palpable, and it is good that we managed to rediscover something, but it would have been way better if we haven’t lost it in the first place. His point is that he feels we are about to do the same with software.

The argument is made in an onion-like fashion, in which getting further from the core is getting further from actual problem-solving. First, there are the overcomplications: extra steps to fundamentally do what we used to do decades ago and overuse of newer/hotter standards to solve simple problems, e.g., LSP being a server just to lookup a label definition. These complications not only made life harder for little gain, but also had some secondary effects, such as being harder for everyone to fully grasp an entire scenario. The extra space required to remember all the extra steps (which competes for space with actual useful information) makes it harder to research problems with the avalanche of noise that certainly will come. The cherry on top is that this process being harder, even though seemly for the sake of it, raises the bar for who will be able to properly understand it, hence fewer useful resources will be available. Some may say we are already witnessing this effect with our browser queries being less useful overall.

After that, we have the problem with levels of abstraction. As we add more and more layers of abstraction, we are getting further away from the actual hardware, regardless if we like it or not the physical computer is what we have at our disposal. Being isolated in a place far from actual computer, with it working as the ground in our system, we run the risk of forgetting what actually happens within it or worse forgetting how to do useful stuff in it ourselves. Blow’s example talks about game engines: if everyone just uses the engines, and know nothing but that, who is developing/improving engines themselves? In other words: if everyone is poking the magical box and being impressed by its magical tricks, who will know how the magic is actually done? Doesn’t sound dangerous when even the magicians don’t know the secrets to their magic? If a “yes” to this question is not tautological, we are doomed to collapse.

Abstraction Descant

Jonathan is not alone fighting against the abstraction tower. Richard P. Gabriel, in his book Patterns of Software, also puts extra points that suggest that abstractions are something to be careful about.

When talking about abstractions being oveused, Gabriel focuses more on the fragile nature of programs built with larger towers of abstractions, rather than being more distant from the actual hardware, like Blow does. This is not a surprise, given that Gabriel is a lisper, hence being a little far from the computer should be ok when programming, assuming that the underlying run time of the language can compensate in terms of performance for your lisp machine.

One of the points brought by Gabriel is the authorship of an abstraction, meaning that there is a certain amount of craft that goes into making the best abstractions; a particular craft can only be mastered by a few. Not everybody creates great abstractions, neither all current available abstractions are great ones (especially ad-hoc ones), hence Gabriel proposes experts should be the ones dedicated to create and refine abstractions. He goes as far as to say that the centralization of the ownerships of great abstractions, implying that most programmers would be creating small and localized ones, would make the programming community lack a sense of purpose, due to the majority of us just “being merely cogs in someone else’s machine”.

Another interesting point is the power of legacy brought by Gabriel: a particular abstraction already in use may influence our vision to make everything fit within its boundaries, regardless of how unnatural that may end up being. Familiarity with a particular abstraction seduces us into a silver-bullet-like mentality, in which everything looks like a nail for our hammer — Churchil’s “We shape our buildings and afterwards our buildings shape us” strikes again. In contrast, if you try to combat the temptation, then you run the risk of actually not leveraging what could be a useful abstraction. Such tug of war makes us more suspicious about abstractions, and we are not certain when to apply it.

Other points are raised, such as the enlargement of vocabulary necessary when creating new abstractions. After all, meaningful labels for things help with communication, and niche abstractions being less useful to broader communities (what some people may identify as Monads being an example), provoke the feeling a bloated environment is being created for no apparent reason. Its side-effect is what Gabriel coined as unhabitable software; software that is really hard to live within and grow alongside it — it just does not feel like home.

We should go Up

John Backus, on the other hand, argued something on the contrary on his Turing award paper, Can Programming Be Liberated from the von Neumann Style? A Functional Style and Its Algebra of Programs. Backus’ point, however, gives the feeling that he took Churchil’s conclusion of legacy’s tremendous weight and tackled its core root cause: the human mind.

When talking about the so-famous von Neumann bottleneck, Backus is not only talking about a hardware bottleneck (in the physical sense of the word), but is also talking about a mental botteneck: the legacy of imperative languages molding our way of thinking, with the programmer melting upon the heat of what currently exists, what you can touch, what the computer actually does in front of you. Once solidified, our creativity may crumble, due to the fixed and static nature of what the physical computer imposed in our brains; we bow before what we currently are limited to and trade whatever abstract thought power may be useful to better solve problems. Backus’s solution? A mental war to propel a shift in how we think, with economy following along after the reverberating victory of those liberated from the bottleneck. In Backus’ paper, functional abstractions are foretold to win this battle. Sadly, it is fair to say that since 1978, if this battle is still ongoing, the applicative style of programming is far from the top dogs when it comes to winning programmers’ hearts.

As we know, a much darker path won this war and the pursue of correctness and consistent design was massacrated by a myriad of ad-hoc practices, ranging all the way from preaching void * all the way to a conflation on what something is and how something is implemented. Winners said there is no need to have more powerful types; everything is just a bunch of bits in the computer when it comes down to it. They said there is no need to verify something as pedantically as possible; scripting languages have faster development cycles. They said being and doing are one in the same; nobody needs denotational design. They said that we better stop raising the levels of abstraction, since the computer still works mostly as an imperative machine, and until that changes, your abstract ideas are just fluff to make you sound smartter! They claim the human mind is not the medium in which problems are meant to be solved, but the tangible computer is all there is to be. The machine should dictate us.

Relational Theory

One of the most outstanding examples of this discourse failing apart is when it comes to databases built on top of relational theory. The idea that your set of logical constraints are somehow tied to the performance of the underlying computer doing the heavy persistance work is something yet to be explained. Logical propositions are what they are, just as abstract as they are, regardless if you are using a monkey to persist the data via a series of cables or via a quantum computer. This conflation causes more harm than expected: understading of the details of the computer doing the storage makes it way into debates on how one should organize your data. Properties guaranteeed by the theory are then dismissed, given that what matters most are the performance trade-offs one might go into when perfecting a system to properly model the business it was intended to mimic.

The disconnect between those fronts is nothing new but myths of this relationship can still be heard. Instead of the abstractions provided by the relational theory prevailing, once again we are faced with the same “simplicity” principle of getting rid of anything mathematical/abstract/formal because of physical concerns of the computer. Remember that this has nothing to do with what are trying to achieve: solve problems with the proper mental images, with the proper abstractions in mind, that can hold the test of time for being solid, even though they require to be materialized via current generation’s computers. Once the ability to express constraints separated from the actual storage becomes common sense, we may see light at the end of this tunnel. However, just as Backus said, we gotta win the minds and the hearts of everyone, so later the market can follow along.

Conclusions

The “simplicity”-above-all folks have a point we can’t dismiss: if we forget what we are grounded to, we may be doomed into a collapse in which no one knows anything anymore. And they have evidence; vibe coding is now the most primal example of how far down we can go without knowing what is actually happening. Worse than high-level languages like Python and Javascript is the actual absense of programming itself, just pick one of the most imprecise ways to describe something, informal natural language, and make code out of it! If anything, this trend is here to accelerate Blow’s fears. The solution, however, is not to get rid of abstractions, but rather to emphasize that to go up, one needs to start from the ground up. Being able to properly leverage abstractions does not imply being forgetful about the transistors under the hood; we should be educated in both fronts. If we aim to reach the levels of productivity of Ken Thompson, and actually be able to code Unix in three weeks, there is a need to know what you are doing, even within the abstractions you are already employing. Even better, knowing the intricacies on how something work, gives you better insight on which abstractions are better to increase productivity levels, hence solving Gabriel’s concerns that only experts may be able to propertly incur into this endeavor. You keep peeling the onion; you keep traversing the abstraction tower; from the most mathematical things on the highest point until you reach the electrons and circuits that live on the ground.

Sadly, it seems to me that the willigness to go into those deep waters requires passion. Passion about the craft of programming that goes beyond a financial interest. Something that may be a few lightyears in the past, where the ones dealing with computer science were unreiling a new scientific thread at the time. It may be the case that we, as a software community, may never be able to decouple ourselves from these conflations on levels of abstractions. There is no doubt in my mind, however, that in the perfect world the computer is just a tool that we try to accomodate solutions that live in our minds. As imperfect as the computer may be for the job, it is the best we currently have. Fighting for this dreamland future, as frustrating as it may be, is better than to just accept that either the only way out is to bend ourselves to the hardware or that there is nothing that can be done and we are doomed to be miserable until the dawn of time.