Why Foundations Matter

TL;DR - without foundations you quickly forget them when working with high levels of abstraction and become headless chickens leading to unnecessary bottlenecks, complexities and brittle implementations.

There is a tendency in the software development landscape that exists today to simply pick up and use whatever gets the job done in the fastest way possible.

Fastest, purely in the terms of development time.

However, this philosophy has lead to a jungle of abstracted away complexities and ever rising memory demands, computational demands and, of course, bandwidth demands (network).

Casey Muratori from Handmade Hero makes the argument (among other significant reasons that I will not get to but are of importance) that this is the case of ‘bad code’ that it is a lack of focus or focus away from the design of how we code things in a non-modular way that is what has led to this.

His reasoning, and my agreement on the cause for that symptom, leads us to conclude why there is an obsession over design patterns and frameworks and various other flavours and styles of coding, especially in the OOP landscape, without really understanding when those design patterns work and what is the essence of those design patterns.

Of course, this comes with tons and tons of lines of code that makes things brittle. Code, or software, that tries to do more things than it needs to do. (a gripe of many in the Unix adjacent fanbase)

However, I would diverge to saying that it is, in fact, the result of not having foundations that bad code comes into existence in the first place.

Everything becomes abstracted away over time. And over time, the craving for more abstraction exists because of the over-production of engineers that don’t really understand or want to understand what is going on under the hood. A feedback loop gets created.

Abstraction is required to build things fast, engineers use abstraction to get from point A to B. The second generation of engineers come along only being exposed to this abstraction and create more abstraction. And so on. In the process, eventually you get a point where the non-abstracted bits are so out of the question that there’s no where to go but to reinvent the wheel to try to understand it (inefficiency at its best).

This is similar to what happens with doing math using calculators.

You don’t need to be a mathematician or know how multiplication works to use a calculator. Certainly, a lot of complicated calculations require a calculator.

For most trivial tasks, this approach is fine. You may go about your life never needing to know the fundamentals or have a deep understanding of math and still do fine.

You can budget, you can calculate your profits or losses, you can figure out discounts and you can, probably, do your taxes.

However, what you cannot do with a calculator and a poor understanding of math is rocket science, civil engineering, nuclear physics, etc.

Math has a correcting mechanism for this where it will whack you in the face if you try to build anything more complex without covering your bases.

However, in software, this fact is not obviated. There is no correction, or very little correction of this. There are always hundreds of different, over-engineered solutions waiting to be applied for you to get the desired results (getting a button to do something, making little charts).

You can get away without knowing much foundations as there is nothing that would stop you from getting the desired results.

In the “real” world, if you don’t know math, you have to get people that do and judge them correctly or get someone else who will be capable of making a judgement about them. You can see their track record and there are several ways to help you do this.

In software, you can get libraries (whatever fanciful names they come up for them and their own definitions such as “frameworks” are all meaningless marketing jargon for what needs to be done, they are not meaningless for getting people to adopt it) to do the task for you.

Seems like they’re the same thing, right?

Except, here’s the thing, there is only one way to do math.

Every single person that is hired for the job has to go through the same steps and have the same correctness in their math to achieve the same task.

On a lower level, this is exactly what happens, or what should happen, in your computer. For every language that you write 2 + 2 in, your computer must do the same thing on the lowest level, ideally, to give you the result you desire.

In software, you can use a game library to do 2 + 2. Or better, you can use one library that spits out the digits and another that does the addition and get the result.

This is obviously ridiculous and certainly no one, outside of jokes, does this.

But it is possible that this is exactly what ends up happening when you’re generations removed from how simple tasks are done.

OOP is especially a victim of this. Functional programming structures become so obviously and needlessly complex in code that you can see what is going on and have some hope of simplification. It is very obvious in FP but not so obvious in OOP.

Another way you could imagine is taking someone that hasn’t done any math in their life and starting them off with abstracted away matrix calculations. I know it sounds ridiculous and impossible but whichever way you can fathom it, such a person may try to get to simple multiplication word problems in a very rounabout manner.

So what is the solution to this problem of high abstraction?

Well, the obvious solution is market pressure due to hitting the limits of Moore’s Law becoming invalid, but that is for another time.

A more intellectually entertaining (hopefully) solution is simply picking and choosing.

Instead of importing libraries upon libraries, having a grassroots understanding of what gets you from point A to B in the least amount of work (or lines of code or computing power) will lead you to make a better decision maker who can pick and choose exactly what part of the abstraction you need for your particular task.

Why use a Swiss Knife if all you need is a file? Or a pocket knife?

To the reader, I apologise that there aren’t any more broad sweeping conclusions to take away from this more than that. The bit I alluded to about Moore’s Law may be coming out some time which, hopefully, offers better takeaways.

But I can offer you something else….

A talk! A talk that you will hopefully enjoy and take much more away from than just about software or programming.

Preventing the Collapse of Civilisation / Jonathan Blow (Thekla, Inc)