Software Foundations for Junior Engineers
principles of software that survive the age of slop
Coding is over! Better learn a new job! Being an engineer in your 20s right now must be terrifying.
I was born in the late 80s and started making DOS games in C at age 10. It was not a job because nobody in Germany believed it would create value beyond making factory automation better. Everyone said i should study something with banking or whatever. I’ll be rich.
Software was distributed for free. Copyright wasnt even really an established thing yet. The standard career path for anyone doing anything with computers was to exit to the United States as soon as possible. IBM had big campuses with all necessities on-site that made it possible for young people to move directly from dorm room to campus without any of the inefficient steps of being an actual adult first.
Around the 2000s, Germany then had lots of computers everywhere. Nobody really understood the consequences yet. The 2000s where a short period of lawlessness where the internet was mostly hackers and phreaks. We finally started to have very large online communities in sort of a self-regulating anarchy. Unlike most elders of the internet, i do not believe young people these days “dont get it”, what it was like. That’s because frankly its not much different now. It’s just elsewhere. While we had to dial into usenet to yell at rob781s shitty opinions, you’re just doing it in discord now.
The difference is largely that knowing computers is an extremly well paying job now. Or it was? It might be hard to tell whats happening right now, for you, someone early in their career as an engineer, or even just developer. Let me get to the point:
nothing is happening
All industries undergo waves of changes. Everything always changes, its the nature of progress. It’s what we as a civilization actually want. If you’re scared for your job, it is because the fruits of progress are not distributed equally. Maybe consider joining a union or other political group. Your job itself .. will be fine.
Engineering is the discipline of structured problem solving. Engineering is not writing code. Software engineering as a discipline is not over, in any way or form, just because some silicon valley clowns make some statements to make number go up.
Back in my days everyone was yelling that C will be dead soon. They made entire new cpus for running java and everone was supposed to learn all these weird enterprise patterns, but ultimately all of that stuff evaporated. We still write C.
I will teach you the fundamentals of why a computer is made the way it is, and you will hold power that transcendes the industry ups and downs.
Everything is a Van Neumann Machine
There is exactly one way that a computer works and the people who disagree have largely lost. MIT has a free course here: https://ocw.mit.edu/courses/6-004-computation-structures-spring-2017/resources/the-von-neumann-model-10-29-/
You might hear people saying van neumann is over, LLMs will just do math in their fuzzy neuron thing, and frankly these people are cringe. You can’t argue that a non-deterministic machine replaces a deterministic machine for tasks that requires a deterministic machine. What kind of twitter pilled freak comes up with this. The entire economy depends on fractions of cents of transactions to be correct. There’s entire industries that do nothing but ensure they are correct. Even if LLMs are 99.9999% correct, why would anyone accept that, with them also being 10000x more expensive to run than a desk calculator?
This is also a good first lesson of whats required to be a good engineer: understand why tools exist, for what purpose. An imperative program will output the same result every time, so use it when you need the same output every time. There’s some caveats for parallel computers of course.
Hidden traps in computer science basics
Computer Science is boring frankly. I studied it, plus Philosophy, and i cant recommend it really. But to be an engineer you will need a few basics anyway which unfortunately are missing from more practical learning approaches.
Undecidability
Underneath a Van Neumann Machine, is a Turing Machine.
There are some problems that can actually never be solved by a Turing Machine. Computers obviously are not self aware. They cannot tell from looking at code, if the code will actually ever finish. That is called the halting problem. Learn here: https://brilliant.org/wiki/halting-problem/
The reason this trivial sounding issue is so important is that you will see it appear in higher level frameworks as hitting a resource limit such as timeouts and recursion limits. Only when you understand the fundamental reasoning of a turing machine, can you understand why these error conditions exist at all.’
Similarly some other problems are called “undecidable”. A computer cannot actually solve them and we have to work around it with aproximation, and hopes and prayers. “Is this program malware?” is actually impossible to know for sure, that’s why we use fuzzy matching. You will sometimes see something called “Undefined Behaviour” which sometimes maps back to undecidability: it is not defined how a computer will behave when executing this code. More often you will have an operating system or framework save you, but sometimes, you wont. This is where the trap springs and where engineers are separated into those who just do the job and those who have the fundamentals.
P/NP
The second, more complicated issue that underlies all computer problems, is called P/NP , which is short for polynomial time vs unbounded or non-deterministic time. The MIT has a free course: https://ocw.mit.edu/courses/6-046j-design-and-analysis-of-algorithms-spring-2015/resources/lecture-16-complexity-p-np-np-completeness-reductions/
It’s a bit more theoretical and out there, let me try to explain the important bits: Some problems are solvable in a fixed amount of steps, where the number of steps is known upfront, no matter what the arguments are. for example adding X+Y is exactly one step.
An example of an NP-hard problem often used is the traveling salesman problem. It sounds as innocent as “Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?”. A novice programmer may think this should be solvable fairly simply, unknowing that this is actually incredibly difficult to the point where numours scientific papers have been written about this specific problem.
The whole O(N) algorithmic complexity discussion you may see in interviews maps back here, to the fundamental math behind NP. It’s not important to understand the math. But you need to understand that some problems cannot be solved with brute force.
The Two Generals Problem
Whenever you encounter a system with more than one turing machine, you will see a problem with many names such as “serialization”, “consensus”, “mutual exclusion”, “synchronization” that all go back to the generals problem. A program cannot know the state of another program. It can send a message but it cannot know if another process has received a message it sent. It cannot “see” into another machines inbox to determine the state of the other system. It can also not simply ask for confirmation, because that that may then be lost.
A whole branch of problems and solutions spans from this fundamental issue, such as synchronizing two processes with locks, inboxes, mutexes, semaphores, all sorts of methods have been developed over the decades at all sorts of levels. They’re all the same thing.
The scaled up version is the Byzantine fault where we also deal with malign actors intentionally trying to intercept or alter communications. This is important in the internet, or any communications over untrusted media. Someone who doesn’t understand computers, often incorrectly assumes this is solved with encryption, when in reality the problem is fundamentally unsolvable.