Video thumbnail for How I learned to love build systems

Build Systems Explained: From TI-89 Games to Enterprise Software

Summary

Quick Abstract

Unravel the mysteries of build systems! From teenage gaming hacks to enterprise-level software, discover how code transforms from text to reality across different platforms. This journey explores the evolution of understanding compilers, linkers, and the often-painful path to cross-platform compatibility. Learn how early experiences shape our understanding of complex tools and the value of exploring beyond the surface.

Quick Takeaways:

  • Early coding experiences, like modifying TI-89 games, provide foundational knowledge.

  • Cross-platform development presents unique challenges, especially with dependencies.

  • Tools like CMake and MSBuild bridge gaps between operating systems.

  • Enterprise-level build systems leverage caching and distributed compilation.

  • Understanding the underlying components unlocks more efficient development.

Why does that seemingly arcane build process exist? Often, seemingly unnecessary complexity has a purpose! Don't accept things at face value, and keep digging!

Early Encounters with Code through Video Games

As a teenager, I was introduced to code in two distinct ways, both centered around video games.

Game Development on TI-89 and Java

In high school, I convinced my parents to buy me a TI-89. I was eager for this model because I knew I could create games with it. During various classes like math, French, history, and philosophy, I spent hours writing TI Basic to make simple games. Later, I advanced to creating games on the computer using the Java programming language in the Eclipse integrated development environment. In both cases, the process was somewhat similar: I'd write text in the editor and then press a button to start the game.

Linux and Game Installation

Another exposure to code came when a classmate brought an Ubuntu CD to school. I installed Linux on an old laptop my father had saved from the dump. I wanted to play games on it, but the process was complex. The Linux community at that time instructed me to go to SourceForge, download a tarball, extract it, run configure, make, sudo make install, and then run the game. I was confused as to why making my own game was as simple as pressing a button, while installing someone else's game required so many commands in the terminal.

Sharing Games and Cross-Platform Challenges

When I felt my games were good enough to share, I was surprised to find out that what ran on my machine didn't necessarily run on others'. I understood that a game consisted of assets like graphics, music, and sounds, along with an executable produced by a compiler from source code. I compared it to my parents using a recipe book to make a delicious dinner. However, I couldn't figure out why I couldn't share an executable like I could a pie. Back then, not many kids used Linux; they mostly had Windows computers and played store-bought or cereal box games.

Turning to the Internet

I turned to the internet to find like-minded people. I had to learn English, and being extremely online came with its own set of traumas. Even among internet users running the same flavor of Linux as me, they still couldn't run my games. There were always issues like missing libraries or not being able to find files.

Cross-Platform Development Aspirations

At 16, I wanted to make a cross-platform 3D adventure game. Building on Linux was possible but distributing was a nightmare. Windows had great games, but I had no idea how they were made. It took me a while to realize that games were made similarly on both platforms, just with different tools. Windows seemed better for art with tools like Photoshop, 3ds Max, and Cubase, while Linux was more accessible for code with its open - source nature.

Discovering MinGW and Making Windows Executables

Finding out about msys and MinGW was a game-changer. I could finally generate a Windows executable. It felt similar to what I did on Linux. I had a terminal, a bash shell, and could run commands. I started to draw parallels between Linux and Windows executables and libraries. On Linux, executables had the executable bit set and showed up green in the terminal, while on Windows, they had the.exe extension and could be double-clicked. Libraries were.so files on Linux and.dll files on Windows.

The Limitations of MinGW and the Introduction to MSVC

However, using MinGW didn't feel like real Windows development. I had to distribute a bunch of Emsys libraries with my games, and it was hard to figure out which ones were needed. The dependency walker tool was a big help. I then learned about MSVC, the actual compiler used by Windows developers. But it was a complete development environment, similar to Eclipse, and I was worried it would lock me into a single platform.

The Arrival of CMake

I encountered CMake when I had to build and install a piece of software. Initially, I was annoyed as I was already used to running configure, make, and make install. But I discovered that CMake could generate Makefiles, Microsoft Visual Studio solutions, and Xcode projects. This meant I could build the same sources on different platforms using native toolchains. I also learned that Visual Studio solutions could be built from the command line with MSBuild and Xcode projects with XcodeBuild.

The Value of Configure Scripts and Cross Compilation

I went back to my TI-89 and found out there was a version of GCC that could build software for it. This was my first experience with cross compilation. I realized the value of the configure script. For my games, I was mainly concerned with Linux and Windows compatibility, relying on libraries that abstracted differences. But these libraries had to be portable. The configure script checked for things like the size of integers in C, running tests to determine compiler characteristics and conditionally enabling or disabling parts of the code.

Other Build Systems

Later, I discovered other build systems like Meson, which many GNOME projects have migrated to. I like Meson because it has pretty colors, can generate Ninja files (faster than Makefiles), download and compile missing dependencies, and on Windows, it can discover Visual Studio installations and invoke MSBuild.

The Tension between Abstraction and Understanding

When I started with Java, I didn't have to know all the underlying concepts like the code editor, compiler, VM, and runtime libraries. This allowed me to quickly start making things, but in the long run, it slowed me down due to the leaky abstraction. There's a tension between providing an all-in-one package for beginners and exposing the basic building blocks of software development.

Rust and Cargo

The Rust language offers an all-in-one experience with Cargo. You can write code and run cargo run, and it does everything from downloading dependencies to linting, type-checking, compiling, linking, and running. However, this doesn't work for everyone, especially large companies dealing with massive amounts of code.

Build Systems for Large Companies

Large companies often have their own build systems. Tools like sccache can cache build artifacts remotely, but it's limited to Rust, C, and C++. These companies' build systems need to handle multiple languages like C, C++, Rust, Java, Kotlin, etc. They can parse metadata in files like cargo.toml and translate it into something their build system can understand.

The Lesson

If something doesn't make sense to you, it could be accidental complexity, or it might just be outside your current worldview. As I gained more experience with different operating systems, computers, and building software over the years, things that once seemed confusing started to make sense.

Conclusion

I hope this video has been helpful. If you have questions about any of the concepts discussed, please leave a comment. You can like the video on YouTube, and if you want to support me, you can contribute on Patreon. I also write articles on my website, which is linked in the description. Take care of yourself.

Was this summary helpful?

Quick Actions

Watch on YouTube

Related Summaries

No related summaries found.

Summarize a New YouTube Video

Enter a YouTube video URL below to get a quick summary and key takeaways.