Last year I tried doing a side project with a talented ex-Google buddy who insisted we set up Bazel to replace my simple Makefile. Three weeks later it still wasn’t working on my Windows box. We had a mixed Python and C++ code base and I like to use MinGW64 gcc on Windows. He blamed Windows and tried to get me to switch to Mac (no thanks, lol) and eventually he lost interest and gave up. The project went on to win an OpenCV funded competition and became the basis of a startup — good job, GNU Make!
So the answer IMHO to “when to use Bazel” is “never” :)
In general, if someone wants to beef up the tooling in my project that I know is fine to begin with, I still say "I don't mind, go ahead, but I will continue using my old tooling until yours works with no regression." Half the time, they abandon it. I don't even know how Bazel works cause I've never felt enough pain with makefiles etc to research an alternative.
Also, Google tooling knowledge doesn't transfer very well to outside projects. Everything is special on the inside there. They use an internal version of Bazel called Blaze that's totally integrated with everything, with entire teams dedicated to the tooling, and only a few officially supported programming languages, so of course it works smoothly.
I used it for several years inside Google, and TBH it was often painful in the ways external users have described in this thread. When I was doing something very "normal" from the point of view of Google's monolithic codebase, i.e. targeting a C++ binary intended to run on Google's distributed computing platform, I found it smooth enough, if awfully verbose. When I was doing something less vanilla, but not necessarily uncommon, e.g. targeting Android, iOS, Windows, it was painful, and I spent a lot of time working around it or understanding the chains/mountains of clever things others had built to work around it.
I understood some of the design decisions and tradeoffs for why it works the way it does, but I didn't like it. I certainly wouldn't reach for it for a personal project. Possibly I still just didn't "get it". I've used dozens* of build systems at this point, but hey.
When was this? I recall they were just getting around to cross-platform builds around 2015 or so (as I had helped in porting some QUIC stuff in my 20% time). I would image 7 years later the picture looks very different over there.
> Google tooling knowledge doesn't transfer very well to outside projects.
A good number of their open source projects fall apart in real-world use. If you visit their repositories you'll often get the sense that beyond working on interesting problems, their engineers have no desire to properly maintain these projects for the next few years. The gamble of relying on a Google product unfortunately also applies to their open source projects.
I roast Google at every opportunity but this does not seem fair to me.
What about Golang? Angular? Dart? Even the bigger stuff like Chrome and Android while a bear to deal with are integrated with every day by a lot of companies.
What is some Google OSS that has failed in the way you mention?
Angular, famous for betraying their devoted developer community with a complete breaking change when going from 1.x to 2.0. No migration plan whatsoever. It was a totally new framework which happened to share the same name. After a few years they relented and put in some work to help migrate existing users, but by that point most people has already put in the work to migrate towards a framework that wouldn't just leave you behind.
>No migration plan whatsoever.
Not 100% true, there were a lot of workarounds and documentation how to upgrade.
Sure it wasn't great, but the few projects I did with Angular 6,7,8 was kinda nice. I still have an AngularJS project in production which is also nice :)
Of course any new JS project is now done in Svelte (nothing simpler) or as nice
Angular 1.x was (is?) still viable. Its not a breaking change because its an entirely optional shift. I was working in Angular 1.x at the time and while there was a lot of hype for 2 we all knew Angular 1.x wasn't going anywhere.
I give it a trial period still. But yes, one time a guy switched our NodeJS project to Typescript. It worked for a little, then stuff started breaking (mainly involving imports/modules). He left for unrelated reasons, so I immediately took us back.
My experience is that Windows is a constant source of headaches if you're supporting development and builds on multiple platforms and trying not to just write every build-related thing twice, Bazel or no Bazel. Likelihood of it being a PITA goes up fast the more complex the build & tools (probably why Make did better than Bazel).
I assume it's OK if it's your only platform—though I got my programming start on Windows, with open-source languages, and distinctly recall how managing the same tools and builds got way easier and more reliable when I switched to Linux. Luckily WSL2 is getting semi-decent and can call out to Windows tools, so it's getting easier to standardize on one or another unixy scripting language for glue, even if some of your tools on Windows are native.
Part of the trouble with it, though, is that there are multiple ways to end up with some kind of linux-ish environment on it, and that all of them introduce quirks or oddities. You end up having to wrestle with stupid questions like "which copy of OpenSSH is this tool using?" It's probably less-bad if you're in a position to strictly dictate what Windows dev machines have installed on them, I suppose. Git is installed, but only with options X, Y, and Z checked, no other configurations allowed; WSL2 is installed and has packages A, B, and C installed; and so on, crucially preventing the installation of other things that might try to pile on more weirdness and unpredictability.
Those kinds of messes are possible on macOS or Linux or FreeBSD or what have you, but generally don't happen. I think it happens on Windows because every tool's trying to vendor in various Linux-compat dependencies rather than force a particular system configuration on users, or having to try to deal with whatever maybe-not-actually-compatible similar tools the user has installed. So they vendor them in to reduce friction and the rate of spurious bug reports. Basically, the authors of these tools are seeing the same problems as anyone trying to configure cross-platform development with Windows in the mix, and are throwing up their hands and vendoring in their deps, which compounds the problem for anyone trying to coordinate several such tools because there's a tendency for everything to end up with weird configurations that don't play well together.
All excellent points but Windows is a very widely used platform (perhaps the most widely used, by some metrics) and it’s actually quite nice for development. It’s hard to take tools seriously that don’t work on it.
Unpopular opinion but I think the Unix approach of lots of little tools with arcane configuration files all blaming each other sucks. I get why it exists, why it became popular, and why it remains popular in some scenarios, but I don’t feel beholden to it.
If we are talking about stuff like backend web development then I'd say no, it's actually a pain to work with - few people bother deploying to Windows these days, many libraries/frameworks treat it like a second class citizen, tooling doesn't translate well (or at all).
Even frontend webdev is very unixy due to node/npm and popularity of Macs in that space.
When you're talking about OpenCV, GPU related things, game dev then the situation is reversed.
I think we each live in our own bubbles. I see very few people on macs or linuxes, almost everyone uses Windows and they're fine with web development. What's so unixy about node/npm? They feel like very good windows citizens.
For the most part npm has worked for me on Windows, but it's so incredibly slow compared to even npm in a Linux VM on the same host. I understand this is mostly due to FS overhead related to a multitude of small files.
Obviously this isn't a deal breaker in most situations, but it is a bit annoying. I've observed similar performance problems running Webpack on Windows in the past.
Add your npm directories to the virus file exclusion lists of Windows Defender. Note that some of the directories may not be obvious (those under AppData for example). The speed improvements are massive for tools like these that touch lots of small files.
Also take a look at this talk on NTFS, file system speed, and how the presenter optimized the Rust installer: https://youtu.be/qbKGw8MQ0i8 . There is an awful lot that can be done to tools that deal with lots of files to perform well under Windows too.
There’s a possibility this is due to corporate antivirus. Many AV solutions (seemingly not Windows Defender) scan every file that’s written, massively slowing down IO, but they can’t scan VHDs. Excluding the folder where node_modules is or running npm inside WSL2 has good performance on my work laptop, in a random windows folder not so much.
> I understand this is mostly due to FS overhead related to a multitude of small files.
No. It was the VFS design.
They have lots hooks and extension points for 3rd party, there is no way to improve without breaking backward compatibility.
It was the reason why many antivirus was broken in the first few releases of windows 10.
Node didn't even run on Windows for a while and then when support was added it was very buggy for a long time - it clearly has roots in Mac/Linux ecosystem and windows was a second class citizen. Can't comment on recent support because haven't done front-end on Windows >5 years.
The only time I see Windows frontend devs is full-stack .NET shops or low cost outsourcing centers. If nothing else then just for practicality - you need to test on Safari and iOS so might as well get Mac devices.
Right—in some sense it's not really Windows' fault that something like Git is barely-portable and relies on a rube-goldberg-machine pile of scripts and binaries in several languages with the result that it has to ship half a unix environment to work somewhere that's very non-unix, or that open-source software in general treats Windows the way web developers treat Firefox.
OTOH, in practice, if you're using a lot of open source tools and have developers working on every single halfway-plausible OS except windows, things are probably pretty OK, but then you throw Windows in the mix and suddenly the time you spend supporting your builds & tools shoots way up. That makes it feel like it's Windows' fault.
My install directories sure look like it's still that way. And I'm not using a decade-old git. I discovered this fact about git far more recently than a decade ago, in fact (though I was using it back then).
Android is an even bigger pain, suddenly you have to call Java code just to map memory and can't even have a simple C api for something as low level as that.
As a development and/or build environment? Never used it that way, myself. Agree that as a build target, and as far as its various first-party developer-facing features and libraries and SDKs and such go, it's got weirdly-many rough edges and half-baked-but-heavily-promoted junk for such a major platform.
Visual Studio has pretty good native support for CMake. Going from Make to CMake would have been a better decision than trying to use Bazel.
There's a standard find_package/find_library macro and definitions for OpenCV (which is your use case here). Then use either Microsoft's C++ package manger or Conan to actually install the libraries.
I had a recent issue on windows with build times increasing 10X, turned out the Unix approach of spawning lots of processes was triggering the corporate antivirus. Didn't cause the build to fail but the result was the antivirus was scanning everything the process touched or wrote to disk. And make + gcc opens and closes a huge number of files.
I mean, Python (moreso its ecosystem) is notoriously difficult to get working w/ Bazel idiomatically. Bazel is advertised as a language agnostic system (and to be fair, it's better at it than, say, Buck), but in practice it tends to work better w/ stacks that already have some degree of hermeticity (Go, Java), whereas YMMV very much with the more loosey-goosey stacks (Python, JS).
IMHO, Bazel is a classic example of Conway's law[0], and it falls squarely in the "big corp" class of software. You have to be running into issues like 6+ digit CI compute costs or latency SLOs in hundreds-of-teams monorepos before Bazel really starts to make sense as a potential technical solution.
I've used bazel at medium size (100+ engineers, multiple languages) and tried to use it at a small size (20 engineers, almost all go with a tiny bit of C) and I think the "never" and "unless there's no other way" answers from this interview are pretty good.
C + Makefile + CCache: I just `find . -name "*.h"` and throw that as the dependency of every C file, and then let ccache/make figure out the rest. My code base runs upwards of 10+ million LOC, and incremental builds are in the "hundreds of milliseconds" and "clean" builds (ccache clean, mind you) are about 30s. Truly clean builds (on a scratch system) are ~2 minutes. We do spend time making sure our very long files still compile very quickly -- we're careful to stay away from code that exercises pathological paths through the compiler. I've found that a good laptop can compile ~20kloc/(s-thread) throughput with parallel builds (it varies from 1000–100000loc/(s-thread)).
That's specific to c. Bazel is often used in cases where you have multiple languages in use, and possibly dependencies between projects using different languages.
> I just `find . -name "*.h"`
That sounds like it would cause problems with not triggering a build if a .h file is deleted. And would trigger unnecessary builds if any .h file is modified. But if your builds are that fast, maybe that isn't a problem.
It is specific to C/C++, for sure -- I started with that.
ccache is going to be as fast as any checksum dependency tracker --- otherwise you're relying on a timestamp, which isn't any better than "bare" Make. We use `*.d` rules from the `-MD` flag to prevent the "deleted header" case.
Considering the brevity of the Makefile (it looks like a "basic" build), performance, and the practical robustness, it's in a pretty great trade-off space.
From what you've described I definitely don't think it would be worth migrating what you have to bazel. My point is just that there are environments where it does provide benefits.
At the risk of looking dumb here, don't incremental builds take care of this, how is caching different? ? Make gives me incremental builds and my makefile lets me define the inputs and the outputs. What am I missing with Bazel?!
1. Make uses mtime to determine whether inputs have changed. Bazel uses a hash of all inputs to the target (file contents + other inputs like environment variables)
2. Make lets you define inputs and outputs, but doesn't enforce them. Bazel sandboxes build actions; if you try to import a dependency that you haven't listed as a direct or transitive dependency, the build will fail. This is leads to users never (or at least very rarely) having to run `bazel clean`.
I mean was there anything wrong about the makefile? Did your product need reproducible builds?
This is becoming more and more of a pet peeve of mine, about to become an outright annoyance; the adding or changing of tools without things actually improving, or the purported improvement not actually being relevant or even close to the most important thing that time should be spent on.
I had a Go project up until earlier this year, I could set it up myself. I picked makefiles to build it (not bazel, that would be overkill; not mage, that would mean more coding), and it worked just fine. I never had a compelling reason to move away from it.
Actually it was more than fine, it was a relief; my previous experience with tools like that had been Maven (where everything is XML and you need a plugin to do basic things like move or remove a file) and the Javascript ecosystem, from before everything was merged into package.json / nodejs-style. Dependency management too.
I used bazel for my latest project, mainly as an excuse to learn it. I ended up spending waaay too much time debugging bazel instead of working on my code, and I still can’t properly support windows because dependencies don’t build.
I will never use bazel again. Not worth the effort required, and doesn’t work out of the box as advertised.
That's like saying you should never build a house with concrete foundations because they take so long to dig and pour and the first layer of bricks doesn't need them anyway!
Get back to me when you work for a company with a monorepo that has to build and test everything in CI for every change (taking something like 200 CPU hours) because they didn't have the foresight to use Bazel.
Software development isn't housing development. You lay a concrete foundation knowing exactly what is being built on top. The typical way of developing software is like adding stories to your house until it falls over, then rebuilding it stronger, which is perfectly fine in the software world. Otherwise things get overengineered.
In practical terms, the makefile or whatever is usually good enough.
If you're big enough to roll your own software ecosystem -- which means you have thousands of developers working on hundreds of closely coupled projects in a single code base -- and you need repeatable builds, then Bazel makes sense.
Bazel isn't designed to make things easy. It isn't even designed to make things fast (though it is easy to add that on due to repeatable builds). Bazel is designed to prevent you from shooting yourself in the foot; providing strong controls on project dependencies, both dependencies of your project as well as other projects' dependence on your project.
That means in 99% of companies the answer is no. You're better off with CMake or something thing like that.
That's a very rare occurrence in software. Honestly it's probably about as common as in housing. You can rebuild parts of a piece of software but the build system is generally really foundational and almost never changes in my experience. Especially if the software isn't a library so nobody except the main developers has to interact with the build system.
The build system can change with hardly any of the code changing. This is evident in how I've never even had to think about the build system at my SWE day job. It's evolved in the meantime.
What's often impossible to replace without a full rewrite is a backend's DBMS, and people tend to underestimate how tight of a coupling that is. Only gets worse with abstraction.
I don't think that's actually the case. Bazel, AIUI, understands the entire dependency tree and can cache steps effectively, which a Makefile cannot (alone) do. (You could certainly build it into one.)
CI tooling is orthogonal; your CI tooling can invoke Bazel or Makefile+custom caching, it wouldn't really matter. But the problem in CI is that you lack state; whereas a developer's machine might e.g., have a bunch of .o files from a previous build, CI will be starting from a clean slate¹. So builds take longer. My understanding of Bazel is that it has decent support for caching and can thus makes those rebuilds a lot faster, which it has b/c it understands exactly the inputs to the steps at hand. (Which, if you write your own caching layer, you'd have to figure out.)
… but … I've also seen the same thing the top commenter has, with Bazel: endless suggestions to use Bazel … but migrating a large existing codebase is pain, and if you're not willing to put your work where your mouth is … then the status quo prevails.
¹While here CI lacking state is a sort of negative b/c of the time we need to rebuild it, normally, I think this is a good thing: it means you're constantly verifying you can build from scratch. Or, at least, something close to scratch, modulo things like vendoring. (I.e., if you do an "apt-get" from Ubuntu's servers in your CI pipeline … well … your build obviously isn't quite from scratch.)
Having experience FAANG monorepos, the other commenter was entirely correct about them papering over tight coupling with tools.
It makes more sense when you realize it was really built to deal with statically linked C++ - there's no guarantee of ABI, so you really need to ensure that all your dependencies are built the same way as your project to ensure compatibility (and the dependency graphs in monorepos get DEEP).
For any language with a stable ABI that utilizes dynamic linking, the complexity is far more then the benefit. The whole monorepo concept and all the tooling that goes with it really makes more sense when you think of it in that context of statically linked code that can't easily be packaged into something as portable as a JAR, so it's critical to build it all the same way.
Google uses a monorepo intentionally, and for good reasons. They're also a massive company. For them, a complex build system makes sense... and even that system isn't exactly Bazel.
Google uses a monorepo because they were popular in 1999, not because they had a specific reason to do so[1]. As the codebase grew, they never changed and instead built tools around it.
That's a perfectly fine choice if you have bodies to throw at the problem and can deal with the consequences of that choice, but most people can't. Google can and so it does. If it wasn't as successful as it is, it probably would have collapsed under the strain of the people and systems required to manage that code.
Ok, could've been chosen for 1999 reasons back then, but there are good reasons today. It's easier to build things collaboratively, esp across teams, when there's just one repo and one branch (I don't count release branches). Yes it takes effort to support due to its size, but that's contained within a few teams, and everyone else benefits from it.
Other tooling decisions place a burden on every SWE. Their config language is a weird trash that everyone has to sorta learn. Their deployment and integration testing tooling have similar qualities, and it's not cause of the monorepo. One consequence is that many teams cannot feasibly deploy microservices, even though it'd often make sense to, which is even worse if they're using C++.
You are talking about a two men project becoming a one man project. You are not the target of a complex build system. At that size, you would be fine with basically anything.
I’m really glade that Meson and Ninja exist however. I hate GNU Make like few things in my toolbox. M4 really is an awful language.
i think its more like for super small startups, the best move is always to keep things as simple as possible (like basic build/deploy scripts) and work more on the product.
Bazel is a fully reproducible and hermetic build system. A lot of painstaking work goes into it producing the exact same artifacts build after build. And that provides some interesting properties that you can leverage for artifact caching, deployments, and CICD.
We very happily runny a polyglot monorepo w/ 5+ languages, multiple architectures, with fully reproducible artifacts and deployment manifests, all deployed to almost all AWS regions on every build. We update tens of thousands of resources in every environment for every build. The fact that Bazel is creating reproducible artifacts allow us to manage this seamlessly and reliably. Clean builds take an hour+, but our GH self-hosted runners often complete commit to green build for our devs in less than a minute.
The core concept of Bazel is very simple: explicitly declare the input you pass to a rule/tool and explicitly declare the output it creates. If that can click, you're half way there.
> Bazel is a fully reproducible and hermetic build system.
Yes, and it's very important to note that Bazel does nothing to solve the problem about having a reproducible and hermetic runtime. Even if you think you aren't linking against anything dynamically, you are probably linking against several system libraries which must be present in the same versions to get reproducible and hermetic behavior.
This is solvable with Docker or exceptionally arcane Linux hackery, but it's completely missing from the Bazel messaging and it often leaves people thinking it provides more than it really does.
Bazel is perfectly capable of producing static binaries, if that's what you want. It's not fair to say that does nothing to solve this problem, it simply does not mandate that the artifacts it produces are static.
Most static binaries are not completely statically linked. The system libstdc++ and libc as well as a few other things are almost always linked dynamically.
In particular, it’s almost impossible to get glibc to link statically. You need to switch to musl or another libc that supports it.
Static linking means no dependencies whatsoever, so it only needs a syscall interface in the kernel. If your binary requires libstdc++, it's not static, period.
Go creates static binaries. I was a bit bummed out to learn that Rust doesn't by default since it requires glibc (and copying a binary to an older distribution fails because glibc is too old).
Yes and no, it's trivial unless you use some of the most common libraries that depend on, i.e. OpenSSL, like pretty much anything that talks with the outside world. Then it's painful. (You mentioned OpenSSL in your comment but somehow I missed that part when I replied. My bad.)
One thing Go has going for it is that they have a more complete standard library so you can do without the pain of cross-compiling—which is what you're doing if most of your libs assume you're in a glibc environment but you really want musl.
I know because I recently tried compiling a Rust application that had to run on an air-gapped older Debian machine with an old glibc, and the easiest solution was to set up a debian container, download Rust and compile there, instead of fixing all the cargo deps that didn't like to be built with musl.
I just went through this recently (built a rust binary on my local machine, sent a binary to another person, they reported GLIBC errors, had to rebuild with the musl target, various crates depending on openssl failed) and found that every library that depended on openssl by default also had a `rusttls` feature in the crate that disabled openssl and enabled rusttls (https://github.com/rustls/rustls) instead.
So I just migrated everything over to that (which consisted of enabling the `rusttls` feature for every crate) and made another build with musl. Everything worked perfectly fine, and since it's not a performance sensitive application, there was basically no drawbacks. The binary became a bit bigger, but still under 4MB (with other assets baked into it) so wasn't a big issue.
Other than libc I'm not sure what else would "almost always" be linked. It's going to depend on the language, but for Go/Rust at least static linking is pretty much the default, and lots of people even link to musl.
For other languages I'd say openssl is probably the next most common.
> but for Go/Rust at least static linking is pretty much the default
I don't think that's true for Rust. It defaults to depending on glibc, at least on Linux, and you need to explicitly build with musl to get static binaries.
Platform ABI typically specifies that libatomic must be dynamically linked, since otherwise libdl will misbehave. Lots of other common libraries including libpthreads, libm, libgfortran, libquadmath also are frequently linked (though commonly now are linked into libc.so for better system performance)
There is no platform ABI mandate nor Linux kernel requirement for userspace applications to be dynamically linked to anything. If you can talk to the kernel, you can do anything those libraries can do.
The linux kernel forces your application to include the vDSO shared library (by pre-loading it). You can ignore it and talk to the kernel directly, but you cannot get the same performance as you could when using that shared library.
On some architectures, atomics are implemented in the kernel by the vDSO shared library (__kernel_cmpxchg), which is supplied to user via libatomic. You can ignore it, but then you cannot interoperate other code (any which uses libatomic.so) without introducing data-races into the code which were not present in the shared-library version, since they may attempt to execute their atomics differently and thus incorrectly.
Others have mentioned statically linked binaries. I thought I'd mention I actually use Bazel to build most of my docker images, and unlike Docker itself, Bazel rules_docker builds reproducible (i.e., same digest) container images by default.
Having spent a great deal of time getting bazel set up as you describe, I feel you have given readers a misleading impression. Bazel does not come out of the box that way. It uses whatever toolchain is laying around on the host, by default. It builds with system headers and links with system libraries. It looks at your environment variables. You need to do a lot of surgery on the toolchain to make it hermetic and reproducible.
I can't comment on Nix, but I work on a project that uses recursive Makefiles and build containers to make the builds (almost) hermetic.
I dread running builds because it takes like ten minutes to run `make all`. I spent like four hours writing Bazel build files for it and all of a sudden my clean rebuilds were taking like five minutes and my incremental builds were taking a couple of seconds. It was fantastic.
Ultimately it was rejected because other people on the project hated working with bazel and weren't familiar enough when it to debug any problems. If a build didn't work, the only solution was to call me because no one else was used to interpreting the bazel errors: if a curl command failed in a script in a docker file somewhere, everyone knew what that meant. But I was the only one who would see "IOException fetching... Error 401" and immediately know that they provided the wrong password.
I also consulted for another project where people were using containers as VMs, running systemd and copying in updated binaries, because they, too dreaded rebuilding the containers. Bazel, again, made it so that they could rebuild everything in a matter of seconds. That project is still happily using bazel last I checked.
Docker is a red herring. The parent's statement can be modified for Docker like this:
> B̶a̶z̶e̶l̶Docker uses whatever toolchain is laying around on the h̶o̶s̶t̶Internet
More precisely, Docker can use exact, well-specified, cryptographically-tamper-proof environments (images); but the only way to actually specify and build such an environment is via shell scripts (in a "Dockerfile"). In practice, such scripts tend to run some other tool to do the actual specification/building, e.g. `make`, `mvn`, `cargo`, `nix`, etc.
If those tools aren't reproducible, then wrapping them in Docker doesn't make them reproducible (sure we can make a snapshot, but that's basically just a dirty cache). In reality, most Dockerfiles seem to run wildly unreproducible commands, e.g. I've encountered Dockerfiles with stuff like `yum install -y pip && pip install foo`.
If those tools are reproducible, then wrapping them in Docker is unnecessary.
Also note that outputting a container is nothing special; most of those tools can do it (e.g. in a pinch you can have `make` run `tar` and `sha256sum` "manually")
The bazel way to do it is to put the tools in an archive, refer to the archive and its checksum in your workspace, and execute the build in a completely empty sandbox.
Can you share a bit about the completely empty sandbox? Is this a build-root with it's own user and environment? Or does it build inside worktree, e.g. a subdirectory. Or can both be done?
The concept of explicitly declared inputs and outputs is awesome. Though it's closed build and the necessity to define the builds down to the compiler level makes Bazel complex and breaks some developer workflows.
For this reason we are building a Bazel competitor with a less restrictive build environment which can also be applied to projects with lower than 1M line of code.
I've been using Bazel for side projects these days, including small retro game projects for old game consoles. The entry price was high, but it works so well I have a hard time imagining working without it.
For retro game projects, the core of your game might be written in C or C++, which you want cross-compiled. That's easily within reach of stuff like makefiles. But then I start adding a bunch of custom tooling--I want to write tools that process sprites, audio, or 3D models. These days I tend to write those tools in Go. I'm also working with other people, and I want to be able to cross-compile these tools for Windows, even though I'm developing on Linux or macOS.
My Bazel repository will download the ARM toolchain automatically and do everything it needs to build the target. I don't really need to set up my development environment at all--I just need to install Bazel and a C compiler, and Bazel will handle the rest. I don't need to install anything else on my system, and the C compiler is only needed because I'm using protocol buffers (Bazel downloads and compiles protoc automatically).
The benefit is that it keeps a huge graph of all dependencies, so everything from the toolchain through your application can be built with hashing and caching. Sort of like make, but decentralized and based on hashed file contents.
Once you have bazel, you can distribute the workload so that you could have thousands of machines each producing artifacts to be shared with other build machines.
Then you can set up your dev machine to rely on those caches, so your local builds either use everything directly from cache or instruct a remote builder to produce the artifact for you.
No matter what you change, because the dependencies are graphed precisely, you only need to rebuild a very tiny set of artifacts impacted by your change.
Of course, this doesn't actually work in practice.
Your builds probably aren't deterministic, so your graph of artifacts won't be either, causing lots of stuff to rebuild. Also, it may work great for something like Java that produces class files, but not provide any caching at all for ruby.
Debugging and supporting it is a full time job for a team of engineers that dramatically outweighs the cost of keeping your projects sensibly sized.
You might think that as the project gets larger, it's totally worth it to use bazel for that sweet caching. But in reality the graph construction and querying will become so bloated that just figuring out which targets need to rebuilt becomes a full time engineering effort that breaks constantly with all tooling upgrades.
Also, the plugin ecosystem is just poor.
Bazel is the perfect storm of computer scientists loving big graphs, Google exporting an open source project and then rebuilding it internally, and inexperienced engineers being sold on a tech as being obviously right because all the big players use it.
This completely mirrors my experience with Bazel as well. As long as, like Google, you have plenty of engineering effort to waste on nothing but maintaining your build system, sure why not.
But if you actually have to get something done for your business to exist, it's a lot of unrelated work to keep the beast fed and happy.
I guess it's theoretically possible, but why would you do this? This sounds like some kind of "I want to prove it can be done" project like running a web server on a Commodore 64, or making a kitchen knife out of cardboard. Yes, you could figure out a way to download and install dependencies using Make and Curl, and maybe you could build the same library for multiple targets using a bunch of variable expansions, possibly by invoking make multiple times, or including the same makefile multiple times. Make sucks; it sucks a lot; I'd be miserable.
And then if you aren't very careful, you'll end up forgetting to declare some dependency, or some flag change will invalidate your build but Make won't do anything, or you'll make a change to your makefile and forget to clean, and then you spend another hour or day debugging some build that was made out of stale parts. I've done this before, which is why I avoid make for everything but the smallest projects.
> without java dependency
Bazel does not have a Java dependency. I think you might be assuming that because Bazel is written in Java, somehow that means that Java must be installed on your computer. This is not true. Bazel is a self-contained executable you can drop in /usr/local/bin.
I really don't understand your perspective here at all. I've seen people defend make, but make is full of so many traps and gotchas--I wonder how someone could use make and then decide that these traps and gotchas are okay. The projects that successfully use it tend to be small projects, use generated makefiles (automake, cmake, etc), or tend to be a bit simpler.
I use make myself, but as a rule of thumb, only for projects with a few files.
Bazel doesn't have any user-facing java dependency.
> what real extra benefit Bazel brings in here
Builds are hermetic by default, so unless the developer chooses to escape the sandbox, everything is guaranteed to build on other machines with no additional setup.
(Also, I genuinely hate when I have to manually install build dependencies system-wide and pray that there will not be any conflicts. Having everything pinned to specific sha256 or git hashes by design is a breath of fresh air)
I used a M1 Mac to try to build tensorflow and tensorflow-text and it is very untrue that everything is guaranteed to build on other machines with no additional setup.
The parenthesized comment is funnier to me because I had to download a specific bazel version to build.
That is true, Bazel itself is still evolving, and there have been breaking changes between versions. Sometimes the required version number is placed in a .bazelversion file, which makes Bazelisk your top-level dependency.
I'd expect Tensorflow to have some non-hermetic build actions, but if choosing a specific Bazel version was the only thing that was required to build it, that's awesome!
So, more tools on top of your tools. And this couldn't be a part of bazel proper for the dame reason MS built a separate tool to discover paths for their tools: overengineering
Bazel is more strict about what constitutes a dependency change. It doesn't use mtime on the file, it relies on the checksum. It also considers the command line flags to be part of the cache key.
So, spurious changes (touching a file) will result in a cache hit, while hidden changes (changing an environment flag used by Make) are caught.
This is particularly important if verifiable builds are needed for SoX compliance.
Does it re-read every single source file to find out which were changed? Sounds like a terrible approach. Or it uses mtime first and checksum after that?
Mtime first is unreliable on modern (fast) machines, you can edit a file twice in a single second. I think bazel has a daemon watch for file changes, but I'm not certain of that.
Bazel has a stateful server component that caches information about the build graph and source files in memory. As you said, this allows it to check mtimes first to avoid re-computing hashes for source files.
Bazel can also read source file hashes from the filesystem if the filesystem supports it.
My understanding is that somehow the bazel server tracks all that and keeps a hot process for monitoring the source files somehow to make the file io more bearable.
Can make do incremental builds based on checksums instead of modification time? How about distributing the build across a cluster in a language agnostic way?
The benefits that it brings is the whole reason it exists: precise dependency tracking.
That means you can't have missing dependency links (very common and hard to debug in Make or CMake for example). You can know what outputs a change affects so you can only build/test a subset of the project in CI. Incremental builds become totally reliable. Etc.
Author here. I wanted to get my head wrapped around when Bazel was an excellent build solution, so I interviewed six people with a lot of Bazel experience and picked their brains.
This somewhat long article hopes to answer questions about Bazel for future people in a similar position to me. If you know a lot of Bazel then you might not learn much but if you’ve vaguely heard of it and are not sure when its the tool that should be reached for I’m hoping this will help.
I particularly enjoyed the history aspect of it, these things give you broad contours of where pools of experience exist and what specific organizations were responsible for driving and advancing things.
I lost a month learning Bazel last year. Never again. Here’s a challenge: Create a Bazel built Angular library using only the publicly available documentation. Here, I’ll save you some time: You can’t.
What documentation exists is flawed and what isn’t flawed has massive holes. It was so bad that I had to ask a few Google employees if there was secret internal documentation somewhere. There isn’t, and, they almost all hated using it as well.
I went back to Make and had the whole repo building in an afternoon.
Bazel is a great idea but like most other Google OSS projects it doesn’t have strong enough documentation to form a community or enough of a community to create good docs.
Even if I brute forced my way into using Bazel, I couldn’t ask an employee to learn it.
We use Bazel’s rules_docker as well, and I would caution someone evaluating it with a note from out experience.
What Bazel does well (and as well as Bazel fits your use-case) Bazel does extremely well and is a reproducible joy to use.
But if you stray off that path even a tiny bit, you’re often in for a surprisingly inexplicable, unavoidable, far-reaching pain.
For example, rules_docker is amazing at laying down files in a known base image. Everything is timestamped to the 1970 unix epoch, for reproducibility, but hey, it’s a bit-perfect reproduction.
Need to run one teensy executable to do even the smallest thing that’s trivial with a Dockerfile? Bazel mist create the image transfer it to a docker daemon, run the command, transfer it back... your 1 kb change just took 5 minutes and 36 gb of data being tarred, gzipped, and flung around (hopefully-the-local) network.
It may not be a dealbreaker, and you may not care, but be forewarned that these little surprises creep up fairly ofen from unexpected quarters!
Edit: after 2-ish years of Bazel, I would say that for 99% of developers and organizations, the most likely answer is "never".
I worked at Coinbase for four years, where Bazel is the build system of choice.
It's so so much worse than the homegrown software it replaced.
Hermetic sounds nice in theory, but doesn't actually matter and comes with a massive cost.
1. Bazel is slow. Like really fucking slow compared to your native build tools. Startup time can be insane if you use multiple rules (languages) in your repo. There's a great feeling when you get to work in the morning to build your go project but you have to install the latest python, nodejs, and ruby toolchains because someone updated them on master. The cache works well, except with a 1000 devs something will always be invalidated on master.
2. The documentation sucks. It's written to explain concepts with how things work, with no examples of how to actually complete tasks you care about. Of course that wouldn't help either because the bazel setup you're using is heavily customised.
3. Everyone on your team now has to learn yet another DSL, and a bunch of commands to run. I would easily spend 5 hours per week either waiting for, or debugging some issue in bazel.
All this for what benefit? Everyone's on the same version of a dependency? Not even sure this is a desirable property.
Also the monorepo is slow as jelly to work with and many tools or editors struggle to open it. Good luck getting code completion to work well too.
It's one of those ideas that are nice in theory, but awful in practice. It's possible that with enough manpower you may be able to use it effectively, but we certainly were not.
I mean, aside from having to dance the JVM maintenance dance, you're also faced with the fact that Java is a memory hog unless you spend half your life tweaking obscure parameters in XML files.
The sooner Java dies the better, frankly. Java was great back in the day, even ahead of its time, but today it's just a relic.
Use Go, use Rust ... hell, practically anything is better than Java.
> Java itself isn't necessarily slow, it's the application design.
Java itself is a language, and cannot be slow or fast. However, the most popular (and thus well-supported) implementations of Java are slow. On those implementations, the idiomatic way of writing Java leads to massive amounts of memory allocations, which force Garbage Collector to allocate much more memory than it needs and perform complex collection strategies. Many allocations also destroy memory locality and completely negate the effect of CPU cache.
> Would it benefit from being compiled with GraalVM?
Maybe, maybe not. But GraalVM, at least for now, is not the standard way of running Java programs.
I think the point of the parent comment was that it's bad to choose Java for anything other than long-running server-side applications (where JIT compilation with Hotspot might actually prove valuable for squeezing out those last few percent of throughput). For anything else, Java is really a sub-optimal choice - the startup time is horrible, the speed is not really great, and the JVM maintenance is painful. For a build tool, which is invoked many times to do relatively small amounts of work (compared to long-running server-side applications), it just doesn't make sense.
>For a build tool, which is invoked many times to do relatively small amounts of work (compared to long-running server-side applications), it just doesn't make sense.
Bazel makes use of a client-server architecture. Java is used for the server component, for which startup time is not an issue. The CLI client is written in C++.
My take - Avoid Bazel as long as you can, for most companies the codebase is not big enough to actually need distributed builds, if you've hit this problem Bazel is probably the best thing you can do today, if you're that big you can probably spare the few dozen headcount needed to make Bazel experience in your company solid.
Bazel takes on dependency management, which is probably an improvement for a C++ codebase where there is no de-facto package manager. For modern languages like golang where a package manager is widely adopted by the community it's usually just a pain. e.g Bazel's offering for golang relies on generating "Bazel configurations" for the repositories to fetch, this alternative definition of dependencies is not what all the existing go tooling are expecting, and so to get the dev tooling working properly you end up generating one configuration from the other having 2 sources of truth, and pains when there's somehow a mismatch.
Bazel hermeticity is very nice in theory, in practice many of the existing toolchains used by companies that are using Bazel are non-hermetic, resulting in many companies stuck in the process of "migration to Bazel remote execution" forever.
Blaze works well in Google's monorepo where all the dependencies are checked in (vendored), the WORKSPACE file was an afterthought when it was opensourced, and the whole process of fetching remote dependencies in practice becomes a pain for big monorepos (I just want to build this small golang utility, `bazel build //simple:simple` and you end up waiting for a whole bunch of python dependencies you don't need to be downloaded).
And this is all before talking about Javascript, if your JS codebase wasn't originally designed the way Bazel expects it you're probably up for some fun.
I find Neil Mitchell's categories of small[1], medium[2], and huge[3] build systems useful. Blaze is absolutely fantastic as a huge build system, as it can correctly specify the exact semantics of systems with lots of components and complexities such as cross-language bindings, autogenerated files, etc. If you are building a huge system, then a small or medium build tool just won't be up to the task. At that kind of scale, somebody is going to be responsible for the build system working, possibly as a full time job, and it makes sense to tap somebody who understands how Bazel works and knows how to put those ideas into practice.
Conversely, as many comments here observe, it is terrible as a small build system. There, you want to be able to get started quickly and pull in dependencies without thinking too hard. A simple but easy to understand approach (even one based on make) might work. This is just a different problem than what Bazel solves.
My personal opinion (and I should emphasize, I am absolutely not speaking for anyone here, least of all Google) is that the best way forward is to take the ideas of Bazel (hermetic and deterministic builds) and package them as a good small build system, perhaps even compatible with Bazel so you don't have to rewrite build rules all the time. I also think compilers and tools can and should evolve to become good citizens in such a world. But I have no idea how things will go, it's equally plausible the entire space of build systems will continue to suck as they have for decades.
I completely agree with this, having spent an awful lot of time with both Bazel and Make. There is a much tighter, cleaner, simpler build system within Bazel struggling to get out. A judicious second take at it with a minimal focus, while taking some of the best ideas, could be wildly successful I think.
It's on my list of things that I will inevitably never get to.
> the best way forward is to take the ideas of Bazel (hermetic and deterministic builds) and package them as a good small build system, perhaps even compatible with Bazel so you don't have to rewrite build rules all the time.
Personally, I've found Bazel's tooling and dependency management to be extremely aggressive at pushing you to online-only development as your project scales in size. A company I worked for that used it lost multiple person-days for every engineer they had when Covid hit and the VPN went to crap.
It's great at being able to offload work to a remote server, but in my opinion that should never be the only way you can get work done. Local development should always be the default, with remote execution being an _option_ when available.
> A company I worked for that used it lost multiple person-days for every engineer they had when Covid hit and the VPN went to crap.
As far as criticisms go, “we lost a few days of work when a global pandemic shut the world down” seems pretty minor.
That said, I too am a fan of offline development and I find that bazel makes that pretty easy for me. I just run bazel fetch before I get on an airplane.
I tried using Bazel a while back but immediately ran into a few issues.
The existing codebase I was working with did not lay out its dependencies and files in the manner expected by Bazel so dealing with dependency/include hell was frustrating.
Then there was a large portion of the project that depended on generated code for its interfaces (ie. similar to protobuf but slightly different) and trying to dive into the Bazel rules and toolchain with no other Bazel experience was not fun. I attempted to build off of the protobuf implementation but kept finding additional layers to the onion that didn't exactly translate to the non protobuf tooling. The project documentation seemed out of date in this area (ie. major differences in the rules engine between 1.0 and subsequent versions) and I couldn't find many examples to look at other than overly simplified toy examples.
All in all a frustrating experience. I could not even get far enough along to compile the interfaces for the project.
Did you find genrule? It’s a generic escape hatch rule that can be used to just run any program. Of course it still needs the input and output dependencies declared and you are responsible for ensuring it is hermetic.
Yes, I started out by using genrule before going down the custom rule path. I was able to properly trigger bazel to generate the interface bindings but then lost the thread when trying to properly configure the output files to act as proper dependencies to other code (ie. generate C++ headers and libraries that could then be properly listed as dependencies by client code.)
I think part of the issue was that the generated code from the 3rd party tool did not align with the Bazel expectations about code layout (ie. paths relative to WORKSPACE). Likely solvable but not in the amount of time I had available to dedicate to the effort.
My experience with Bazel at a startup (also used Blaze at Google):
The good:
- Amazing for Go backends. I can push a reproducible Docker image to Kubernetes with a single Bazel command. I can run our entire backend with a single command that will work on any developer's box.
- Amazing for testing. All of our backends tests use a fully independent Postgres database managed by Bazel. It's really nice not having to worry about shared state in databases across tests.
- We can skip Docker on macOS for development which provides on the order of a 10x speedup for tests.
- BuildBuddy provides a really nice CI experience with remote execution. Bazel tests are structured so I can see exactly which tests failed without trawling through thousands of lines of log output. I've heard good things about EngFlow but BuildBuddy was free to start.
- Really nice for schema driven codegen like protobufs.
The bad:
- Bazel is much too hard for TypeScript and JavaScript. We don't use Bazel for our frontend. New bundlers like Vite are much faster and have a developer experience that's hard to replicate with Bazel. Aspect.dev is doing some work on this front. One large hurdle is there's not automatic BUILD file dependency updater like Gazelle for Go.
- Windows support is too hard largely because most third party dependencies don't work well with Windows.
- Third party dependencies are still painful. There's ongoing work with bzlmod but my impression is that it won't be usable for a couple of years.
- Getting started was incredibly painful. However, the ongoing maintenance burden is a few hours per month.
> Bazel is much too hard for TypeScript and JavaScript. We don't use Bazel for our frontend. New bundlers like Vite are much faster and have a developer experience that's hard to replicate with Bazel. Aspect.dev is doing some work on this front. One large hurdle is there's not automatic BUILD file dependency updater like Gazelle for Go.
I built this[0] for the Bazel + yarn setup that we use at Uber. We currently manage a 1000+ package monorepo with it.
> I can run our entire backend with a single command that will work on any developer's box.
Curious wouldn't `go run` give you the same? pure go code is supposed to be portable, unless you have cgo deps I guess?
> I can push a reproducible Docker image to Kubernetes with a single Bazel command.
That's definitely an upside over what would otherwise would probably default to a combination of Dockerfiles and scripts/Makefiles, does it worth bringing in the massive thing that is Bazel? depends I guess.
I'm curious: would you say your experience with golang IDEs / gopls is degraded? did you do anything special to make it good? I often feel like development is more clunky and often I just give up on the nice-to-haves of a language server e.g often some dependencies in the IDE aren't properly indexed, I can probably get Bazel to do some fetching, reindex and get it working, but it will take 3-4 minutes and I just often choose to live with the thing appearing as "broken" in the IDE and getting less IDE features.
We use protobufs and pggen. Bazel transparently manages the codegen from proto file to Go code.
> would you say your experience with golang IDEs / gopls is degraded?
Yes, that's our biggest pain point with Go and Bazel. I haven't been able to coax IntelliJ into debugging Bazel managed binaries. To enable IntelliJ code analysis, we copy the generated code into the src directory (with a Bazel rule auto-generated by Gazelle) but don't add it to Git.
I've tried the IntelliJ with Bazel plugin a few times but I've always reverted back to stock IntelliJ.
> Amazing for testing. All of our backends tests use a fully independent Postgres database managed by Bazel. It's really nice not having to worry about shared state in databases across tests.
I would love to know more about this. We use bazel and Postgres, but there is no Postgres isolation between tests currently which is very predictably a large source of flakyness, and also a blocker for increasing test parallelism.
I’ve been scheming on perhaps using pg_tmp orchestrated outside of bazel, but if there’s a way to do everything in bazel that would make the transition way easier.
Not OP, but we start small postgres instances in our Python tests with some py.test fixtures. The actual postgres binaries are provided by Bazel, so nothing relies on outside installations.
That said, this requires one postgres instance for each test, which is pretty heavy. So we keep our tests pretty coarse at the Bazel level (i.e. one Bazel test is actually 10+ Python tests). I've thought about other approaches, such as using a Bazel persistent worker to wrap test execution and manage a single Postgres instance with empty databases created from templates, but never got around to trying anything.
In addition to the techniques in the linked comment, I'm currently investigating managing the entire Postgres data dir as a Bazel output directory. That way, we avoid executing the schema for every test. Instead, we'd start a new Postgres cluster against the existing data dir without re-executing the schema or creating from a template DB.
I migrated a monorepo including C++, Kotlin, Java, TypeScript and Python to Bazel. It's no small feat, and the DX varies widely across languages and platforms, but it's absolutely worth it. `bazel test //...` from a fresh clone builds & tests everything including gRPC/protobuf code generation, custom code generation, downloading packages, toolchains and linters, dependencies between langs, test suites with coverage across multiple languages.
Integration testing is a breeze through data dependencies. The reproducibility guarantees means we can reference container image SHAs in our Terraform and if the image didn't change the deploy is a no-op.
Bazel is an outstanding build system that handily solves a lot of practical problems in software engineering. Not just "at scale". Practical problems at any scale.
Regardless of whether you should use Bazel or not, my hope is that any future build systems attempt to adopt Bazel's remote execution protocol (or at least a protocol that is similar in spirit):
Tools like Earthly (and other BuildKit HLB frontend languages) will help more teams get some of the main benefits of Bazel with a lower bar of complexity. If you already know how to write Dockerfiles and Makefiles, you can write Earthfiles without much additional learning necessary. It provides a build that is repeatable, incremental, self-contained, never dirty, shared-cacheable, et cetera.
There’s an issue I reported (along with a proof of concept fix) over 4 years ago, that has yet to be fixed: building a mixed source project containing Go & C++ & C++ protocol buffers results in silently broken binaries as rules_go will happily not forward along the linker arguments that the C++ build targets (the protobuf ones, using the built in C++ rules) declare.
Not very confidence inspiring when Google’s build system falls over when you combine three technologies that are used commonly throughout Google’s code base (two of which were created by Google).
If you’re Google, sure, use Bazel. Otherwise, I wouldn’t recommend it. Google will cater to their needs and their needs only — putting the code out in the open means you get the privilege of sharing in their tech debt, and if something isn’t working, you can contribute your labor to them for free.
Does protobufs rely on static initialization or did you link the wrong bug? That would be extremely strange because there’s generally a disallowance of any / all non trivial static initialization in library code Google-wide, especially for protobufs.
Bazel is a great tool, but as mentioned in the article the support from Google is rather limited. If bazel ever got to the level Terraform is with providers it could really take off. Something I'd love to see is a .bazel_mod file where you could pull in other bazel workspaces (similar to go.mod) and reference packages within those workspaces.
I haven't used Turborepo, but I have really liked wireit. wireit is from the Lit team at Google. They understand both Bazel and JS. As I understand, wireit is their attempt to make the Bazel niceties JS-native.
Would you mind explaining why? I use Nx and it's been good, not perfect tho, so at this point I'm very skeptical about toolchaining solutions that claim they 'just work'
Last time I looked at Nx it was very opinionated and I couldn't get it to work with an existing repo. In contrast I got running with turborepo in about 15 minutes in a fairly complex workspace project.
Its very flexible, but in this case the flexibility is simple: you specify your input globs, output globs and task dependencies for each package.json command you want it to support and it will use that as the basis to detect cache invalidation.
The thing is, something like Bazel only pays dividends on a large complex project. When you're starting from scratch it's unlikely you'll have any problems that Bazel helps to solve, but you'll be spending a lot of time on the ceremony of setting it up.
One big gotcha that folks need to appreciate is that you don't get the full power of bazel just by getting baseline bazel to work and build things. You need to really be sensitive to your physical dependencies and not lump similar code together just because they play a similar role and are convenient to place in the same file rather than having that same source sharded out to distinct files.
You need to partition source into all its relevant pieces and lump no more functionality in that source than is needed by clients. Bazel encourages the most minute fine grained resolution of dependencies because if you do that and manage it right you unlock the power of the cache. Once you successfully partition all the bits of software to their segregated sections of the build graph DAG then you're positioned to really leverage the test result cache and can start bypassing huge expensive tests that are completely irrelevant to the body of code you're developing on.
Bazel is not just a build toolchain it is a test toolchain as well. The problems it solves with tests with respect to running only relevant & applicable tests to the changes being pushed is what many people have dreamed about before or came up with some crude solution to approximate the dependencies. If my test did not print out `(cached)` I know something in the dependency graph was disturbed in some way that may be completely not obvious to me, especially in a gigantic repo.
Bazel query is also worth mentioning to visualize dependencies with its xml output and which can also enforce architectural barriers asserting empty `somepath` queries between two targets that should not meet in the middle somewhere.
It would be neat one day if bazel integrates in some semantic logical level of dependency checking beyond the physical but that may be too expensive of an operation.
Kinda tangential, but I still can't believe there are "Modern" CMake books hundreds of pages long. I still end up using it sometimes, but I have no confidence in the rules I write, specially when compared to good old Visual Studio solutions or my favorite tool which is Premake.
+1 for gradle, not enough people realise it's a very general build system (not just for java / android).
The caching is indeed very good if you set it up properly.
In the hands of a level 5 gradle wizard it can also be extended to do ANYTHING, with the usual caveat that you may experience regrets when your gradle wizard leaves.
Back in 2016 or so, I was in need of a more capable build system to replace a combination of disorganized cmake and XML IAR project files. Having come from Google, I briefly considered bazel, but it just didn't seem mature enough to build on top of. I went with SCons, and built a bazel-like abstraction layer on top of it. My goal was to abstract away SCons in 90% of use cases. It's still in use 6 years later, and the company's engineering team has grown by at least a factor of 10. People love to complain about build infrastructure, but it's held up pretty well.
as an angular developer, Bazel was promised to replace webpack behind the scenes and suddenly everything would get fast and all our problems would be solved. Months... Years went by until eventually it became an opt-in solution and I guess they just gave up trying to use it.
> Bazel, as Blaze, was created to solve the challenges of Google’s code base, so it’s no surprise that it works well there and that people who’ve spent a lot of time using it make a strong case for everyone using it.
I've worked at Google for a long time. Every time I get a chance to work on something outside Google's codebase, it's like a breath of fresh air.
I wouldn't recommend Bazel or Blaze to anybody. Sorry to the people who wrote it.
Under "When to Use Bazel - Jason’s Opinion", that is not a particularly good description of how builds (especially code generation) worked at most of Amazon. At least he sort of gestures at the non-universality of his experience by mentioning his experience was in Prime Video and only for 8 months. I have no trouble believing that various pockets of the company found various different wrong ways to do things, I've seen some of those wrong ways myself. But taking a dependency on shared code (for better or for worse) rather than copying something yourself should not have been hard or infrequently done, and wouldn't be for most people using the standard tools. Generating code and having multiple downstream builds depend on that same generated code (e.g. the output artifact in the form of a .jar, in the case of Java) should also be very straightforward, though it was also possible (if a bad idea) for each consumer to run the same generator.
Builds at Amazon were definitely not problem-free, but they were different problems that don't really resemble that description.
I looked at Bazel ~5 years ago for a C++, Scala, JS, Python codebase and it looked like the perfect solution, but there seemed to be a lot of breaking changes in Bazel at the time so I passed in the end.
Then I started a project this year that will be a Rust, Swift, Kotlin, JS codebase (maybe C# too if Rust for Windows isn’t stable enough) and I had a look at Bazel again, but already the official tutorial from last year didn’t work on the latest stable release so I gave up on it again. Anyone had similar experience and pushed through? Is it worth it?
I currently have a bunch of Python scripts and will most likely end up with Gradle which is ok but not great in my experience.
Bazel doesn't allow targeting a lot of platforms (especially embedded) from Rust, even when the Rust ecosystem supports these targets. Something is off with its design if new work needs to be done for every platform that's already available behind an interface that's as consistent as what rustc gives.
I remember struggling with Bazel and Tensorflow for months!
I just went to see how things are and it seems that things have not changed much in years, e.g.: https://github.com/bazelbuild/bazel/issues/14639
This is kind of project that you won't probably need soon, unless you are really being paid extra money for the suffering.
Honestly, even considering this question for any service/codebase that is under 1,000 developers contributing daily commits, or 10,000,000 users is just a vanity project.
I use bazel at work and it does make cross language work very easy. I mostly write C++ so it's one of the few good options out there. But I've also used CMAKE with vcpkg for personal projects and found that to be a really good option. Vcpkg is really the best package manager for C++ right now and seems to work seamlessly on Windows, too. Hopefully bazel's bzlmod can take off and get as many libraries as vcpkg.
I mostly do embedded C and my favorite use of bazel not mentioned in the article is the near-seamless swapping of toolchains. It makes unit testing very pleasant.
Only tangentially related, but can someone give me the pitch for Earthly? I think I'm just missing something basic.
Our CI consists of tests, builds and deployments (all docker containers) that are kicked off in bitbucket pipelines. Is there some way that either our local development or CI builds can be improved by using earthly? Appreciate any info!
It took me some time to start understanding bazel properly. But after that using it is very easy and enjoyable (especially inside a monorepo). But learning curve is getting very steep very fast outside of simple BUILD rules.
Edit: enjoyable to a point that i half considering doing contracts for bazel based build systems.
So the answer IMHO to “when to use Bazel” is “never” :)