Musings about build systems

I was just hacking the other day our somewhat horrible hairball of an ant build to try to mold it to my understanding of how a build should work when I had a sort of an epiphany about why on earth I dislike Ant so much.

Apart from the syntax (and I think it is universally agreed by now, that XML is a horrible, horrible language for scripting like tasks), what hit me is that while ant is okay for simple one-shot single project builds, it really falls on it's knees as soon as you start building complex modularized systems where you need some sort of dependency management.

Now, before I go on, I must mention that I have heard of alternatives like make, Maven, Buckminster, b3 and the ilk, but I really don't know much about any of those (well, by the time I started to write this piece, I had taken a peek at Buckminster). I am not really what you would call a "build person" - I'm using Ant, because it's what comes with most of the projects I've been involved in...

For simple and mostly declarative stuff, Ant is great. It's easy as long as you don't have to do any of the "smart" stuff and even with some of the smart stuff, it works fine.

Where I run into the wall is where one build file goes out and mucks around with something from another build. Like for example taking a jar out of another build and putting it on a classpath or offering it up for some deployment operation for copying. The way you do this today with ant, is you call a target from another build (and, my god, is the syntax cumbersome there if all you want is to call a specific target in a specific ant file) and after that task finishes point a property at the location of a resource that task was supposed to produce. And cross your fingers, that you've got it all right and the other ant file has not been changed to put that resource actually in some other place or saved with another (arguably better) name.

It all kind of messy and yucky and brittle in so many ways, that I just shudder every time I have to think about diving into one of those ant build files...

So I was hacking away at yet another ant build involving many dependencies as I suddenly realized, that what I was actually looking for was a sort of interaction that I've come to expect from a programming language.

The thing I needed, was a build scripting model where each build script is an equivalent of an object in a programming language, it's properties are ... well, properties of the build script object and targets are methods on that object that always return some sort of resource set structure.

In this fantasy build language of mine, a build target could call to other build targets and get a set of resources they've produced as a return value to be used as it pleases. These calls could be chained together, optionally applying various filters to the the resultsets of one build step before passing them to the next.

The syntax of the build script would have to be lightweight (no XML here), easy to follow, clean and mostly declarative with a touch of functional features for performing some of the advanced operations.

So far, the closest thing I have seen to match my imaginary build scripting language is b3, but that one is not quite ready yet. In some ways Buckminster's model also matches what I had in mind, but that is negated by the fact that it uses XML as an underlying language for describing the components, queries and it feels a bit cumbersome with all those separate types of artifacts thrown around...

Comments

Henrik Lindberg said…
Hi Roland, love to hear more about your ideas for a "dream build language" - this is what b3 is supposed to be. Check out documentation and stuff at http://www.eclipse.org/modeling/emft/b3 if ypu have not already done so.

Next milestone for b3 will have a very nice language frontend to running buckminster build.
- henrik
Henrik Lindberg said…
Regarding Buckminster and XML - note that there are special editors - you don't have to work with the XML directly unless you really want to.
Luolong said…
Yes, Henrik, I actually started this blog post about 6 months ago (is that so long already?) and at that time I was only somewhat familiar with Buckminster. Someone pointed out to me the b3 project so I checked it out.

Now that I have followed the b3 development much closer for a long time (and read the "Penultimate guide" book), I must admit, b3 comes really really very close to the vision I had.

My original idea was actually much less ambitious that those b3 addresses (e.g. b3 also handles materialization/provisioning and publishing aspects via p2 and VCS support, that I completely ignored in my imaginary build system)
Luolong said…
As for Buckminster special editors - last time I looked, the forms based editor was only available for CQUERY and CSPEC files, RMAP editor was basically a glorified XML editor and CSPECX did not have an editor at all.

BOM editor/viewer was also quite nice as a visualization tool showing all the dependencies, but to the extent I am aware - you could not use it to edit the BOM specification.

And also, in some situations - at least for me - seeing textual representation of the configuration offers much better overview, but the XML is in some ways less than ideal format for human consumption.
Anonymous said…
Roland, this has changed in 3.6. WE now have real (EMF based) editors for both the RMAP and the MSPEC. BOM's are not intended to be edited.
Luolong said…
Thanks Thomas - I'll take another look at it as soon as 3.6 gets released.
Luolong said…
Too bad, that active/visible progresso on b3 has kind of withered to almost nothingness...
Henrik Lindberg said…
It is not dead - it is just resting :)
You are welcome to pitch in ;-)
Luolong said…
Well, I'd say it is in a "deep hibernation" atm.

And I would most definitely love to pitch in... It's just that there are soooo many interesting things to do and only one of me :)

Popular posts from this blog

Comparing Jenkins vs TeamCity - Part 1

Comparing Jenkins vs TeamCity - Part 2