How to implement "selective checkout, merge and build use case"

Question

We have a rather complex application with a large number of source files organized in folders where one folder-tree corresponds to one module. We're currently using SVN in order to version control and are looking at ways to use SVN for "selective checkout, merge and build" of selected modules. Each module has a specific setup of config-, documentation- and make-files making them "semi autonomous". When I say "semi autonomous" I refer to the fact that every module relies on one *core module* which provides the base functionality (foundation) needed by all the other modules.

The reason why we would like to be able to perform "selective checkout, merge and build" is that we need to be able to deliver a specific set-up of modules at a given time. That checkout would result in a checkout of the "core" module together with the other selected modules. After the checkout has been performed we'd like to be able to perform the following "post checkout" operations:

1. merge the modules config-fragments into a single config-file 

2. merge the modules doc-fragments (in Latex) into a single doc-file 

3. execute the modules make-files in order to build them 

Is there a recommended best practice to accomplish this using SVN or are there any complementing tools which could help us out here?

Answer 1

Some people use svn:externals for this kind of thing, which can eventually turn into a disaster because it doesn't scale and only records relationships in one direction. You can't easily ask: "Which modules use a particular module M?" But you can ask "Which modules does module M depend on?" Since svn:externals use a set of segregated working copies they don't mix well with 'svn merge' -- you can't atomically merge a change set which affects multiple modules.
I've seen this approach both succeed and fail. Generally, I guess the more aware everyone is about the limitations of svn:externals the better it can be made to work.

If you really need this to scale over time and project size, you should consider a dependency management tool and leave the version control tool out of this. Unfortunately these tools tend to be very domain specific. For the Java world there's Maven, for embedded Linux there are several native package managers (rpm, deb, etc.), for Dotnet stuff there's NuGet, on Unix you have Makefiles... The best choice depends on your domain.

It sounds like you're already using make. Have you considered writing a Makefile which ties things together in a way you need and manages the dependencies? This may not be the answer you were looking for but it might keep things simple in the long term. You could take an intense look at how Linux and *BSD make-based build systems manage to target a wide variety of platforms from a single source tree to find some inspiration.