How to use git to store large files without keeping track of versions?

Question

I have some data files that need to be stored along with source code. These data files are large, but I don't need to keep their versions. I only need to keep the versions of the source code.

git-annex is mainly for large files with version. Therefore, it is not suitable for my situation.

Does anybody know whether there is a way to use git to manage source code (with version) as well data files (without version)?

Answer 1

Have a look at git bup which is a git extension to smartly store large binaries in a git repo.

You'd want to have it as a submodule but you won't have to worry about the repo getting hard to handle. One of their sample use cases is storing VM images in git.

Technique 1: sparse checkout
A mild help to the binary assets problem is sparse checkout (available since Git 1.7.0]). This technique allows to keep the working directory clean by explicitly detailing which folders you want to populate. Unfortunately it does not affect the size of the overall local repository but can be helpful if you have a huge tree of folders.

1. Clone the full repository once: git clone
2. Activate the feature: git config core.sparsecheckout true

3. Add folders that are needed explicitly, ignoring assets folders:

   echo src> .git/info/sparse-checkout
   Read the tree as specified: git read-tree -m -u HEAD

After the above you can go back to use your normal git commands, but your work directory will only contain the folders you specified above.

Technique 2: Use of submodules
Another way to handle huge binary asset folders is to split those into a separate repository and pull the assets in your main project using submodules. This gives you a way a way to control when you update the assets. See more on submodules in these posts: core concept and tips and
alternatives.

If you go the way of the submodules way you might want to checkout the complexities of handling project dependencies, since some of the possible approaches to the huge binaries problem might be helped by the approaches I mention there.