limit memory usage on large repositories using GIT

334 views

I've been using git for some time now, and host my remote bare repositories on my shared hosting account at Dreamhost.com. As a protective feature on their shared host setup, they enact a policy that kills processes that consume too much memory. This happens to git sometimes.

By "sometimes" I mean on large repos (>~500MB), when performing operations like git gc and git fsck and, most annoyingly, when doing a clone. It seems to happen in the pack phase, but I can't be sure exactly.

I've messed around with the config options like pack.threads and pack.sizeLimit, and basically anything on the git config manpage that mentions memory. I limit all of these things to 1 or 0 or 1m when applicable, just to be sure. To be honest, I really don't know what I'm doing ;)

Oddly enough, I'm having trouble reproducing my issue with anything but git fsck. Clones were failing in the past, but after a successful git gc, everything seems to be ok(?)

Anyway, I'd like some advice on what settings limit memory usage, and exactly how to determine what the memory usage will be with certain values.

posted Jul 10, 2013 by anonymous

Looking for an answer? Promote on:

Facebook Share Button

Twitter Share Button

LinkedIn Share Button

Similar Questions

+2 votes

Http Git hosting & Software repositories on Git server

I'm trying to build Git hosting server for a group of people.

I see there are Git hosting that give ability to users to create and manage their repositories. I want to give user ability to create and manage their repository only using http. But in the instruction of http protocol, user with write privileges must login in order to create the repositories.
Is it can work with http protocol or it must be with ssh?
I would like get recommendation about a software if exist. Is there any software, I can use, that give the ability to see weby and/or visually view status of repositories on the Git server, without cloning or pulling a snapshot from the git server to my local machine and without login to the git server?

+1 vote

Best practice of organizing the GIT repositories?

I want to understand the best practice of organizing the GIT repositories. Let's say:
1. We are a large bank with many line of businesses and tons of application.
2. Each LOB has a large number of applications.

How many repositories shall I use? How do we organize the applications inside one repository? Any reference?

+1 vote

How do I change where the (server) repositories live in GIT?

I have git version 1.8.4.msysgit.0 installed. If I run this as a daemon, where are the repositories located? (i.e. where do i do a mkdir foo.git; cd foo.git; git init --bare?)

How do I change where the (server) repositories live?

+2 votes

Any experience with git-p4 to interact with Perforce repositories?

I'm a big Git fan and had the chance to "convert" some colleagues to the benefit of Git over Subversion, leading the conversion process and coaching them for their day-to-day use.
I'm now working in a place where the VCS is Perforce (sigh!). I must admit the cultural gap is non-negligible.
That being said, I did a quick search and found that there is git-p4 to use Git as a "client" to Perforce. Before getting that to work on my workstation, I'd like to know if anybody ever used that tool? If so, what kind of experience have you had with it? Are there any pitfalls, etc...
I'll probably get on setting that toolset in the next few days (next week) and will update this topic with any relevant info.

0 votes

Git, remote bare repositories, Dropbox backed

This may have been asked so many times. But I may not understand the solution.

I'm following: http://git-scm.com/book/en/Git-on-the-Server-Setting-Up-the-Server

to create a Git server (remote bare repository). The host containing the Git server has got Dropbox installed. I'd like Dropbox to be my backup of the remote repository.

Here's my problem.

Dropbox files is located /opt/Dropbox. Repos are located /opt/Dropbox/repos. I've created a soft-link from /repos to /opt/Dropbox/repos.

I'd like to set origin to this remote repository.

git remote add origin git@gitserver:/repos/someproj.git
git push origin master

However, it seems the soft link, i.e. /repos, won't allow me to write to the bare repository. The error I receive is:

~/Documents/workspace/test-git:11:35$ git push origin master
fatal: '/repos/someproj.git' does not appear to be a git repository
fatal: The remote end hung up unexpectedly
~/Documents/workspace/test-git:11:35$

Can anyone help with this scenario?

...