Mauricio Scheffer Mauricio Scheffer - 4 months ago 49
Git Question

Git shallow submodules

Is it possible to have shallow submodules? I have a superproject with several submodules, each with a long history, so it gets unnecessarily big dragging all that history.

All I have found is this unanswered thread.

Should I just hack git-submodule to implement this?

Answer

New in the upcoming git1.8.4 (July 2013):

"git submodule update" can optionally clone the submodule repositories shallowly.

See commit 275cd184d52b5b81cb89e4ec33e540fb2ae61c1f:

Add the --depth option to the add and update commands of "git submodule", which is then passed on to the clone command. This is useful when the submodule(s) are huge and you're not really interested in anything but the latest commit.

Tests are added and some indention adjustments were made to conform to the rest of the testfile on "submodule update can handle symbolic links in pwd".

Signed-off-by: Fredrik Gustafsson <iveqy@iveqy.com>
Acked-by: Jens Lehmann <Jens.Lehmann@web.de>

That means this works:

git submodule add --depth 1 -- repository path
git submodule update --depth -- [<path>...]

With:

--depth::

This option is valid for add and update commands.
Create a 'shallow' clone with a history truncated to the specified number of revisions.


atwyman adds in the comments:

As far as I can tell this option isn't usable for submodules which don't track master very closely. If you set depth 1, then submodule update can only ever succeed if the submodule commit you want is the latest master. Otherwise you get "fatal: reference is not a tree".

That is true.
That is, until git 2.8 (March 2016). With 2.8, the submodule update --depth has one more chance to succeed, even if the SHA1 is directly reachable from one of the remote repo HEADs.

See commit fb43e31 (24 Feb 2016) by Stefan Beller (stefanbeller).
Helped-by: Junio C Hamano (gitster).
(Merged by Junio C Hamano -- gitster -- in commit 9671a76, 26 Feb 2016)

submodule: try harder to fetch needed sha1 by direct fetching sha1

When reviewing a change that also updates a submodule in Gerrit, a common review practice is to download and cherry-pick the patch locally to test it.
However when testing it locally, the 'git submodule update' may fail fetching the correct submodule sha1 as the corresponding commit in the submodule is not yet part of the project history, but also just a proposed change.

If $sha1 was not part of the default fetch, we try to fetch the $sha1 directly. Some servers however do not support direct fetch by sha1, which leads git-fetch to fail quickly.
We can fail ourselves here as the still missing sha1 would lead to a failure later in the checkout stage anyway, so failing here is as good as we can get.


MVG points out in the comments to commit fb43e31 (git 2.9, Feb 2016)

It would seem to me that commit fb43e31 requests the missing commit by SHA1 id, so the uploadpack.allowReachableSHA1InWant and uploadpack.allowTipSHA1InWant settings on the server will probably affect whether this works.
I wrote a post to the git list today, pointing out how the use of shallow submodules could be made to work better for some scenarios, namely if the commit is also a tag.
Let's wait and see.

I guess this is a reason why fb43e31 made the fetch for a specific SHA1 a fallback after the fetch for the default branch.
Nevertheless, in case of “--depth 1” I think it would make sense to abort early: if none of the listed refs matches the requested one, and asking by SHA1 isn't supported by the server, then there is no point in fetching anything, since we won't be able to satisfy the submodule requirement either way.

Comments