Limiting what branches I track from an upstream Git repository

February 5, 2021

I track a number of upstream Git repositories for software we use or that I'm interested in, where by 'track' I mean that I keep a local copy and update it periodically. I've been growing more and more unhappy with how noisy this process has been getting, so recently I did some work on making this tracking quieter. Sadly this left me with one remaining pain point, the repository for Grafana.

Grafana's repository is unusual because the Grafana developers work in a profusion of short lived feature branches that appear in the main repository. When I do a 'git pull' of the Grafana repository after only a few days, I get a shower of newly created branches and another shower of recently deleted remote branches. As far as I can see, 'git fetch' has no particularly good way to suppress this output through a .git/config option; the most you can do is run 'git fetch' entirely quietly.

In order to deal with this, I've now switched to tracking only a limited range of upstream branches. Ideally I would only need to track one upstream branch, the main one, but in practice the Grafana developers create a separate branch for each significant release and I also want to track those branches, just in case. Because of this need to track additional branches combined with Grafana's branch naming practices, the result is a little messy and imperfect.

The normal Git refspec for an upstream 'origin' repo is, in .git/config format:

[remote "origin"]
  fetch = +refs/heads/*:refs/remotes/origin/*

(The plus sign is a bit magical but is probably okay here.)

To track only a limited subset of branches, we need to change this to one or more 'fetch =' lines that are more specific. Fortunately, you can have multiple 'fetch =' lines; unfortunately, Git's support for wildcard matching here is relatively limited, so we can only do so much given Grafana's branch naming scheme. What I use is:

[remote "origin"]
  fetch = +refs/heads/master:refs/remotes/origin/master
  fetch = +refs/heads/v*:refs/remotes/origin/v*

Grafana's version branches are called things like 'v7.4.x', but it also has a few other branches that start with 'v' and we can't use a wildcard match of 'v[1-9]*'. I could list out all of the current major versions that I care about, but that would leave me having to manually change things every so often (such as when 'v8.0.x' is created, which the Grafana people are working on). For now I've decided to accept a few extra branches (currently four); if more unwanted 'v*' branches show up, I may change my mind.

(More recent versions of Git than any version I have access to allows for negative refspecs in 'git fetch', and even documents wildcards a bit in the git-fetch manpage.)

This change of 'fetch =' lines limits what will be pulled in the future, but it doesn't do anything to prune the unwanted feature branches I currently have (and I'm not certain git will still notice deleted upstream branches, since I've probably told it to not look up information on them). To get rid of them, we need to manually delete the local 'origin/<...>' branches using 'git branch -d -r origin/<...>'. I generated a list of branches to remove by starting with 'git branch -r' and then filtering out remote branches I didn't want to remove, then used a big shell for loop to do the work.

Possibly there are better Git ways to do this. I am somewhat of a brute force person when it comes to Git, because I don't know the ins and outs; when I want to do something new, I make a strategic expedition into the manpages and Internet searches, looking for a relatively obvious answer.

PS: Grafana could make my life easier here by putting all of their release branches in a specific namespace, like 'release/v7.4.x'. Then I could have a fetch spec of 'refs/heads/release/*' and it wouldn't match any other feature branches, because of course you wouldn't put feature branches in release/.


Comments on this page:

The simplest way is to just do

git branch -Dr `git for-each-ref refs/remotes/origin --format='%(refname:short)'`
git fetch

which just blows away all of the remote branches from origin and recreates the ones you want. Since deleting a ref doesn’t delete the object the ref points to, that fetch will be a lightweight operation.

The only reason not to do this is if you wanted to preserve the reflogs for the branches you care about (which won’t matter to anyone who doesn’t know what that means).

By dozzie at 2021-02-06 08:38:08:

@cks:

 [remote "origin"]
   fetch = +refs/heads/*:refs/remotes/origin/*

(The plus sign is a bit magical but is probably okay here.)

It's not magical (in my opinion) and it definitely is necessary here. Plus sign, as you more or less know already, makes the git fetch invocation that's running behind the scenes add --force option (only for selected refs, of course).

When you're tracking content of a remote branch, git uses two steps approach: the first step is to fetch and store all the remote state locally (refs/remotes/${name}/*), and then uses this local image of remote state to make a usable branch for actual work (refs/heads/*).

For your own local things plus sign of course is potentially dangerous (and the same for push destinations), but for local image of a remote state, you want it updated even if it's not a fast-forward update.

By Simon at 2021-02-06 14:13:28:

Grafana's version branches are called things like 'v7.4.x', but it also has a few other branches that start with 'v' and we can't use a wildcard match of 'v[1-9]*'.

You could just expand the wildcard manually in this case:

[remote "origin"]
  fetch = +refs/heads/master:refs/remotes/origin/master
  fetch = +refs/heads/v1*:refs/remotes/origin/v1*
  fetch = +refs/heads/v2*:refs/remotes/origin/v2*
  fetch = +refs/heads/v3*:refs/remotes/origin/v3*
  fetch = +refs/heads/v4*:refs/remotes/origin/v4*
  fetch = +refs/heads/v5*:refs/remotes/origin/v5*
  fetch = +refs/heads/v6*:refs/remotes/origin/v6*
  fetch = +refs/heads/v7*:refs/remotes/origin/v7*
  fetch = +refs/heads/v8*:refs/remotes/origin/v8*
  fetch = +refs/heads/v9*:refs/remotes/origin/v9*

Not nice but should work for you use case.

Written on 05 February 2021.
« There are limitations to what expendable addresses can help with
Talkd and 'mesg n': a story from the old Unix days »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Feb 5 22:39:11 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.