The life and death of TarGit

Initially written starting 23:22 UTC on the 11th of October, 2022 a.d. by Amelia Bjornsdottir (she, they)

This article is a short story, in five parts, about the genesis, lifetime, decline, death, and future of a hack.

Genesis

Some weeks or months ago, we don’t remember which, we wanted to share the source code of UmbrellixSSG with a friend. We didn’t have Git HTTP dumb protocol set up at that point, so we decided to tarball the bare Git repository and post it in the Web server’s document root. This worked well and was good, but didn’t scale because it was manual.

This was the genesis of the infamous paragraph on the page for UmbrellixSSG:

Temporarily, you can download the bare git repository, manually updated so sometimes out of date, from this website as a zstd compressed tar archive (/software/ussg/ussg.tar.zstd). Create a directory ‘ussg.git’, then run ‘zstdcat ussg.tar.zstd |tar -Cussg.git -xvvf -’, then ‘git clone ussg.git ussg’.

which described the method of downloading, extracting, and checking out a TarGit repository. This paragraph was then changed to:

You can download the bare git repository, now automatically updated, from this website as a zstd compressed tar archive (/~git/ussg.tzst). Create a directory ‘ussg.git’, then run ‘zstdcat ussg.git.tzst |tar -Cussg.git -xvvf -’, then ‘git clone ussg.git ussg’.

which also describes the method of downloading, extracting and checking out a TarGit repository manually.

Lifetime

A few days ago, we started looking at ways to cause Gitolite to trigger automatic tarball creation, to ~git/public_html, and came up with a gruesome hack shell script to be run from a Gitolite post-git trigger. This hack (which you can check out by running git clone https://git.umbrellix.net/u/ellenor/gitolitehacks; it is the file named gitolite/pubtar) worked well enough and was alright, but we realized it wasn’t going to scale well when we started trying to write TarGit-Clone, a script to automate that hack.

Decline

On the prompting of a blog article on TheSparkTree entitled Git Mirror Anywhere using the Dumb Http Protocol, to said script, we added an invokation of git update-server-info. This was the beginning of the end for TarGit. Now, someone with a TarGit repository could just so easily update it without downloading the whole repository again by running git pull https://git.umbrellix.net/the-git-repository-they-had.

Death

At that point, TarGit was more or less obsolete. It was no longer necessary to use it, and it’s effort over and above just cloning the dumb-protocol HTTP repo.

Future?

A future? For a dead hack? You had better believe it.

TarGit reduces the number of HTTP requests to initially check out a repo from however many Git makes, to just two - the repo URL/targit-url (a plaintext URL containing the TarGit URL), and the TarGit URL itself. This should make cloning moderately-sized repositories on Umbrellix' git subsite much quicker.

TarGit-Clone will likely switch from using a .tar.zstd file link directly, to using a Git repository with the targit-url in its root. If that’s empty or 404 it will just pass through to git clone. It is also going to be a valuable learning experience for me in writing a hopefully-prj-compatible slashpackage package (see our slashpackage cue card, for more information). Shell libraries from it will make it into libsuitcase and into our slashpkg tools.

You hear that, slashpkg with 3 letter pkg? Keep that word in your memory banks. It’s already come up a few times (it’s the subject of RFC 1) on this website, and it will come up again, as I expand my thinking and horizons on it ad come up with a sustainable build system for it. It’ll come up again, reader. But until then…

puts on instrumental semi-hopeful allegro music

See you soon!

Oh, I almost forgot! Stagit!

Stagit fork

So, we installed Stagit, somewhat incorrectly, at https://git.umbrellix.net., but there’s a small problem - it does not integrate with the rest of the website at all. This is not sustainable for us.

We plan to create, and release, a fork of Stagit, that is able to use UmbrellixSSG as a filter, enabling better integration with the rest of the website. The way we will do this is not entirely clear to us, although I suspect that we’ll likely abuse sh -c in an exec*()-family call. We also plan to write a suitable recursive stagit-index, as the stagit-index currently available to us is woeful and doesn’t support subdirectories.