mirror: new command to make a mirror of a repo

This creates a mirror of a full repo by downloading all files listed in
the index, and the ones that are generated based on that data, e.g. icons
of different resolutions.  This could be useful for setting up mirrors of
small repositories, instead of having to learn and manage rsync or
something else for mirroring.  This just needs a working repo.

It uses wget in a batch mode with the aim as being as efficient as
possible.  wget mirroring over HTTP is always going to be less efficient
than rsync, but it shouldn't be so bad since it uses --continue to check
whether it has already downloaded a file.  I suppose it could be extended
to use ETags for a little more efficiency.

I developed this creating a test mirror of f-droid.org, which is now a bit
ironic, since I added a specific check to prevent people from using this
on f-droid.org.
This commit is contained in:
Hans-Christoph Steiner 2017-11-29 12:40:04 +01:00
parent 5607ccdc41
commit 2162703a1a
3 changed files with 149 additions and 0 deletions

1
fdroid
View file

@ -48,6 +48,7 @@ commands = OrderedDict([
("btlog", _("Update the binary transparency log for a URL")),
("signatures", _("Extract signatures from APKs")),
("nightly", _("Set up an app build for a nightly build repo")),
("mirror", _("Download complete mirrors of small repos")),
])