If `fdroid server update` is run with config that includes an archive, but
the 'archive' subdir does not exist, create it. This mirrors the code that
is in `fdroid update`. Seems to trivial to move to common.py.
To support a fully offline build/signing machine, there is the "local copy
dir". The repo is generated on the offline machine and then copied to a
local dir where a thumb drive or SD Card is mounted. Then on the online
machine, using `fdroid server update --sync-from-local-copy-dir` allows
the whole server update process to happen in a single command:
0. read config.py on online machine's repo
1. rsync from the local_copy_dir to the current dir
2. copy to serverwebroot, awsbucket, etc.
In `fdroid server update`, the rsync command used --update, which
`man rsync` says: "skip files that are newer on the receiver". That could
cause issues of the public repo getting out of sync with the private,
master repo. --archive is a better sync method since it aims to exactly
reproduce the sending dir to the receiving dir.
This allows a dir to be specified in config.py that `fdroid server update`
will automatically rsync the repo to. The idea is that the path would
point to an SD card on a fully offline machine that serves as the secure
repo signing machine.
rsync improvements for fdroid server update
This is a couple of improvements to how `fdroid server update` uses `rsync`.
(also, please remove the branch when accepting any of my merge requests)
This allows the SSH key used to sync with the server to be specified via
the config.py or the command line. I need it for running automated tests
and setups.
rsync uses the modification time and size of the file when deciding whether
to update a file. These are relatively easy to control in malicious code,
so instead make rsync use a full MD5 checksum when decided whether the
index needs to be updated. I suppose we could add an option to use
checksum checking on all files, but since the signed repo already provides
a checksum check, it seems not worth the added load on the process.
Also, renamed 'index' to 'indexxml' to make it clear what is the XML and
what is the JAR.
Test updates and related bug fixes
I just set up some big tests of generating repos based on feeding as many random APKs into `fdroid update` as possible. On our jenkins server, the tests copy all of the APKs that the jenkins server has generated and builds a repo from them. This process caught lots of little glitches in the whole process. While these little glitches are usually caused by problematic APKs, `fdroid update` should handle them gracefully. Hopefully this set of fixes accomplishes that.
