The *-nightly git repo always uses the _master_ branch. The `index_only:`
support maintains a separate local branch since its git repo should be as
small as possible. The full repo should be maintained for mirrors not
using `index_only:` so that when it force-pushes, it does not need to always
push all the files, only the updated ones. So the full repo should be
maintained in the _master_ branch, and only the `index_only` mirrors should
have their own branch.
This adds a test case to reproduce this error:
https://gitlab.com/fdroid/fdroidclient/-/jobs/10347168516
This case also applies to any setup that used `servergitmirrors:` before the
`index_only:` feature was added. This also applies to cases if the repo
maintainer manually clones the *-nightly repo into _fdroid/git-mirror/_
This file can be treated like the other index files in repo/. This also has
the advantage that it will automatically get synced by @CiaranG's existing
sync scripts.
Don't deploy versions of to GitHub releases where a `releaseChannels`
value is set in index-v2.json. (This usually would mean it's a alpha or
beta version.)
Use whatsNew text (if available) as release notes text when deploying to
Github releases. This feature will always use 'en-US' locale texts,
since English is the lingua franka on GitHub. Additionally this change
also adds a config option to preprend a static text to those release
notes.
There is no longer any reason for these to be intertwined.
This deliberately avoids touching some files as much as possible because
they are super tangled and due to be replaced. Those files are:
* fdroidserver/build.py
* fdroidserver/update.py
# Conflicts:
# tests/testcommon.py
# Conflicts:
# fdroidserver/btlog.py
# fdroidserver/import_subcommand.py
This moves all of the serverwebroot: logic into a function, and adds tests.
I did this because I ran into issues in the logic in main():
Traceback (most recent call last):
File "/builds/eighthave/fdroidserver/fdroid", line 22, in <module>
fdroidserver.__main__.main()
File "/builds/eighthave/fdroidserver/fdroidserver/__main__.py", line 230, in main
raise e
File "/builds/eighthave/fdroidserver/fdroidserver/__main__.py", line 211, in main
mod.main()
File "/builds/eighthave/fdroidserver/fdroidserver/deploy.py", line 753, in main
s = serverwebroot.rstrip('/').split(':')
AttributeError: 'dict' object has no attribute 'rstrip'
Since update_serverwebroot() is part of the public API, this function should
work without setting `fdroidserver.deploy.options` or
`fdroidserver.deploy.config`.
If there was a global default on a machine that was something other than
'master', these things would crash with:
Traceback (most recent call last):
File "/home/hans/code/fdroid/server/fdroid", line 22, in <module>
fdroidserver.__main__.main()
File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 230, in main
raise e
File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 211, in main
mod.main()
File "/home/hans/code/fdroid/server/fdroidserver/deploy.py", line 833, in main
push_binary_transparency(BINARY_TRANSPARENCY_DIR,
File "/home/hans/code/fdroid/server/fdroidserver/deploy.py", line 705, in push_binary_transparency
local.pull('master')
File "/usr/lib/python3/dist-packages/git/remote.py", line 1045, in pull
res = self._get_fetch_info_from_stderr(proc, progress, kill_after_timeout=kill_after_timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/git/remote.py", line 848, in _get_fetch_info_from_stderr
proc.wait(stderr=stderr_text)
File "/usr/lib/python3/dist-packages/git/cmd.py", line 604, in wait
raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
cmdline: git pull -v -- local master
stderr: 'fatal: couldn't find remote ref master'
When using rsync or s3cmd, the upload is done in multiple passes. This
reduces the chance of interfering with an existing client-server
interaction.
- rsync: In the first pass, upload without the index files and delay
the deletion as much as possible. That keeps the repo functional
while this update is running. Then second pass uploads the index
files.
- s3cmd: In the first pass, only new files are uploaded. In the
second pass, changed files are uploaded, overwriting what is on the
server. On the third/last pass, the indexes are uploaded, and any
removed files are deleted from the server. The last pass is the
only pass to use a full MD5 checksum of all files to detect changes.
