The *-nightly git repo always uses the _master_ branch. The `index_only:`
support maintains a separate local branch since its git repo should be as
small as possible. The full repo should be maintained for mirrors not
using `index_only:` so that when it force-pushes, it does not need to always
push all the files, only the updated ones. So the full repo should be
maintained in the _master_ branch, and only the `index_only` mirrors should
have their own branch.
This adds a test case to reproduce this error:
https://gitlab.com/fdroid/fdroidclient/-/jobs/10347168516
This case also applies to any setup that used `servergitmirrors:` before the
`index_only:` feature was added. This also applies to cases if the repo
maintainer manually clones the *-nightly repo into _fdroid/git-mirror/_
This code already depends on GitPython, and hopefully the common.vcs* stuff
can eventually go away entirely. GitPython should provide those bits
already, and they are maintained by someone else.
https://github.com/gitpython-developers/GitPython/pull/2029
Oftentimes, the file that is copied is stripped, in which case, the file
size is different. Using a file size check here means it will rerun the
strip and copy every time `fdroid update` is run for any image that needs
to be stripped. If the source's ctime is newer than the destination, then
the process should run since it is a newly created file. Even more so with
mtime, since the destination's mtime is reset based on the source's.
Unlike screenshots, the featureGraphic, icon, promoGraphic, and tvBanner
should be placed directly in the locale directory instead of in a
dedicated subdirectory (in the F-Droid metadata structure). For version
1.0.0 of the Triple-T structure this currently isn't done. Instead, the
graphics are treated as screenshots are.
To illustrate:
en-US/listing/icon/icon.png
en-US/listing/featureGraphic/play_store_feature_graphic.png
Should end up as:
en-US/icon.png
en-US/featureGraphic.png
But instead they currently end up as:
en-US/icon/icon.png
en-US/featureGraphic/play_store_feature_graphic.png
This patch should fix it.
It seems the erroneous behavior was introduced in
a4169484fdCloses#1260
Package repos come from untrusted sources, in terms of the buildserver. They
should be handled in VMs and containers as much as possible to avoid
vulnerabilities. As far as I could tell, `fdroid update` only has a single
place where it executes any VCS system: if there is .fdroid.yml present in
a package repo, then it will fetch the commit ID using git.
For better security properties, this implements a simple function to just
read the files to get that commit ID. The function that executes git to do
the same thing is relabeled "unsafe". That is used for status JSON
everywhere, but that runs on fdroiddata.git and fdroidserver.git, which are
trusted repos.
The unsafe version is also used in places where git.Repo() is needed for
other things.
Fixes:
Traceback (most recent call last):
File "/home/fdroid/fdroidserver/fdroid", line 22, in <module>
fdroidserver.__main__.main()
File "/home/fdroid/fdroidserver/fdroidserver/__main__.py", line 222, in main
raise e
File "/home/fdroid/fdroidserver/fdroidserver/__main__.py", line 203, in main
mod.main()
File "/home/fdroid/fdroidserver/fdroidserver/update.py", line 2774, in main
fdroidserver.index.make(archived_apps, archapks, repodirs[1], True)
File "/home/fdroid/fdroidserver/fdroidserver/index.py", line 132, in make
make_v2(
File "/home/fdroid/fdroidserver/fdroidserver/index.py", line 728, in make_v2
output["repo"] = v2_repo(repodict, repodir, archive)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/fdroid/fdroidserver/fdroidserver/index.py", line 686, in v2_repo
repo["icon"] = config["archive" if archive else "repo"]["icon"]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
KeyError: 'icon'
!1627 missed this, so stages only get included in running.json. That means
the stages info is only visible while update is running, making it hard to
use.
This should help us profile what takes so long in `fdroid update`. It also
gives feedback so that people can see how close to done it is, or where it
failed.
This is based on how incremental status JSON works for `fdroid build`.
This file can be treated like the other index files in repo/. This also has
the advantage that it will automatically get synced by @CiaranG's existing
sync scripts.
Really, this is not meant to be set by the user in the config. But if they
add something harmless that'll be ignored anyway, it seems that throwing an
error is too much. So only throw the error if it is set wrongly.
