This should help us profile what takes so long in `fdroid update`. It also
gives feedback so that people can see how close to done it is, or where it
failed.
This is based on how incremental status JSON works for `fdroid build`.
This file can be treated like the other index files in repo/. This also has
the advantage that it will automatically get synced by @CiaranG's existing
sync scripts.
Really, this is not meant to be set by the user in the config. But if they
add something harmless that'll be ignored anyway, it seems that throwing an
error is too much. So only throw the error if it is set wrongly.
`keypass: {env: keypass}` has been in use in production repos for
years. That is not anything new. It makes it possible to maintain
_config.yml_ publicly even when it needs secrets. This change makes
sure it is possible to use {env: foo} syntax anywhere where a string
value is valid. The "list of dicts" values can be str, list of str or
list of dicts with str.
Before the {env: keypass} syntax, the actual password was just inline
in the config file. Before this commit, it was only possible to use
{env: key} syntax in simple, string-only configs, e.g. from
examples/config.yml:
This eliminates the need to have these installed on the signing server:
* python3-qrcode
* python3-requests
The signing server currently uses a git clone to run _fdroidserver_ and
`apt-get install` for dependencies. This leaves "qrcode" in
"install_requires" since moving it to "extras_require" would break
`fdroid update` and `fdroid nightly` for anything that does
`pip install fdroidserver`:
https://gitlab.com/eighthave/fdroidserver/-/jobs/9386520037
This outputs YAML in a string that is suitable for use in regexps
and string replacements, as well as complete files. It is therefore
explicitly set up to avoid writing out headers and footers.
This is a key piece of the ongoing `PUBLISH` _config.yml_ migration. There was uneven implementation of which YAML parser to use, and that could lead to bugs where one parser might read a value one way, and a different parser will read the value a different way. I wanted to be sure that YAML 1.2 would always work.
This makes all code that handles config files use the same `ruamel.yaml` parsers. This only touches other usages of YAML parsers when there is overlap. This does not port all of _fdroidserver_ to `ruamel.yaml` and YAML 1.2. The metadata files should already be YAML 1.2 anyway.
# Conflicts:
# fdroidserver/lint.py
This makes it easy to track all the places that use config.yml, and
hopefully makes things feel cleaner. This also standardizes all places
where config.yml is written out to use UTF-8 as the file encoding.
This also includes a lot of black code format fixes.
Looks like the ~index-v2 work removed this error case, and the old bash
integration test case failed to catch it. This reestablishes this error.
@mindston's refactoring of the old bash test suite caught this issue, e.g.
!1587
verified.json can get quite large on verification.f-droid.org, and for some
unknown reason, it sometimes corrupts it when writing it out. All the data
is already available in all the other JSON files, so this just automatically
reconstructs it. Its a hack, but it took me much less time than I've
already spent trying to troubleshoot why it writes out corrupt verified.json.
ZipFile.namelist() produces a string per file. The filename could contain
newline chars, including at the beginning and end. ^$ in regex matches
around newline chars. \A\Z matches the beginning/end of the full string.
This is exactly the same as obfusk's r'\AMETA-INF/(?s:.)*\.(DSA|EC|RSA)\Z'
but in a readable format that is also easily searchable, and standard for
this code base.
https://github.com/obfusk/fdroid-fakesigner-poc/blob/master/fdroidserver-regex.patch#1251
Some places in the code that need this, like verify.py, do not have
app and build instances, but do have appid and versionCode. And
fdroidserver/build.py is going away.
index.xml is for old clients that are stuck in the past forever. So the
format should not change at all. Python 3.13 changed minidom so it no
longer converts " to an XML entity.
154477be72
This makes writing test cases a lot easier. For example:
======================================================================
ERROR: test_devices (tests.test_install.InstallTest.test_devices)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/hans/code/fdroid/server/tests/test_install.py", line 31, in test_devices
devices = fdroidserver.install.devices()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hans/code/fdroid/server/fdroidserver/install.py", line 225, in devices
p = common.SdkToolsPopen(['adb', "devices"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hans/code/fdroid/server/fdroidserver/common.py", line 2921, in SdkToolsPopen
return FDroidPopen([abscmd] + commands[1:],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hans/code/fdroid/server/fdroidserver/common.py", line 3024, in FDroidPopen
result = FDroidPopenBytes(commands, cwd, envs, output, stderr_to_stdout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/hans/code/fdroid/server/fdroidserver/common.py", line 2987, in FDroidPopenBytes
if output and options.verbose:
^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'verbose'
