Since metadata files are now YAML, programs should be able to just read in
the .yml files, then use them with fdroidserver methods without having to
know about the App class. The App class just provides syntactic sugar by
allowing dict keys to be accessed as attributes.
Now that the mismatch between 'builds' and 'Builds' has been fixed, it is
now possible to read metadata/*.yml files with a standard YAML parser like
PyYAML, then output them using `metadata.write_metadata()`. Other API
functions in fdroidserver should also work in this case.
I did this as I was working on fdroiddata!7860
The .txt format was the last place where the lowercase "builds" was used,
this converts references everywhere to be "Builds". This makes it possible
to load metadata YAML files with any YAML parser, then have it possible to
use fdroidserver methods on that data, like metadata.write_metadata().
The test files in tests/metadata/dump/*.yaml were manually edited by cutting
the builds: block and putting it the sort order for Builds: so the contents
should be unchanged.
```
sed -i \
-e 's/app\.builds/app.get('Builds', \[\])/g' \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\])/app.get('Builds', \[\])/g" \
-e "s/app\.get('Builds', \[\])\.append/app\['Builds'\].append/g" \
-e "s/app\['builds'\]/app.get('Builds', [])/g" \
*/*.*
```
Ideally, an fdroid repo should be running from a clean git repo, so that
all changes are tracked in git. This change is useful in seeing which
changes and/or files are not in git. If there are modified files, the
dirty flag will be set, so this info can help debugging that.
The key fingerprint should be only hex digits, everything else can be
discarded. That makes it easy to use this function various fingerprint
formats, including the common, human-readable forms spaces between pairs or
quartets.
test_sdk_exists() succeeds on an empty directory so it does not give any
new information. Contrary, test_sdk_exists() succeeds on an empty
directory, so find_sdk_tools_cmd() returned None even though the tools
where not found, before.
Now that the description formatting is removed, there is no need to load
all of the app metadata before operating on a single one. This change
makes lint and rewritemeta only load the metadata for the apps it is actually
operating on. Before, it would always load all metadata files. #845closes#678
Back when fdroidserver was built around aapt, that was needed to
guarantee that a compatible version of aapt was used. Now, aapt is
only optionally used for getting the APK ID, so this was just
complicating maintenance.
When using fdroidserver methods as an API, the full setup might not
have taken place. `app` instances can always just be a dict, the App
class is mostly just a typing shortcut. This is incremental, it only
affects a couple of functions in fdroidserver/update.py.
None of the config options in config.py require Python code. YAML is a
common config data format, and it is also used for build metadata. It is
also much safer to use since it can be pure data, without anything
executable in it. This also reduces the attack surface of the fdroid
process by eliminating a guaranteed place to write to get code executed.
With config.py, any exploit that can get local write access can turn that
into execute access by writing to the config.py, then cleaning up after
itself once it has what it needs. Switching to YAML removes that vector
entirely.
Also, this removes the config_file argument. It is not used in either
fdroidserver or repomaker. Also, it probably wouldn't work since so
much of the code assumes that the current working dir is the root of the
repo.
Up until now, the buildserver has not included androguard. Since a
good version of androguard (v3.3.3+) is included in stretch-backports
and the buildserver is already setup to use stretch-backports, this
sets up the buildserver with androguard.
closes#627
Ultimately we want to get to using apksigner by default everywhere,
this gets us closer to that by setting up all new repos to use
apksigner by default in the config.py
.idsig files contain an apksig v4 (https://source.android.com/security/apksigning/v4)
new versions of apksigner make this signature by default and it ends up
in /repo. Without this patch it would be included into the index as a
file to be downloaded by users. F-Droid Client crashes when it
encounters such an apk entry.
It's fine to have these signature files in the repo though, maybe
fdroidclient can make use of them at some point in the future (they are
intended to support streaming app installations).
