Commit graph

3677 commits

Author SHA1 Message Date
Hans-Christoph Steiner
1bcd9a8489 metadata: handle empty files and dicts, and error out on non-dicts 2023-04-28 10:55:20 +02:00
Hans-Christoph Steiner
2b81a66b79 App.comments is no more since !772 2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
dbe21b2b94 metadata: transform all TYPE_STRING values w/ _normalize_type_string()
Before this, there were separate post-parse paths for app-fields versus
build-flags.  This makes all TYPE_STRING values always go through the same
post-parse code path.
2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
a8531a03a6 metadata: refactor into one function to do YAML post processing
It used to be that there had to be separate post processing steps depending
on whether it was parsing .txt or .yml.  The .txt format is long gone!

!772
2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
3869e1374b metadata: force TYPE_STRING fields to string in internal dict
* YAML 1.2's boolean is 'true' so this makes the conversion correct.
* rewritemeta would also have to be changed to support this.
2023-04-27 21:51:10 +02:00
Hans-Christoph Steiner
c0ae09e0df metadata: remove strange app arg construct from parse_yaml_metadata()
My guess is that this is some kind of vestige of the old code structure,
back when there was .txt and .yml formats.  This makes it a normal Python
function: input as arg, return value is the result.
2023-04-27 21:51:10 +02:00
Hans-Christoph Steiner
41972e6525 warn on all unrecognized build flags
No need to put the list of keys in a set beforehand, just report all build
flags that are invalid.
2023-04-27 21:51:10 +02:00
Hans-Christoph Steiner
32ea4ef19a
enable black on fdroidserver/asynchronousfilereader/__init__.py 2023-04-27 11:34:18 +02:00
Hans-Christoph Steiner
2273d8203b
enable black on fdroidserver/signatures.py 2023-04-27 11:34:16 +02:00
Hans-Christoph Steiner
77f9ac9fa8
enable black on fdroidserver/rewritemeta.py 2023-04-27 11:34:12 +02:00
Hans-Christoph Steiner
eb81af57bb
enable black on fdroidserver/init.py 2023-04-27 11:34:10 +02:00
Hans-Christoph Steiner
7c692a4532 index-v2 'mirrors' fully settable from config
This lets mirrors: in config.yml be the same list-of-dicts format as it is
in index-v2.  This also includes a data format conversion to maintain the
right format for the old, unchanging index v0 and v1 formats.

#928
#1107
2023-04-27 07:53:02 +00:00
Hans-Christoph Steiner
ad960c6ecd
remove last maven: quirks and make it always a string
It turns out that the maven: field was originally declared as a TYPE_STRING,
given that it was not given a different type in metadata.py's 	flagtypes.
The code was confused because it was given a default value of `False` rather
than `None` as the rest of the TYPE_STRING fields have.

This construct in build.py means maven: should always be a string:

        if '@' in build.maven:
            maven_dir = os.path.join(root_dir, build.maven.split('@', 1)[1])
        else:
            maven_dir = root_dir
2023-04-24 11:08:48 +02:00
Gregor Düster
d971f19e92
metadata: Remove code ensuring YAML 1.2 compatibility
This is no longer necessary b/c the current YAML parser, ruamel.yaml,
implements YAML 1.2 in contrast to the previous parser PyYAML.
2023-04-18 17:08:44 +02:00
Gregor Düster
cff57e1a53
metadata: Remove ununsed code 2023-04-18 17:08:43 +02:00
Jason Donenfeld
6a239cbde4 Add maven central staging repo to allowed list 2023-04-12 12:44:42 +00:00
Jochen Sprickerhof
f5cdeff507
Write status json upon fdroid update start (Closes: #1114) 2023-04-11 17:14:58 +02:00
Hans-Christoph Steiner
36d2a8f899
all ndk paths in config must be strings
The paths in the config must be strings because they are used in things
like env vars where they must be strings.  Plus lots of other places in the
code assumes they are strings.  This is the first step to defining the
border of where paths can be pathlib.Path() and where they must be strings.
2023-04-11 13:54:19 +02:00
Hans-Christoph Steiner
898624dcac
common.get_android_tools_versions() for modern NDK dir layout
NDK r10e is still in use, but is so old, it does not have the modern
sdkmanager package metadata.
2023-04-11 13:54:17 +02:00
Hans-Christoph Steiner
0dd5a7db64
remove NDK download handling in favor of fdroid/sdkmanager 2023-04-11 13:54:05 +02:00
Jochen Sprickerhof
889b8cb372 Don't copy per version anti features to all versions
make_v0() modified the apps data structure to copy an anti feature for a
specific version to all versions resulting in index-v1 and -v2 to
contain wrong anti feature annotations. This patch fixes this and adds a
test that the data structure is no longer modified.

The bug shadowed bugs in the AF implementation of -v1 and -v2 resulting
in not coping the version specific data. This is corrected as well.
This is also tested now.

For -v2 the AF dict is now sorted to make the result reproducible.

Finally The NoSourceSince AF was added as a per version and overall AF
in -v1 and is now only applied as an overall AF and the test is updated
accordingly.
2023-04-04 14:20:41 +00:00
Licaon_Kter
35133a9a8f Scanner - add newer sonatype maven repos 2023-04-04 11:00:13 +00:00
Jochen Sprickerhof
2a5b127773
vmtools: Properly initialize provider member 2023-03-21 14:00:58 +01:00
Hans-Christoph Steiner
9749b26baa pylint: Reenable C0201 consider-iterating-dictionary
* C0201: Consider iterating the dictionary directly instead of calling .keys() (consider-iterating-dictionary)
2023-03-20 16:33:16 +01:00
Hans-Christoph Steiner
1fbfae355a pylint: Reenable R1723 no-else-break
* R1723: Unnecessary "elif" after "break", remove the leading "el" from "elif" (no-else-break)
2023-03-20 16:33:16 +01:00
FestplattenSchnitzel
abc752314b pylint: Reenable E1101 no-member 2023-03-20 16:33:16 +01:00
linsui
88eac942ec checkupdates: skip when there is no match for version
... name regex in CheckUpdateData
2023-03-20 14:31:08 +00:00
Hans-Christoph Steiner
b92e280eab fix code format for new black rule 2023-03-20 14:47:48 +01:00
Hans-Christoph Steiner
e903952029 always use defusedxml.ElementTree, never xml.etree.ElementTree
defusedxml is now available and installed everywhere, including the
buildserver VM and the buildserver host.

This fixes bandit's error because it didn't understand the try: block
* https://gitlab.com/eighthave/fdroidserver/-/jobs/3965835264
* https://bandit.readthedocs.io/en/1.7.5/blacklists/blacklist_calls.html#b313-b320-xml
2023-03-20 14:44:13 +01:00
Hans-Christoph Steiner
4295d254f9 build: actually raise exception when ndk: entry is invalid
pylint said:
fdroidserver/common.py:4238:8: W0133: Exception statement has no effect (pointless-exception-statement)
2023-03-20 14:43:44 +01:00
FC Stegerman
a5978ad8b2 metadata.Build.ndk_path(): find long form ndk too 2023-03-20 13:39:47 +00:00
Hans-Christoph Steiner
e6ff596b4c update: handle messed up namespaces in AndroidManifest.xml 2023-03-20 13:24:59 +00:00
Hans-Christoph Steiner
f3e49f4bcb
download_repo_index_v2() for verified downloading of index-v2 2023-03-09 12:35:29 +01:00
Hans-Christoph Steiner
dd16076651
add verify_jar_signature() to verify entry.jar 2023-03-09 12:34:55 +01:00
Hans-Christoph Steiner
723873456d
versioned download_repo_index_v1() to be future proof 2023-03-09 12:34:52 +01:00
FestplattenSchnitzel
2dda9db1f1
Remove support for Buildozer 2023-03-09 12:03:57 +01:00
Hans-Christoph Steiner
98448e0481 purge all references to zipalign, that is delegated to other things
Things like apksigner and @obfusk's tools handle this now.
jarsigner is used in the test, since that's the most common use of
`common.find_sdk_tools_cmd()`.

closes #1100
2023-02-22 14:45:51 +01:00
Jochen Sprickerhof
bae94931b4 Fix openjdk detection on different architectures
The pattern in Debian is:

java-$VERSION-openjdk-$DEB_HOST_ARCH

The regex for $DEB_HOST_ARCH did not match arm64, armel and armhf. As it
can be any string just match anything.
2023-02-22 11:24:39 +00:00
Hans-Christoph Steiner
24df262f6b handle str and pathlib.Path in getvcs() 2023-02-21 11:20:15 +01:00
Jochen Sprickerhof
671d1eba4c
Fix compare of Path objects 2023-02-20 11:59:16 +01:00
Hans-Christoph Steiner
b8f59097f7 purge all vestiges of the unused fdroid stats, closes #839
* for f in locale/*/LC_MESSAGES/fdroidserver.po; do msgattrib --set-obsolete --no-wrap --ignore-file=locale/fdroidserver.pot -o $f $f; done
* sed -i 's, \.\./fdroidserver/stats\.py,,' locale/*/LC_MESSAGES/fdroidserver.po
2023-02-19 22:54:38 +01:00
Izzy
04dc297982 downgrade missing ipfs_cid WARNING to DEBUG 2023-02-19 18:49:24 +00:00
Jochen Sprickerhof
e243983ff8 Cleanup upon failing sign_apk (Closes: #1085) 2023-02-19 10:55:23 +01:00
Hans-Christoph Steiner
f468270aa8
support Java 20, which is already in Debian/unstable, closes #1070 2023-02-17 16:35:20 +01:00
Hans-Christoph Steiner
0c9f62a5fe signindex: add JSON check for entry.json
Ran this to generate/refresh the test index files:
`cd tests; ../fdroid update --nosign --pretty`

* converts config.py description to a single line, since the values are no
  longer stripped, so this give the same output.

closes #835
2023-02-17 14:34:53 +00:00
FC Stegerman
2e080c8170 make F-Droid-specific changes to apksigcopier.py 2023-02-17 13:08:23 +00:00
FC Stegerman
c68e1489bd import apksigcopier v1.1.1 2023-02-17 13:08:23 +00:00
Jochen Sprickerhof
0f151a3779 v2: move video to translated text (Closes: #1095)
It contains a link to the actual video and can be translated.
2023-02-16 21:42:09 +01:00
Jochen Sprickerhof
04230dc7c0
common: support PosixPath in log message 2023-02-16 15:17:14 +01:00
Jochen Sprickerhof
fef42dbc87
build: support PosixPath in log message (Closes: #1096) 2023-02-16 14:29:31 +01:00