Commit graph

1313 commits

Author SHA1 Message Date
Andrew Gunnerson
679ba6f552
Add support for Python 3.12
Python 3.12 completely removed the builtin `distutils` module. This
commit replaces its use with the `packaging` package, which is an
external dependency, but maintained by the Python developers.

Signed-off-by: Andrew Gunnerson <accounts+gitlab@chiller3.com>
2023-12-07 18:15:05 -05:00
Hans-Christoph Steiner
6d541e3ef6
establish config/mirrors.yml as a way to set up mirrors
The mirrors: entry in config.yml is great for quick access and shorter
mirror lists.  Now that we are adding a lot more metadata to the mirrors,
including potentially the full text of the privacy policy, having this
in its own file should make these cases easier to manage.
2023-12-07 14:38:26 +01:00
linsui
e2e7782ef3 add --force to git deinit 2023-11-25 22:11:19 +08:00
Hans-Christoph Steiner
3e9f27612e Revert "Clone without blobs to save space" !1382
This seems to make git ignore the _fsck skipList_ files.

This reverts commit 24ea1f6852.

closes #1177
fdroid-bootstrap-buildserver!34
!1408
2023-11-20 10:07:05 +01:00
linsui
f99f430e9e deinit git submodule 2023-11-14 01:06:49 +08:00
sim
24ea1f6852 Clone without blobs to save space 2023-11-09 12:13:29 +00:00
Hans-Christoph Steiner
3757add164 test_sdk_exists to be based on apksigner, that's the requirement
Before, lots of pieces of the Android SDK were required for fdroidserver to
operate, like aapt, zipalign, etc.  Now, apksigner is the only requirement.

%"support APK Signature v2+"
!889
2023-09-17 23:31:07 +02:00
Hans-Christoph Steiner
64b8ee772c throw useful error if a config YAML file is not a dict 2023-09-08 10:42:27 +02:00
Hans-Christoph Steiner
9df8caca07 log error if config file name is not known
The case of Anti-Features keys and filenames is quite confusing.  I was
confused to find out that config/antiFeatures.yml is correct, while
config/antifeatures.yml is nothing.  This throws an error message to make
that clear.
2023-07-25 14:31:09 +00:00
Hans-Christoph Steiner
4e28fad55a use constants for names used in the config dict
Hopefully this helps with the Anti-Features case confusion:
* antifeatures
* antiFeatures
* AntiFeatures
2023-07-25 14:31:09 +00:00
Hans-Christoph Steiner
bc58959315 purge remnants of repo/categories.txt, which is long unused 2023-06-02 13:26:03 +02:00
Jochen Sprickerhof
5723a1dafe
Create repo/icons without failure
Fixes fdroid lint CI.
2023-05-12 09:12:28 +02:00
Hans-Christoph Steiner
c2bc52dd85 use constant for default locale 2023-05-11 10:37:45 +02:00
Hans-Christoph Steiner
b04c7ff539 load_localized_config() returns a dict in a stable order
I renamed the variables while I was at it, to make it clearer.
2023-05-10 13:17:40 +02:00
Hans-Christoph Steiner
74a23284e1 common: load_localized_config() should make repo/ if not present
For 1,000,000 checks, this adds:
* ~4 seconds of runtime on a server with very slow disks.
* ~0.7 seconds of runtime on my laptop with a fast SSD.
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
49362b5fd1 move load_locale() and file_entry() to be accessible by all modules
* load_locale -> common.load_localized_config() since common handles config
* file_entry -> metadata.file_entry() since metadata handles data format
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
1e4e2489aa
add keep_when_not_allowed config/option 2023-05-05 09:49:32 +02:00
Hans-Christoph Steiner
36d2a8f899
all ndk paths in config must be strings
The paths in the config must be strings because they are used in things
like env vars where they must be strings.  Plus lots of other places in the
code assumes they are strings.  This is the first step to defining the
border of where paths can be pathlib.Path() and where they must be strings.
2023-04-11 13:54:19 +02:00
Hans-Christoph Steiner
898624dcac
common.get_android_tools_versions() for modern NDK dir layout
NDK r10e is still in use, but is so old, it does not have the modern
sdkmanager package metadata.
2023-04-11 13:54:17 +02:00
Hans-Christoph Steiner
0dd5a7db64
remove NDK download handling in favor of fdroid/sdkmanager 2023-04-11 13:54:05 +02:00
Hans-Christoph Steiner
1fbfae355a pylint: Reenable R1723 no-else-break
* R1723: Unnecessary "elif" after "break", remove the leading "el" from "elif" (no-else-break)
2023-03-20 16:33:16 +01:00
FestplattenSchnitzel
abc752314b pylint: Reenable E1101 no-member 2023-03-20 16:33:16 +01:00
Hans-Christoph Steiner
e903952029 always use defusedxml.ElementTree, never xml.etree.ElementTree
defusedxml is now available and installed everywhere, including the
buildserver VM and the buildserver host.

This fixes bandit's error because it didn't understand the try: block
* https://gitlab.com/eighthave/fdroidserver/-/jobs/3965835264
* https://bandit.readthedocs.io/en/1.7.5/blacklists/blacklist_calls.html#b313-b320-xml
2023-03-20 14:44:13 +01:00
Hans-Christoph Steiner
4295d254f9 build: actually raise exception when ndk: entry is invalid
pylint said:
fdroidserver/common.py:4238:8: W0133: Exception statement has no effect (pointless-exception-statement)
2023-03-20 14:43:44 +01:00
Hans-Christoph Steiner
dd16076651
add verify_jar_signature() to verify entry.jar 2023-03-09 12:34:55 +01:00
Hans-Christoph Steiner
98448e0481 purge all references to zipalign, that is delegated to other things
Things like apksigner and @obfusk's tools handle this now.
jarsigner is used in the test, since that's the most common use of
`common.find_sdk_tools_cmd()`.

closes #1100
2023-02-22 14:45:51 +01:00
Jochen Sprickerhof
bae94931b4 Fix openjdk detection on different architectures
The pattern in Debian is:

java-$VERSION-openjdk-$DEB_HOST_ARCH

The regex for $DEB_HOST_ARCH did not match arm64, armel and armhf. As it
can be any string just match anything.
2023-02-22 11:24:39 +00:00
Hans-Christoph Steiner
24df262f6b handle str and pathlib.Path in getvcs() 2023-02-21 11:20:15 +01:00
Jochen Sprickerhof
671d1eba4c
Fix compare of Path objects 2023-02-20 11:59:16 +01:00
Hans-Christoph Steiner
b8f59097f7 purge all vestiges of the unused fdroid stats, closes #839
* for f in locale/*/LC_MESSAGES/fdroidserver.po; do msgattrib --set-obsolete --no-wrap --ignore-file=locale/fdroidserver.pot -o $f $f; done
* sed -i 's, \.\./fdroidserver/stats\.py,,' locale/*/LC_MESSAGES/fdroidserver.po
2023-02-19 22:54:38 +01:00
Izzy
04dc297982 downgrade missing ipfs_cid WARNING to DEBUG 2023-02-19 18:49:24 +00:00
Jochen Sprickerhof
e243983ff8 Cleanup upon failing sign_apk (Closes: #1085) 2023-02-19 10:55:23 +01:00
Hans-Christoph Steiner
f468270aa8
support Java 20, which is already in Debian/unstable, closes #1070 2023-02-17 16:35:20 +01:00
FC Stegerman
2e080c8170 make F-Droid-specific changes to apksigcopier.py 2023-02-17 13:08:23 +00:00
Jochen Sprickerhof
04230dc7c0
common: support PosixPath in log message 2023-02-16 15:17:14 +01:00
linsui
58f167b7e5 Add build.binary 2023-02-16 20:16:52 +08:00
Jochen Sprickerhof
ecfbdef60f
Fix pathlib usage
Regression from 4a581bdf.
2023-02-16 09:23:07 +01:00
Simon Brand
4a581bdfb6 Remove path workarounds for old python versions 2023-02-15 19:25:48 +00:00
Jochen Sprickerhof
4a6ccae71a
install_ndk: cleanup tempdir 2023-02-06 14:52:32 +01:00
fdroid-bot
e71afa1299 Android NDK None (None) 2023-02-02 13:35:29 +00:00
linsui
a3f760f849 Add codeberg.org 2023-01-16 09:36:05 +00:00
FC Stegerman
c81ad4485c verify_apks(): compare to unsigned APK if copying fails 2022-11-30 10:39:46 +00:00
Hans-Christoph Steiner
c2567d71d1
nightly: fail if *-nightly git repo is not publicly available 2022-11-16 20:16:21 +01:00
FC Stegerman
e58637374c
verify_deprecated_jar_signature(): use temp dir instead of $PWD/.java.security 2022-11-15 19:10:38 +01:00
Hans-Christoph Steiner
01f7dfd0b3 index-v2.jar is not a valid file, remove references
entry.jar is the signed file, it references index-v2.json.
2022-11-15 13:22:09 +00:00
Michael Pöhn
24d88705fa 🔍 add scanner_signature_sources config option
This adds the option to configure which set of signatures `fdroid
scanner` should use, by configuring it in `config.yml`. It allows
fetching signatures in our custom json format. It also adds 3 additional
sources: 'suss', 'exodus', 'etip'
2022-11-15 09:20:26 +00:00
Jochen Sprickerhof
1bb963d768
jarsigner: allow weak signatures
openjdk-11 11.0.17 in Debian unstable fails to verify weak signatures:

jarsigner -verbose -strict -verify tests/signindex/guardianproject.jar

         131 Fri Dec 02 20:10:00 CET 2016 META-INF/MANIFEST.MF
         252 Fri Dec 02 20:10:04 CET 2016 META-INF/1.SF
        2299 Fri Dec 02 20:10:04 CET 2016 META-INF/1.RSA
           0 Fri Dec 02 20:09:58 CET 2016 META-INF/
 m  ?  48743 Fri Dec 02 20:09:58 CET 2016 index.xml

  s = signature was verified
  m = entry is listed in manifest
  k = at least one certificate was found in keystore
  ? = unsigned entry

- Signed by "EMAILADDRESS=root@guardianproject.info, CN=guardianproject.info, O=Guardian Project, OU=FDroid Repo, L=New York, ST=New York, C=US"
    Digest algorithm: SHA1 (disabled)
    Signature algorithm: SHA1withRSA (disabled), 4096-bit key

WARNING: The jar will be treated as unsigned, because it is signed with a weak algorithm that is now disabled by the security property:

  jdk.jar.disabledAlgorithms=MD2, MD5, RSA keySize < 1024, DSA keySize < 1024, SHA1 denyAfter 2019-01-01, include jdk.disabled.namedCurves
2022-11-14 17:59:56 +01:00
Jochen Sprickerhof
beed746740
Replace custom code by verify_jar_signature() 2022-11-14 17:58:35 +01:00
Jochen Sprickerhof
557fe87d44 Run shell scripts with -e (Closes: #1035)
Make sudo, init prebuild, build and Prepare fields lists and only
concatenate them with '; ' before execution. This allows arbitrary
commands inside the fileds (even && and ';') as we don't need to split
the commands again for rewritemeta.
2022-11-03 12:25:31 +00:00
linsui
5a28f20301 make versionCode/build.timeout an integer
Co-authored-by: Jochen Sprickerhof <git@jochen.sprickerhof.de>
2022-10-14 08:42:26 +00:00