Commit graph

3596 commits

Author SHA1 Message Date
Michael Pöhn
07a366a4d6 add tests for scanner.load_exodus_trackers_signatures 2022-07-17 15:50:07 +02:00
Michael Pöhn
544772d3ab add missing "or" to scanner cli synopsis 2022-07-17 15:50:07 +02:00
Michael Pöhn
a1677b5cb0 add test case for _exodus_compile_signatures 2022-07-17 15:50:07 +02:00
Jochen Sprickerhof
3bd09ef7f4 Integrate Exodus (Closes: #566, #1008)
Code taken from:

https://github.com/Exodus-Privacy/exodus-core/blob/v1/exodus_core/analysis/static_analysis.py
2022-07-17 12:09:03 +00:00
Jochen Sprickerhof
ea9299f216 Use binary scanner with fdroid scanner path/to.apk
Closes: #806
2022-07-17 12:09:03 +00:00
linsui
534c170615 NDK r25 2022-07-14 06:35:55 +00:00
Hans-Christoph Steiner
3182b77d18
use apksigner to sign index-v2 with modern, supported algorithms
The current signing method uses apksigner to sign the JAR so that it
will automatically select algorithms that are compatible with Android
SDK 23, which added the most recent algorithms:
https://developer.android.com/reference/java/security/Signature

This signing method uses then inherits the default signing algothim
settings, since Java and Android both maintain those.  That helps
avoid a repeat of being stuck on an old signing algorithm.  That means
specifically that this call to apksigner does not specify any of the
algorithms.

The old indexes must be signed by SHA1withRSA otherwise they will no
longer be compatible with old Androids.

apksigner 30.0.0+ is available in Debian/bullseye, Debian/buster-backports,
Ubuntu 21.10, and Ubuntu 20.04 from the fdroid PPA.  Here's a quick way to
test:

for f in `ls -1 /opt/android-sdk/build-tools/*/apksigner | sort ` /usr/bin/apksigner; do printf "$f : "; $f sign --v4-signing-enabled false; done

closes #1005
2022-06-07 16:56:23 +02:00
Jochen Sprickerhof
e6c875b77e Archive _src.tar.gz.asc not .log.gz.asc 2022-06-07 11:37:29 +00:00
Jochen Sprickerhof
9933f54093 Fix --nosign semantics
This reverts parts of 197ca7e36 as it never triggered the
FDroidException as 'keystore' not in common.config is always False.
2022-06-07 13:22:02 +02:00
Jochen Sprickerhof
1073dd57f7 Make webBaseUrl configurable 2022-06-04 11:56:04 +02:00
Jochen Sprickerhof
72e3d07acb Simplify primary mirror logic
Assume repo_url/archive_url to be valid URL and mark it as a primary
mirror.
2022-06-04 11:56:04 +02:00
Jochen Sprickerhof
edc9390628 Pass --scan-binary to build server 2022-06-03 14:27:58 +00:00
Jochen Sprickerhof
8a2f013508 Archive .idsig and .log.gz.asc files as well 2022-06-03 10:56:22 +00:00
Jochen Sprickerhof
4ce79a7eaa explicitly re-raising exceptions
To fix pylint.
2022-06-01 22:52:22 +02:00
Hans-Christoph Steiner
c6dcc82ca4 allow common.get_apk_id() to be used in the API
If a project uses fdroidserver as a library, then just calls
common.get_apk_id(), it will now work.  Before, that project would have had
to include something like `common.config = {}` to avoid a stacktrace.
2022-05-26 22:18:21 +02:00
FestplattenSchnitzel
5951a6c2b1 [import] Fix crash when no Gradle subdir is found 2022-05-26 17:59:41 +00:00
fdroid-bot
762c945c42 Android NDK None (None) 2022-05-26 15:10:14 +00:00
Danilo Bargen
a4d0698628 Fix parsing of smartcardoptions config
With the previous code, a trailing newline would result in an empty
space being part of the list. When this is passed to keytool, it fails
with "Illegal option: ".

Instead of doing overly complicated regex based string substitution
followed by parametrized splitting, we can simply use `.split()`
without any parameters, and Python will automatically strip any
whitespace.
2022-05-26 16:47:33 +02:00
Hans-Christoph Steiner
5448fa2a2d
update: hide confusing androguard warning that is irrelevant here
This is related to androguard features that fdroidserver does not use:
WARNING: Requested API level 31 is larger than maximum we have, returning API level 28 instead.
2022-05-23 23:17:42 +02:00
Hans-Christoph Steiner
4ce3da476d
btlog: run black to format code 2022-05-23 23:17:40 +02:00
Hans-Christoph Steiner
7be0ec3572
btlog: include index-v2.json entry.json and entry.jar 2022-05-23 23:17:37 +02:00
Hans-Christoph Steiner
2448f070e9 fix tests and docstring error 2022-05-23 15:34:30 +02:00
Hans-Christoph Steiner
293975d081
refactor comment into docstring for update_serverwebroot 2022-05-23 14:41:37 +02:00
Hans-Christoph Steiner
053a64718a
deploy: handle index-v2 files on two pass sync methods
When using rsync or s3cmd, the upload is done in multiple passes. This
reduces the chance of interfering with an existing client-server
interaction.

- rsync: In the first pass, upload without the index files and delay
  the deletion as much as possible.  That keeps the repo functional
  while this update is running.  Then second pass uploads the index
  files.

- s3cmd: In the first pass, only new files are uploaded.  In the
  second pass, changed files are uploaded, overwriting what is on the
  server.  On the third/last pass, the indexes are uploaded, and any
  removed files are deleted from the server.  The last pass is the
  only pass to use a full MD5 checksum of all files to detect changes.
2022-05-23 14:41:34 +02:00
Jochen Sprickerhof
b07d23ff5c Don't include disabled apks in the index
This needs a rerun of `fdroid update --clean`.

In case a build is disabled delete_disabled_builds takes care of
deleting it from the repo. But this only works if the apk follows the
normal name pattern. Otherwise it will stay in the folder and be picked
up by process_apks and added to the index.

Closes: #1002
2022-05-23 11:47:53 +00:00
Jochen Sprickerhof
d70e5c2cd9 Index v2 2022-05-23 10:39:17 +00:00
linsui
13534bf02e scanner: add me.pushy 2022-05-21 12:25:11 +00:00
Hans-Christoph Steiner
73c31e6e63
signindex: do not remove index-v1.json after signing
With ~index-v2, the model is changing to offer the plain JSON file for easy
consumption.  Then gpgsign will also provide a detached PGP signature for
systems that would rather verify based on PGP signatures than JAR signatures.

!1080
closes #969
2022-05-17 15:38:06 +02:00
Hans-Christoph Steiner
107593d3ba net: improve parsing filenames from URL
This is more relevant to issuebot than fdroidserver, but it can't hurt here
also.
2022-05-09 20:59:15 +02:00
linsui
c16d170f12 scanner: add kotlinx-coroutines-play-services 2022-05-07 01:49:56 +00:00
linsui
90bc8e1e8f scan gradle.kts 2022-05-07 00:43:55 +08:00
linsui
4f04bd9200 Update scanner.py 2022-05-05 03:39:49 +00:00
Jochen Sprickerhof
50e680be93 Silence dexdump output in build --verbose 2022-05-03 18:30:02 +02:00
Jochen Sprickerhof
c80fdd5ce8 Handle BadZipFile 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
925cdbe542 Improve recursion warning, thx Hans! 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
9eaab6050e Fix variable name 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
91320fa10e Rework code 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
7ec3905734 Log zip recursion limit 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
5fc88431f8 Simplify code 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
d0d56173c9 Support nested apks 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
635f12e438 Use latest build-tools version 2022-05-03 14:59:52 +00:00
Jochen Sprickerhof
9c8e128064 [scanner] replace apkanalyzer by dexdump 2022-05-03 14:59:52 +00:00
Hans-Christoph Steiner
189ff9cd7b
build: improve error message when NDK SHA-256 does not match 2022-04-26 10:25:34 +02:00
Jochen Sprickerhof
d05bcafe14 Test index before signing 2022-04-25 22:26:59 +02:00
Sergey Bobrenok
46f4205fff Avoid zero-length prefixes in PATH
A zero-length prefix in PATH is a legacy feature that indicates the
current working directory.

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03

Found in ru.nsu.bobrofon.easysshfs, see:
https://gitlab.com/fdroid/fdroiddata/-/merge_requests/10953#note_921802636
2022-04-24 16:33:33 +03:00
Jochen Sprickerhof
253f2e5fef [scanner] Deep copy json structs (Closes: #985)
The lists in the DEFAULT_JSON_PER_BUILD where only initialized and thus
shared between builds resulting in accumulating the entries over all
versions. As per 2f5d780c4 DEFAULT_JSON_PER_BUILD is used as an API, so
use deepcopy to work around it.
2022-04-21 12:58:07 +00:00
Jochen Sprickerhof
5f3eb601df Add check for repo/archive_url 2022-04-21 11:16:14 +02:00
Jochen Sprickerhof
c19f46b706 [checkupdates] move gradle var check to parser
3638acddc added a check if the version name string is actually a
unresolved gradle variable. This moves the check into the
common.parse_androidmanifests() as it is the only where the it could
happen. This also resolves the case where checkupdates returns
"Unknown".

Closes: #751
2022-04-14 17:47:44 +02:00
Licaon_Kter
1df6b1db90 Import - some fixes for flutter apps 2022-04-12 10:08:13 +00:00
Jochen Sprickerhof
b6ea0d6acd Consider namespace for applicationId
> If the application ID is not explicitly defined using the
> applicationId property, it automatically takes on the same value as
> the namespace.

https://developer.android.com/studio/build/configure-app-module

Found in com.amanoteam.unalix, see:

https://gitlab.com/fdroid/fdroiddata/-/merge_requests/10895#note_907080341
2022-04-10 09:27:17 +02:00