Commit graph

1215 commits

Author SHA1 Message Date
Jochen Sprickerhof
0e071a689d Support hex versionCode in build command line
Example: fdroid build net.gaast.giggity:0x200
2019-11-29 20:02:24 +01:00
Jochen Sprickerhof
b83c3c9e18 Support hex in versionCode
Example: https://github.com/Wilm0r/giggity/blob/master/app/src/main/AndroidManifest.xml#L2
2019-11-29 20:01:29 +01:00
Hans-Christoph Steiner
e6bf586e74
common: make v2 signature message a debug message
https://gitlab.com/fdroid/fdroidserver/issues/703#note_238122327
closes #703
2019-11-13 11:59:23 +01:00
Hans-Christoph Steiner
3354e66bd3
common: use standard format tags when generating the log name 2019-11-13 11:59:20 +01:00
Hans-Christoph Steiner
2cbde77798
build: do not crash if SVN URL pre-validation fails; log error
This is the problem:
https://jenkins.debian.net/job/reproducible_fdroid_build_apps/704/console
2019-11-13 11:59:18 +01:00
Hans-Christoph Steiner
0a8af2c55f Merge branch 'fix-readline-in-srclibs' into 'master'
build: fix bad regexs when removing signingConfig from srclibs

See merge request fdroid/fdroidserver!686
2019-10-25 08:18:08 +00:00
Hans-Christoph Steiner
afaa24f2fd
build: fix bad regexs when removing signingConfig from srclibs
I went through the source of all apps in fdroiddata for examples, and found
some that use readLine() for things totally unrelated to signingConfigs.

https://gitlab.com/fdroid/fdroiddata/merge_requests/4775#note_234132902
2019-10-23 12:44:47 +02:00
Michael Pöhn
7fa3c34e5b update tests for fixed log deployment+changelog 2019-10-15 15:19:18 +02:00
Michael Pöhn
d665106813 fix disappearing build logs when deploying 2019-10-15 14:28:45 +02:00
Michael Pöhn
d5ab303d83 lint: license override config option + require FSF/OSI approved licenses by default 2019-10-02 22:14:09 +02:00
Gerhard Olsson
7c4bed0558 UpdateCheckIgnore: Exception if version not matched
Ignoreversions were checked also if version did not match which raised the exception
The 'version check' runs in some unexpected situations like just
to find directories where to find the AutoName.
2019-07-23 00:21:28 +02:00
Hans-Christoph Steiner
26af94974a Merge branch 'rsync-buildslogs-to-webroot-repo' into 'master'
build: rsync buildlogs to <webroot>/repo

See merge request fdroid/fdroidserver!651
2019-07-10 08:55:04 +00:00
Michael Pöhn
2c87b5e6f9 deploy build logs: no timestamps 2019-07-06 16:48:56 +02:00
Michael Pöhn
f30983368c build: rsync buildlogs to <webroot>/repo 2019-07-06 16:48:56 +02:00
relan
781b55f777 checkupdates: match whole words
Match only whole words when looking for versionCode, versionName, etc. in
manifests. A real build.gradle example:

    flutterVersionCode = '1'
    flutterVersionName = '1.0'
    ...
    defaultConfig {
        versionCode 53
        versionName "2.0.3"
    }

Before this change checkupdates was erroneously getting version code and
version name from the first two lines and failing to find a new release.
2019-06-13 19:51:44 +03:00
Hans-Christoph Steiner
17dc231dc9 update: fix running without androguard
Soon, we can rip out all the aapt parsing stuff, but not yet!
2019-02-03 15:50:39 +01:00
Hans-Christoph Steiner
2907aa3ea3 fallback to v3 signatures if v1 and v2 are not available 2019-02-01 09:18:00 +01:00
Hans-Christoph Steiner
d96f5ff660 support APK Signature V2 when apksigner is installed
This was done with much help from @uniqx.  This is the first level of
supporting APK Signatures v1, v2, and v3.  This is enough to include
APKs with any combo of v1/v2/v3 signatures.  For this to work at all,
apksigner and androguard 3.3.3+ must be installed.

closes #399
2019-02-01 09:17:56 +01:00
Hans-Christoph Steiner
ea84014f9b make signer_fingerprint() accept DER encoded certs
androguard's v2 signature methods return DER-encoded certificates.  The DER-
encoded certificates are parsed from JAR Signature's raw "Signature Block
File".
2019-02-01 08:50:57 +01:00
Hans-Christoph Steiner
1f346b3149 force_exit() to make exit work with all fdroid build conditions
The build command has to use some threading stuff to handle the timeout and
locks.  This seems to prevent the command from exiting, unless this hack is
used.
2018-11-14 14:27:32 +01:00
Hans-Christoph Steiner
57556aceee remove redundant open() arg: encoding='utf8'
By default, open() returns a str:
https://docs.python.org/3/library/functions.html#open

By default, str is UTF-8:
https://docs.python.org/3/library/stdtypes.html#str

This used to matter on Python 2.x, but this code is 3.x only now.
2018-10-19 15:01:34 +02:00
Hans-Christoph Steiner
d2216b756b fix typos in strings 2018-09-24 17:09:15 +02:00
Hans-Christoph Steiner
6b57cb6b7c fix strict Application ID checks
* upper case letters are allowed at all positions
* there must be a "." separator
2018-09-24 17:07:27 +02:00
Hans-Christoph Steiner
11d46072ab use androguard primitives to speed up finding debuggable flag
androguard parses the whole APK before handing the instance back, this uses
the primitives to just find the <application android:debuggable=""> value,
then stop parsing.

#557
2018-09-21 14:56:46 +02:00
Hans-Christoph Steiner
a3cecc16a3 use partial androguard binary XML parsing to speed up APK ID lookup
Normally, androguard parses the entire APK before it is possible to get any
values from it.  This uses androguard primitives to only attempt to parse
the AndroidManifest.xml, then to quit as soon as it gets what it needs.
This greatly speeds up the parsing (1 minute vs 60 minutes).

fdroid/fdroidserver#557
2018-09-21 14:56:46 +02:00
Hans-Christoph Steiner
fa09337b4b APK_ID_TRIPLET_REGEX only matches first line of aapt output
Stop expensive aapt parsing after the first line when looking with
APK_ID_TRIPLET_REGEX.  As is seen with the `aapt dump badging` output files
in tests/build-tools/, the first line is the only line that will ever match.

#557
2018-09-21 14:56:32 +02:00
Hans-Christoph Steiner
807bf3d26b build: reuse common methods for getting metadata from APKs
This splits out the code that gets the list of native ABIs supported, then
uses the standard methods for the rest.
2018-09-18 10:55:51 +02:00
Hans-Christoph Steiner
487c4d02f3 handle package: line output from aapt v28
fdroid/fdroiddata!3484
fdroid/fdroiddata!3562
fdroid/fdroidserver!548
2018-09-18 10:55:51 +02:00
Hans-Christoph Steiner
9d12b1dc61 add strict, tested validation of Android/F-Droid package names
Android has stricter rules than Java for Package Names, but anything the
Python regex thinks is valid must be valid according to Java's rules too.

https://developer.android.com/studio/build/application-id
2018-09-07 14:17:39 +02:00
Hans-Christoph Steiner
9bccb2c73e temp fallback to built-in ElementTree if defusedxml is not there 2018-09-03 23:02:31 +02:00
Hans-Christoph Steiner
11b3e5be3a update: throw exception for APKs with invalid Application ID
Android Application IDs must be valid Java Package Names.  While the build
tools likely validate the Application ID, it is possible to manually create
a malicious APK.
2018-09-03 22:56:08 +02:00
Hans-Christoph Steiner
5d161cc9fd validate appid when reading metadata files
The metadata file must be named after the Application ID of the app it is
describing, and Android Application IDs must be valid Java Package Names.
2018-09-03 22:56:08 +02:00
Hans-Christoph Steiner
3ffe2860f3 gitlab-ci: add 'bandit' security scanner to all runs
bandit is used by Radically Open Security and is part of the GitLab Ultimate
Static Application Security Testing (SAST) suite.

https://docs.gitlab.com/ee/user/project/merge_requests/sast.html
2018-08-29 17:48:06 +02:00
Hans-Christoph Steiner
4d13a904f3 use defusedxml to avoid DoS attacks while loading XML 2018-08-29 17:44:54 +02:00
Hans-Christoph Steiner
cc94ebca30 use global constant for the 'xmlns:android' XML namespace 2018-08-29 17:44:54 +02:00
Hans-Christoph Steiner
a089614225 checkupdates: remove magic number 99999999 from HTTP checks
It is vestigal from old code and no longer is needed.
2018-08-29 17:24:24 +02:00
Hans-Christoph Steiner
4503e7a92a replace unneeded eval() call and support negative versionCodes 2018-08-29 17:24:04 +02:00
Michael Pöhn
4dcfa95d6e fix: jarsigner and keytool lookup regression 2018-08-21 03:05:43 +02:00
Cyril Russo
81641b4628 Fixed precommit checks 2018-08-20 11:17:36 +02:00
Cyril Russo
4303b0fac1 Apply suggestion from @uniqx to using shutil.which instead of iterating path by hand 2018-08-17 20:14:54 +02:00
Cyril Russo
ace33bcfc0 Reverted the change in the default config.py
Improved the detection of keytool and jarsigner by also searching the PATH environment variable
2018-08-17 16:46:46 +02:00
Cyril Russo
4f21045cad Fix keytool not found on MacOSX (when using Java from Apple)
Fix keytool parsing error due to localisation keytool -list command.
Always fallback to english so the parsing makes sense.
2018-08-17 15:32:29 +02:00
Michael Pöhn
f53d1abcce fix: make sure ensure_final_value always has a clearly defined return value 2018-08-14 11:07:44 +02:00
Michael Pöhn
74ae6a6a3b set locale explicitly when using keytool 2018-08-03 04:36:00 +02:00
Hans-Christoph Steiner
4352aa2792 update: pass resourceIds through if they can't be deferenced
People do odd tricks with `aapt --rename-manifest-package` and androguard
doesn't handle them all.

* https://github.com/androguard/androguard/issues/503
2018-07-17 12:37:13 +02:00
Marcus Hoffmann
9889a98dea
build: improve gradle experience
This expands the gradle wrapper shell script used by the buildserver for
usage outside the buildserver environment. It also allows downloading
whitelisted versions of gradle if they are not yet deployed to the
buildserver by simply upsating the copy of fdroidserver (in contrast to
having to reprovision the whole buildserver).

We first move the buildserver/gradle shell script to the repo root
as gradlew-fdroid, as it's an fdroid specific gradle wrapper.
We also now sync it inside the build VM before each build.

We then add a list of whitelisted gradle distributions taken from the
makebuildserver script.

The script additionally now reads two env vars which tell it where to
expect installed versions of gradle and where it might store downloaded
gradle .zip files. Both of those are configurable from config.py. As the
first should normally just be a subdir of the second it's not exposed in
the example config.py but only used by the buildserver config.py.

Default config now uses this internal gradle wrapper but a path to a
custom wrapper or specific gradle distribution can still be set from
config.py.

Closes fdroid/fdroidserver#98
Ref: fdroid/fdroidserver#370
2018-07-11 11:49:46 +02:00
Hans-Christoph Steiner
d20e8613fe build: only test for all Android SDK paths where it is needed
Now that androguard can be used, the android-sdk is no longer required for
most operations.
2018-06-25 10:52:12 +02:00
Hans-Christoph Steiner
c3b361c91b remove 'from distutils.util import strtobool' dependency
Most of distutils is being removed from the Python standard library, and
this is totally not needed here.
2018-06-25 10:52:12 +02:00
Hans-Christoph Steiner
8f2ee4bd1d run 'zipalign' using standard flags used by Gradle Android Plugin
Nice find by @equeim!  -p was added in build-tools-23.0.0

https://developer.android.com/studio/publish/app-signing#sign-manually

closes #288
2018-06-19 15:40:48 +02:00
Michael Pöhn
752f6a4ba2 applied review suggestions for deploying (build) logs 2018-06-19 15:18:13 +02:00