Commit graph

3677 commits

Author SHA1 Message Date
Hans-Christoph Steiner
c4a3354c75 update: support flavors and alternate project dirs for Triple-T
closes #542
2019-02-27 16:40:47 +01:00
Hans-Christoph Steiner
67834697e0 update: get localized text/images from Triple-T Play Publisher 2.x
closes #579
closes #567
2019-02-26 23:42:39 +01:00
Hans-Christoph Steiner
7798dcf9dd index: keep mirrors in the order specified in config.py
The mirrors are displayed in fdroidclient in the order they are received,
and there might be some reason why a repo maintainer wants them in a
specific order.  The danger is that if the mirrors are set in config.py
using "mirrors = {'foo', bar'}" they will have a randomized order since it
will be in a set.  They should be set using [] or () to have a fixed order.
2019-02-22 21:38:20 +01:00
Hans-Christoph Steiner
548f73d988 nightly: search subdirs called 'fdroid' for APKs
@MTRNord tracked down this bug while working on his app
https://gitlab.com/Nordgedanken/simplematrix
2019-02-22 21:32:33 +01:00
Michael Pöhn
35e67295e9 Merge branch 'mvglasow-master-patch-68763' into 'master'
Clarify warning message about skipped sudo commands

See merge request fdroid/fdroidserver!608
2019-02-09 18:08:23 +00:00
Marcus Hoffmann
94c6b776fe build: log buildserver guest ram + cpu 2019-02-07 19:48:03 +01:00
Pierre Rudloff
b3754af137 Add cloudrail to blacklist 2019-02-07 10:26:03 +01:00
Hans-Christoph Steiner
17dc231dc9 update: fix running without androguard
Soon, we can rip out all the aapt parsing stuff, but not yet!
2019-02-03 15:50:39 +01:00
Hans-Christoph Steiner
09bbca4a51 update: force all "SDK Version" values to int when parsing aapt 2019-02-01 09:56:36 +01:00
Michael Pöhn
dd695c650e update: treat target and min sdk version as int 2019-02-01 09:56:21 +01:00
Hans-Christoph Steiner
2907aa3ea3 fallback to v3 signatures if v1 and v2 are not available 2019-02-01 09:18:00 +01:00
Hans-Christoph Steiner
d96f5ff660 support APK Signature V2 when apksigner is installed
This was done with much help from @uniqx.  This is the first level of
supporting APK Signatures v1, v2, and v3.  This is enough to include
APKs with any combo of v1/v2/v3 signatures.  For this to work at all,
apksigner and androguard 3.3.3+ must be installed.

closes #399
2019-02-01 09:17:56 +01:00
Hans-Christoph Steiner
ea84014f9b make signer_fingerprint() accept DER encoded certs
androguard's v2 signature methods return DER-encoded certificates.  The DER-
encoded certificates are parsed from JAR Signature's raw "Signature Block
File".
2019-02-01 08:50:57 +01:00
Hans-Christoph Steiner
709d35be0e Merge branch 'patch-1' into 'master'
change glob filter for graphics to not filter out valid IETF RFC5646 locales (fixes #621 and #357)

Closes #357 and #621

See merge request fdroid/fdroidserver!611
2019-01-31 22:09:41 +00:00
Tobias Zwick
1cbd19e33d change back link to fastlane docs 2019-01-30 11:29:55 +01:00
Michael Pöhn
a787d123a2 fdroid build: hide --on-server cli flag 2019-01-29 11:23:45 +01:00
Tobias Zwick
50b1e648b2 remove listing about the expected directory structure from the docstring, instead, link the current fastlane docs 2019-01-28 19:44:03 +01:00
Tobias Zwick
412185f058 Merge branch 'master' into patch-1 2019-01-28 19:30:26 +01:00
Hans-Christoph Steiner
217e6a7c30 metadata: fix pyflakes "use ==/!= to compare str, bytes, and int literals"
$ ./hooks/pre-commit || export EXITVALUE=1
WARNING: ruby is not installed, using dummy placeholder!
fdroidserver/metadata.py:1180: use ==/!= to compare str, bytes, and int literals
fdroidserver/metadata.py:1184: use ==/!= to compare str, bytes, and int literals
fdroidserver/metadata.py:1186: use ==/!= to compare str, bytes, and int literals
fdroidserver/metadata.py:1189: use ==/!= to compare str, bytes, and int literals
ERROR: pyflakes tests failed!
2019-01-27 23:00:00 +01:00
westnordost
a227ad6e1a change glob filter for graphics to not filter out valid IETF RFC5646 locales (fixes #621 and #357)
Also, update the documentation comment to be absolutely clear.

The function used to copy images from any locale directory starting with at least two lowercase letters correctly, but only add those to the app metadata that started with two letters plus one big letter or a "-". The same problem did not exist for adding texts to the metadata.
2019-01-23 11:21:19 +00:00
Hans-Christoph Steiner
aa47e93f36 Revert "Revert "lint: check if CurrentVersion is older than oldest build entry""
This reverts commit a5ec1703c9.

fdroid/fdroiddata@42bb872b46
2019-01-21 23:00:33 +01:00
Hans-Christoph Steiner
1174b14d1e Merge branch 'neonsoftware/master' into master
buildserver: include build-tools 28.0.3
  Added test aapt output files for build-tools 28.0.3
  update.py: Still aapt output parsing, setting regex to catch 'name=' without prefixes, needed for build-tools 28.0.3

fdroid/fdroidserver!591
2019-01-21 22:39:35 +01:00
Michael von Glasow
f8f17a430d Reword warning message about sudo commands being skipped 2019-01-15 14:14:09 +00:00
Michael von Glasow
868516f3fd Clarify warning message about skipped sudo commands 2019-01-15 12:36:17 +00:00
Francesco Cervigni
984d276c1b Added test aapt output files for build-tools 28.0.3 2019-01-13 19:27:02 +01:00
Marcus Hoffmann
a5ec1703c9 Revert "lint: check if CurrentVersion is older than oldest build entry"
This reverts commit 78473e7dab.

This is currently blocked by some apps in fdroiddata being reset to a
broken state.
2019-01-10 14:48:52 +01:00
Marcus Hoffmann
e1c547cfdf Revert "Revert "Merge branch 'random-fixes' into 'master'""
This reverts commit f6f2fb0b89.

Only one of the included commit should have been reverted.
2019-01-10 14:48:29 +01:00
Marcus
f6f2fb0b89 Revert "Merge branch 'random-fixes' into 'master'"
This reverts merge request !605
2019-01-10 13:46:49 +00:00
Francesco Cervigni
64445520b0 update.py: Still aapt output parsing, setting regex to catch 'name=' without prefixes, needed for build-tools 28.0.3
build-tools 28.0.3 added a new field in the end 'compileSdkVersionCodename=', which also accidentally ends with the string 'name='.
The purpose of this regex was to catch the 'packageName' field, which is in ht eaapt ouput the exact ' name=', therefore added whe non-caracter \W prefix match.

sample aapt output (28.0.3):
package: name='com.a.b.app' versionCode='1' versionName='1.0' compileSdkVersion='28' compileSdkVersionCodename='9'

previously, regex was catching second occurence, so '9'

See merge request !582
2019-01-08 22:46:47 +01:00
Hans-Christoph Steiner
76b026e572 lint: check whether the locale was included in an f-droid.org URL 2019-01-03 21:00:01 +01:00
Hans-Christoph Steiner
78473e7dab lint: check if CurrentVersion is older than oldest build entry
This causes F-Droid to never install this app, except for manually via the
Versions list in AppDetails.
2019-01-03 21:00:01 +01:00
Hans-Christoph Steiner
a2aef721d8 add timeout to net.http_get() and index.download_repo_index()
The requests docs recommend this:
http://docs.python-requests.org/en/master/user/quickstart/#timeouts

And mirror-monitor was hanging forever on a bad mirror.
2019-01-03 20:26:40 +01:00
Hans-Christoph Steiner
33b097a3aa deploy: upload to VirusTotal using index-v1.json or index-v1.jar
* A mirror of the repo will have only index-v1.jar.
* A buildserver with separate signing might have only index-v1.json
2018-12-17 14:33:02 +01:00
Hans-Christoph Steiner
869cc114a3 create index.get_index_from_jar() for parsing index-v1.json
Also removes redundant comments, should be in the doc string!
2018-12-17 14:30:05 +01:00
Hans-Christoph Steiner
13016c5d63 verify: add --output-json option to generate status reports
This has to do custom dict/encoder/decoder tricks in order to make sure
that each run of `fdroid verify` on the same files doesn't add duplicate
entries.
2018-12-07 13:13:10 +01:00
Hans-Christoph Steiner
e85256e248 verify: rename var remoteapk to remote_apk to fit other style 2018-12-07 11:19:26 +01:00
Hans-Christoph Steiner
51d961630b fix aapt scraping of <uses-permission> with maxSdkVersion
3e0d1beb09 changed this logic a bit, and it
wasn't quite right.  Then changing the SDK Versions to integers everywhere
seemed to bring this out more.
2018-12-06 13:42:45 +01:00
Hans-Christoph Steiner
653d5fbd88 update: fix running on trusty and things with old PIL/Pillow 2018-12-06 12:49:48 +01:00
Michael Pöhn
31ca2092a1 yaml parsing: script build flags can now be lists 2018-11-29 21:53:37 +01:00
Michael Pöhn
c15a7508e7 write yaml script metadata as lists 2018-11-29 21:53:37 +01:00
Michael Pöhn
afdc0be954 yaml build flag prebuild: keep writing string rathern than list of strings when theres just 1 entry 2018-11-29 21:53:37 +01:00
Michael Pöhn
942de28fa5 yaml metadata: split prebuild build field to list 2018-11-29 21:53:37 +01:00
Michael Pöhn
ddcab79817 fix .vagrant directory based vm provider lookup 2018-11-16 16:53:01 +01:00
Hans-Christoph Steiner
1f346b3149 force_exit() to make exit work with all fdroid build conditions
The build command has to use some threading stuff to handle the timeout and
locks.  This seems to prevent the command from exiting, unless this hack is
used.
2018-11-14 14:27:32 +01:00
Hans-Christoph Steiner
117d63cca5 update: use 'replace' mode to handle non-UTF8 description files
Fastlane/Triple-T app store metadata files must be in UTF-8.  Before this
would crash if they were not.  This changes the handling to just replace
the non-UTF8 chars with a �.  Here's the stacktrace:

CRITICAL: Unknown exception found!
Traceback (most recent call last):
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 164, in <module>
    main()
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 159, in main
    raise e
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 138, in main
    mod.main()
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2010, in main
    insert_localized_app_metadata(apps)
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 885, in insert_localized_app_metadata
    os.path.join(root, f))
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 700, in _set_localized_text_entry
    text = fp.read()[:limit]
  File "/usr/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 94: invalid start byte
2018-10-19 15:06:16 +02:00
Hans-Christoph Steiner
57556aceee remove redundant open() arg: encoding='utf8'
By default, open() returns a str:
https://docs.python.org/3/library/functions.html#open

By default, str is UTF-8:
https://docs.python.org/3/library/stdtypes.html#str

This used to matter on Python 2.x, but this code is 3.x only now.
2018-10-19 15:01:34 +02:00
Hans-Christoph Steiner
1d13bbf6b6 import: ensure categories from CLI flag are always a list
categories must always be a list in YAML  closes #472
2018-10-11 11:59:42 +02:00
Hans-Christoph Steiner
bfdf581201 import: use valid placeholder values for versionCode/versionName
fdroid/fdroidserver!559
closes fdroid/fdroidserver#548
2018-10-10 16:02:34 +02:00
Michael Pöhn
aea2589c67 metadata: remove redundant build flags list creation
picked out of fdroid/fdroidserver!559
2018-10-10 16:02:34 +02:00
Hans-Christoph Steiner
8b251da79f update: do not set targetSdkVersion if the APK is missing it
699b3e4c69 got it wrong for targetSdkVersion.
Also, one confusing thing is that aapt outputs "sdkVersion: '3'" for
com.politedroid_3.apk but no "sdkVersion:" for no.min.target.sdk_987.apk.
F-Droid never really supported running on android-1 or android-2, so it
seems pointless to debug support for them.
2018-10-10 15:29:02 +02:00