Commit graph

3139 commits

Author SHA1 Message Date
Michael Pöhn
06cec2041d improve bitcoin validation regex + testcases 2019-05-07 22:43:05 +02:00
Hans-Christoph Steiner
5959a395c4 nightly: archive older versions; remove archive if git mirror is full 2019-05-01 21:16:15 +02:00
Hans-Christoph Steiner
869c68dbdb deploy: if git mirror > 1GB after deleting history, delete the archive
git mirrors are meant to be an easy way to host a repo that is zero
maintenance.  They are not meant to be the canonical repo with full,
preserved archive.  This option provides the zero maintenance mode.
2019-05-01 21:16:15 +02:00
Hans-Christoph Steiner
f0460dea6e scanner: allow local Debian Maven repo file:///usr/share/maven-repo
It is now possible to build an app using only things in Debian.  Since the
buildserver will always control the contents of file:///usr/share/maven-repo,
it is the most safe repo there is.
2019-05-01 21:16:15 +02:00
Hans-Christoph Steiner
dd2f9d60f8
publish: fix stupid error in repro-signing and add integration test
stoopid mistake in ea84014f9b reported
by @CiaranG
2019-04-11 14:06:51 +02:00
Licaon_Kter
0ea03ddfaa metadata.py - typo 2019-04-08 16:30:32 +00:00
Michael Pöhn
67731470cc Revert "Merge branch 'write-yaml-overhaul' into 'master'"
This reverts merge request !630
2019-04-01 10:24:00 +00:00
Michael Pöhn
d45431aa9e Merge branch 'write-yaml-overhaul' into 'master'
use pyyaml for writing metadata instead of ruamel

Closes #611

See merge request fdroid/fdroidserver!630
2019-03-30 16:53:11 +00:00
Nico Alt
3798a884a6 Flush file before passing it to next function
When downloading a repo index, the downloaded index got written to a
file with `.write()` in a `with` clause. Before the file got actually
written to the disk, it got already passed into the next function,
resulting in a `VerificationException`:
```
JAR signature failed to verify: /tmp/tmppq2r51r0
jarsigner: java.util.zip.ZipException: zip file is empty
```

This behavior got introduced in
869cc114a3.

I've found this bug with help of Repomaker's tests: https://gitlab.com/fdroid/repomaker/merge_requests/215#note_148994053
2019-03-22 17:15:58 +01:00
Hans-Christoph Steiner
06ef99be6a deploy: show HTTP errors when virustotal error fails.
Otherwise, this was posting mystery JSON parsing errors because it was
trying to parse the HTTP error page as JSON.  This also moves the
virustotal API key to post data so it is not printed out in the stacktrace.
2019-03-22 10:54:28 +01:00
Michael Pöhn
2683b37044 yml metadata write: do not use local functions 2019-03-19 01:01:18 +01:00
Michael Pöhn
63afc0acb5 use pyyaml for writing metadata instead of ruamel 2019-03-18 22:45:35 +01:00
Hans-Christoph Steiner
c4a3354c75 update: support flavors and alternate project dirs for Triple-T
closes #542
2019-02-27 16:40:47 +01:00
Hans-Christoph Steiner
67834697e0 update: get localized text/images from Triple-T Play Publisher 2.x
closes #579
closes #567
2019-02-26 23:42:39 +01:00
Hans-Christoph Steiner
7798dcf9dd index: keep mirrors in the order specified in config.py
The mirrors are displayed in fdroidclient in the order they are received,
and there might be some reason why a repo maintainer wants them in a
specific order.  The danger is that if the mirrors are set in config.py
using "mirrors = {'foo', bar'}" they will have a randomized order since it
will be in a set.  They should be set using [] or () to have a fixed order.
2019-02-22 21:38:20 +01:00
Hans-Christoph Steiner
548f73d988 nightly: search subdirs called 'fdroid' for APKs
@MTRNord tracked down this bug while working on his app
https://gitlab.com/Nordgedanken/simplematrix
2019-02-22 21:32:33 +01:00
Michael Pöhn
35e67295e9 Merge branch 'mvglasow-master-patch-68763' into 'master'
Clarify warning message about skipped sudo commands

See merge request fdroid/fdroidserver!608
2019-02-09 18:08:23 +00:00
Marcus Hoffmann
94c6b776fe build: log buildserver guest ram + cpu 2019-02-07 19:48:03 +01:00
Pierre Rudloff
b3754af137 Add cloudrail to blacklist 2019-02-07 10:26:03 +01:00
Hans-Christoph Steiner
17dc231dc9 update: fix running without androguard
Soon, we can rip out all the aapt parsing stuff, but not yet!
2019-02-03 15:50:39 +01:00
Hans-Christoph Steiner
09bbca4a51 update: force all "SDK Version" values to int when parsing aapt 2019-02-01 09:56:36 +01:00
Michael Pöhn
dd695c650e update: treat target and min sdk version as int 2019-02-01 09:56:21 +01:00
Hans-Christoph Steiner
2907aa3ea3 fallback to v3 signatures if v1 and v2 are not available 2019-02-01 09:18:00 +01:00
Hans-Christoph Steiner
d96f5ff660 support APK Signature V2 when apksigner is installed
This was done with much help from @uniqx.  This is the first level of
supporting APK Signatures v1, v2, and v3.  This is enough to include
APKs with any combo of v1/v2/v3 signatures.  For this to work at all,
apksigner and androguard 3.3.3+ must be installed.

closes #399
2019-02-01 09:17:56 +01:00
Hans-Christoph Steiner
ea84014f9b make signer_fingerprint() accept DER encoded certs
androguard's v2 signature methods return DER-encoded certificates.  The DER-
encoded certificates are parsed from JAR Signature's raw "Signature Block
File".
2019-02-01 08:50:57 +01:00
Hans-Christoph Steiner
709d35be0e Merge branch 'patch-1' into 'master'
change glob filter for graphics to not filter out valid IETF RFC5646 locales (fixes #621 and #357)

Closes #357 and #621

See merge request fdroid/fdroidserver!611
2019-01-31 22:09:41 +00:00
Tobias Zwick
1cbd19e33d change back link to fastlane docs 2019-01-30 11:29:55 +01:00
Michael Pöhn
a787d123a2 fdroid build: hide --on-server cli flag 2019-01-29 11:23:45 +01:00
Tobias Zwick
50b1e648b2 remove listing about the expected directory structure from the docstring, instead, link the current fastlane docs 2019-01-28 19:44:03 +01:00
Tobias Zwick
412185f058 Merge branch 'master' into patch-1 2019-01-28 19:30:26 +01:00
Hans-Christoph Steiner
217e6a7c30 metadata: fix pyflakes "use ==/!= to compare str, bytes, and int literals"
$ ./hooks/pre-commit || export EXITVALUE=1
WARNING: ruby is not installed, using dummy placeholder!
fdroidserver/metadata.py:1180: use ==/!= to compare str, bytes, and int literals
fdroidserver/metadata.py:1184: use ==/!= to compare str, bytes, and int literals
fdroidserver/metadata.py:1186: use ==/!= to compare str, bytes, and int literals
fdroidserver/metadata.py:1189: use ==/!= to compare str, bytes, and int literals
ERROR: pyflakes tests failed!
2019-01-27 23:00:00 +01:00
westnordost
a227ad6e1a change glob filter for graphics to not filter out valid IETF RFC5646 locales (fixes #621 and #357)
Also, update the documentation comment to be absolutely clear.

The function used to copy images from any locale directory starting with at least two lowercase letters correctly, but only add those to the app metadata that started with two letters plus one big letter or a "-". The same problem did not exist for adding texts to the metadata.
2019-01-23 11:21:19 +00:00
Hans-Christoph Steiner
aa47e93f36 Revert "Revert "lint: check if CurrentVersion is older than oldest build entry""
This reverts commit a5ec1703c9.

fdroid/fdroiddata@42bb872b46
2019-01-21 23:00:33 +01:00
Hans-Christoph Steiner
1174b14d1e Merge branch 'neonsoftware/master' into master
buildserver: include build-tools 28.0.3
  Added test aapt output files for build-tools 28.0.3
  update.py: Still aapt output parsing, setting regex to catch 'name=' without prefixes, needed for build-tools 28.0.3

fdroid/fdroidserver!591
2019-01-21 22:39:35 +01:00
Michael von Glasow
f8f17a430d Reword warning message about sudo commands being skipped 2019-01-15 14:14:09 +00:00
Michael von Glasow
868516f3fd Clarify warning message about skipped sudo commands 2019-01-15 12:36:17 +00:00
Francesco Cervigni
984d276c1b Added test aapt output files for build-tools 28.0.3 2019-01-13 19:27:02 +01:00
Marcus Hoffmann
a5ec1703c9 Revert "lint: check if CurrentVersion is older than oldest build entry"
This reverts commit 78473e7dab.

This is currently blocked by some apps in fdroiddata being reset to a
broken state.
2019-01-10 14:48:52 +01:00
Marcus Hoffmann
e1c547cfdf Revert "Revert "Merge branch 'random-fixes' into 'master'""
This reverts commit f6f2fb0b89.

Only one of the included commit should have been reverted.
2019-01-10 14:48:29 +01:00
Marcus
f6f2fb0b89 Revert "Merge branch 'random-fixes' into 'master'"
This reverts merge request !605
2019-01-10 13:46:49 +00:00
Francesco Cervigni
64445520b0 update.py: Still aapt output parsing, setting regex to catch 'name=' without prefixes, needed for build-tools 28.0.3
build-tools 28.0.3 added a new field in the end 'compileSdkVersionCodename=', which also accidentally ends with the string 'name='.
The purpose of this regex was to catch the 'packageName' field, which is in ht eaapt ouput the exact ' name=', therefore added whe non-caracter \W prefix match.

sample aapt output (28.0.3):
package: name='com.a.b.app' versionCode='1' versionName='1.0' compileSdkVersion='28' compileSdkVersionCodename='9'

previously, regex was catching second occurence, so '9'

See merge request !582
2019-01-08 22:46:47 +01:00
Hans-Christoph Steiner
76b026e572 lint: check whether the locale was included in an f-droid.org URL 2019-01-03 21:00:01 +01:00
Hans-Christoph Steiner
78473e7dab lint: check if CurrentVersion is older than oldest build entry
This causes F-Droid to never install this app, except for manually via the
Versions list in AppDetails.
2019-01-03 21:00:01 +01:00
Hans-Christoph Steiner
a2aef721d8 add timeout to net.http_get() and index.download_repo_index()
The requests docs recommend this:
http://docs.python-requests.org/en/master/user/quickstart/#timeouts

And mirror-monitor was hanging forever on a bad mirror.
2019-01-03 20:26:40 +01:00
Hans-Christoph Steiner
33b097a3aa deploy: upload to VirusTotal using index-v1.json or index-v1.jar
* A mirror of the repo will have only index-v1.jar.
* A buildserver with separate signing might have only index-v1.json
2018-12-17 14:33:02 +01:00
Hans-Christoph Steiner
869cc114a3 create index.get_index_from_jar() for parsing index-v1.json
Also removes redundant comments, should be in the doc string!
2018-12-17 14:30:05 +01:00
Hans-Christoph Steiner
13016c5d63 verify: add --output-json option to generate status reports
This has to do custom dict/encoder/decoder tricks in order to make sure
that each run of `fdroid verify` on the same files doesn't add duplicate
entries.
2018-12-07 13:13:10 +01:00
Hans-Christoph Steiner
e85256e248 verify: rename var remoteapk to remote_apk to fit other style 2018-12-07 11:19:26 +01:00
Hans-Christoph Steiner
51d961630b fix aapt scraping of <uses-permission> with maxSdkVersion
3e0d1beb09 changed this logic a bit, and it
wasn't quite right.  Then changing the SDK Versions to integers everywhere
seemed to bring this out more.
2018-12-06 13:42:45 +01:00
Hans-Christoph Steiner
653d5fbd88 update: fix running on trusty and things with old PIL/Pillow 2018-12-06 12:49:48 +01:00