Commit graph

3139 commits

Author SHA1 Message Date
Hans-Christoph Steiner
bf6004b08e
rsync status JSON as first and last steps of command runs 2020-02-19 14:50:35 +01:00
Hans-Christoph Steiner
c19ef45fd5
build: write status JSON 2020-02-19 14:50:32 +01:00
Hans-Christoph Steiner
55935d9a41
signindex: write status JSON 2020-02-19 14:50:30 +01:00
Hans-Christoph Steiner
c965e7d969
deploy: write status JSON 2020-02-19 14:50:29 +01:00
Hans-Christoph Steiner
ddbdff83bd
gpgsign: write status JSON 2020-02-19 14:50:26 +01:00
Hans-Christoph Steiner
4bb590b6e5
publish: write status JSON 2020-02-19 14:50:24 +01:00
Hans-Christoph Steiner
e8a6d0f813
checkupdates: write status JSON 2020-02-18 23:45:52 +01:00
Hans-Christoph Steiner
d16478b10b
update: write status in JSON repo file, using new internal API 2020-02-18 23:45:48 +01:00
Hans-Christoph Steiner
5459a461db
common.get_head_commit_id() to get string commit ID from HEAD 2020-02-18 23:16:53 +01:00
Hans-Christoph Steiner
fafaa8f81f
standardize on path.endswith() for file extension checks
Its already widely used, this just removes the exceptions.  It is also more
Pythonic
2020-02-18 23:16:51 +01:00
Hans-Christoph Steiner
202602937d
import: ignore results if build/appid dir already exists 2020-02-18 23:16:49 +01:00
Hans-Christoph Steiner
32a29b3304
import: get git commit ID from git repo, e.g. master instead of '?' 2020-02-18 23:16:44 +01:00
Hans-Christoph Steiner
271b74af7d fix remove_signing_keys() for Kotlin gradles files (*.gradle.kts) 2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
68b793e308 support kotlin "*.gradle.kts" files in more places
closes #613
2020-02-13 22:32:51 +01:00
Marcus Hoffmann
9ae41cc1ff add support for gradle kotlin scripts
https://guides.gradle.org/migrating-build-logic-from-groovy-to-kotlin/
Closes fdroid/fdroidserver#613

cherry-picked from Bubu/fdroidserver@7d2e9f9c
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
83ffeb855f prefer build.gradle with Android Plugin as source of package/version/code
These days, the location that overrides all the others is in the android{}
block of the build.gradle file that loads the com.android.application
plugin.  So this should be the preferred place to read these values.

test files GPL licensed: https://github.com/Integreight/1Sheeld-Android-App
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
ad92b4c678 scanner: add --force option for scanning disabled apps/builds 2020-02-13 13:58:13 +01:00
Hans-Christoph Steiner
e037ee5972 import: generate skeleton build fields for react-native and flutter 2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
1153ac24fd import: overhaul URL validation to use urllib.parse
Python provides us a lovely URL parser with some level of validation built
in.  The parsed URL is then much easier to validate.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
e9a6c84efd import: split URL parsing from code cloning
This makes things testable and easier to follow.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
bfe587979d import: make it work most of the time with git repos
This includes real tests too.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
697a0641f1
fix sudo syntax from 7a1fa6e857
fdroid/fdroidserver!713
2020-02-11 19:44:18 +01:00
Hans-Christoph Steiner
7a1fa6e857 build: keep DEBIAN_FRONTEND env var when running sudo: script
closes #636
2020-02-05 20:36:25 +01:00
Hans-Christoph Steiner
6b03f995be deploy: check if file is on androidobservatory before uploading
Check if file is on androidobservatory before uploading.
androidobservatory.org redirects to a GET request after the POST, and
this seems to partially fail with requests.  The checking code was
broken.
2020-02-05 20:34:03 +01:00
Hans-Christoph Steiner
140bd29c94 Merge branch 'random-bag-of-fies' into 'master'
random bag of fixes

Closes #719

See merge request fdroid/fdroidserver!710
2020-02-05 19:12:19 +00:00
licaon-kter
135c92f3d0 Add Microsoft Appcenter SDK to forbidden list 2020-02-02 03:50:54 +02:00
Hans-Christoph Steiner
5c82956561
mirror: make .asc downloading opt-in with --pgp-signatures
Lots of third party repos do not use .asc PGP signatures at all, so having
this optional prevents tons of 404 Not Found errors.
2020-01-31 15:38:02 +01:00
Hans-Christoph Steiner
6ce22bff56
update: strip newlines from name/summary/video entries
These entries are hardcoded as a single line in all the app stores, so
newlines should be stripped to get the data simple to use.  This is in
contrast with the on-disk format for Fastlane and Triple-T, which includes
a newline in the title.txt and short_description.txt files.  I think all
files in those systems are normalized to end in a newline.
2020-01-31 15:37:59 +01:00
Hans-Christoph Steiner
3b5e3a62a3
move getappname+getcvname to checkupdates, the only place they're used 2020-01-31 15:37:57 +01:00
Hans-Christoph Steiner
3df276cc3c
fix all bandit B310 urllib_urlopen
"Audit url open for permitted schemes. Allowing use of ‘file:’’ or custom
schemes is often unexpected."

https://bandit.readthedocs.io/en/latest/blacklists/blacklist_calls.html#b310-urllib-urlopen
2020-01-31 15:37:30 +01:00
Marcus Hoffmann
8961485125 remove dscanner subcommand
It's unused and unmaintained. It could  potentially be revived as a
plugin at a later point.
2020-01-31 14:17:12 +01:00
Michael Pöhn
69fc38c668 repair fdroid --version 2020-01-28 11:43:09 +01:00
Michael Pöhn
7a4254efa2 make main script pythonic 2020-01-28 11:43:09 +01:00
Marcus
3bdbbfd45b Merge branch 'longer_titles' into 'master'
bump title char_limit to 50

Closes fdroiddata#1906 and #726

See merge request fdroid/fdroidserver!702
2020-01-14 13:54:41 +00:00
Hans-Christoph Steiner
df7d7adf78
mirror: optionally fetch build logs and src tarballs 2020-01-14 11:36:11 +01:00
Hans-Christoph Steiner
e76a0c9d6a
git_mirror_size_limit config option to set max git mirror size
GitHub and GitLab have some kinds of limits on how big a git repo can be,
this makes that option configurable.  This also is very useful for tests.
2020-01-14 11:36:08 +01:00
Hans-Christoph Steiner
4fa11ef4fc
deploy: detect virustotal size limits while uploading
This prevents the size limits from blocking the whole deploy.
2020-01-14 11:19:02 +01:00
Hans-Christoph Steiner
058b47e484
deploy: fix virustotal report fetching, use GET and query string 2020-01-14 11:19:00 +01:00
Hans-Christoph Steiner
f2b48575e6
deploy: github/gitlab use bare git repos, only size the .git/ dir
needs test: generate giant APKs by including AndroidManifest.xml and and
large file from /dev/urandom, then sign it.  Then add those to the git repo.
2020-01-14 11:18:53 +01:00
Marcus Hoffmann
96ba194d65 bump title char_limit to 50
This follows google play in doing so.
Closes fdroidserver#726, fdroid/fdroiddata#1906.
2020-01-08 14:20:18 +01:00
Hans-Christoph Steiner
098bce2134 Merge branch 'git_fetch_force_tags' into 'master'
git force fetch tags

Closes #718

See merge request fdroid/fdroidserver!699
2020-01-02 13:45:22 +00:00
Gerhard Olsson
cf9bff1d56 AutoUpdateMode: Do not include + in suffix
Add suffix to version only
2020-01-02 13:27:12 +00:00
Marcus Hoffmann
83526e09a3 update: fix unarchiving of allowed disabled algorithm
6d0b1bbe6fae0909683f2c6a154515bc4bfcb674 didn't handle the
allow_disabled_algorithm case at all, so we add it back.

This additionally fixes a (previously existing) bug where setting
allow_disabled_algorithms to True didn't move apks back from archive to
repo. Introduce a new test for this.

The disabled_algorithm archiving logic is still all over the place so
ideally that needs a future refactor.
2020-01-01 15:25:43 +01:00
Marcus Hoffmann
a7a83e1ee3 update: keep CVC of an app out of the archive
We want to keep the stable version (CV) of an app in /repo with highest
priority and only move it to /archive when ArchivePolicy is set to 0.

For this to work we need two changes here:

1) when sorting apks by version code we insert the apk corresponding to
the CVC of the app afterwards with highest priority. So when walking the
list of apks afterwards the CVC apk is always kept first.

2) Instead of the two pass algorithm of moving things back and forth to
the archive we instead figure out where each apk of an app goes in first
before actually moving them into the right place.

Fixes: fdroid/fdroidserver#385
2020-01-01 15:25:43 +01:00
Marcus Hoffmann
3403402fbc git force fetch tags
The behaviour of fetching tags changed in git 2.20. We need to force
fetch tags to restore the earlier behaviour and make fdroid git
operations work with newer git versions.

Closes fdroid/fdroidserver#718
2019-12-23 14:48:52 +01:00
Hans-Christoph Steiner
e9830fc4ab Merge branch 'faster_yaml' into 'master'
metadata: use yaml C implementation when available

Closes #716

See merge request fdroid/fdroidserver!696
2019-12-23 10:09:50 +00:00
Marcus Hoffmann
01d00d54ca metadata: use yaml C implementation when available
This is an order of magnitude faster. Requires the C yaml bindings to be
installed.

Fixes fdroid/fdroidserver#716
2019-12-23 01:59:33 +01:00
Marcus Hoffmann
5642dc56ae remove unused function
The users of this were removed years ago in
013315bf10.
2019-12-23 01:29:06 +01:00
Michael Pöhn
d2481375bb Merge branch 'archive_description' into 'master'
archive description

See merge request fdroid/fdroidserver!694
2019-12-15 16:34:16 +00:00
Hans-Christoph Steiner
7d40e89341
checkupdates: split out vercode parsing into testable function 2019-12-03 23:51:48 +01:00