Commit graph

3677 commits

Author SHA1 Message Date
Hans-Christoph Steiner
1106795583 deploy: stop uploading to androidobservatory if its already up there 2020-05-14 11:55:09 +02:00
Hans-Christoph Steiner
30f2d62597 import: fix --omit-disable flag, its a boolean 2020-05-14 11:55:09 +02:00
Hans-Christoph Steiner
ba854ab24a support Cordova's gradle file
https://cordova.apache.org/docs/en/latest/guide/platforms/android/index.html
2020-05-14 11:55:09 +02:00
Hans-Christoph Steiner
77367f1bff server: disable extraneous HTTP errors in upload_apk_to_virustotal
This should be in the lowest level so that both the low level call,
upload_apk_to_virustotal(), and the next level up, upload_to_virustotal(),
both inherit this.
2020-05-14 11:55:09 +02:00
Torsten Grote
db96753902
Fix nightly --archive-older command line option 2020-05-11 10:23:26 -03:00
Michael Pöhn
810a6587df add --force-yamllint option to lint 2020-04-25 17:39:23 +02:00
Michael Pöhn
b076e8cba7 add srclib spec parser 2020-04-25 17:22:48 +02:00
Michael Pöhn
ed2c5f6f5b run yamllint on srclibs when running 'fdroid lint' 2020-04-25 15:57:21 +02:00
Michael Pöhn
975538a7a7 make yamllint optional 2020-04-24 15:47:47 +02:00
Michael Pöhn
4e69ff582f run yamllint when parsing failed and also on fdroid lint runs 2020-04-24 15:47:31 +02:00
Jochen Sprickerhof
30c654459f Use ImportError for Python < 3.6 (Closes: !734)
ModuleNotFoundError is a subclass of ImportError, so this should not
change anything.
2020-04-17 23:26:06 +02:00
Michael Pöhn
911e86fc87 support copying yml srclibs to buildserver vm during builds 2020-04-16 11:32:37 +02:00
Michael Pöhn
5741e6930b yml srclibs: support Subdir as list 2020-04-16 11:31:59 +02:00
Michael Pöhn
1ac7d612b1 yml srclibs: support multiline Prepare 2020-04-16 11:31:59 +02:00
Michael Pöhn
aa020af040 srclibs: forward yml parsing error cause 2020-04-16 11:31:59 +02:00
Michael Pöhn
ee3d8d2f18 srclib: yml parsing + tests for yml and txt 2020-04-16 11:31:59 +02:00
Michael Pöhn
2c86d38028 srclib: no spaces in dict key 2020-04-16 11:31:59 +02:00
Michael Pöhn
c6a97939f1 rename parse_srclib to parese_txt_srclib + test case 2020-04-16 11:31:59 +02:00
Hans-Christoph Steiner
c47f9ef123
index: xml.dom.minidom no longer sorts attribs
It seems now that xml.dom.minidom preserves the order of attributes, rather
than sorting them.  We assume alpha-sort, so this manually

This diff in the test suite running on Debian/testing pointed it out:
https://gitlab.com/fdroid/fdroidserver/-/jobs/486970383

```diff
--- /builds/fdroid/fdroidserver/tests/repo/index.xml	2020-04-11 13:36:57.000000000 +0000
+++ repo/index.xml	2020-04-11 13:41:44.000000000 +0000
@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="utf-8"?>
 <fdroid>
-	<repo icon="fdroid-icon.png" name="My First F-Droid Repo Demo" pubkey="308204e1308202c9a003020102020434597643300d06092a864886f70d01010b050030213110300e060355040b1307462d44726f6964310d300b06035504031304736f7661301e170d3136303931333230313930395a170d3434303133303230313930395a30213110300e060355040b1307462d44726f6964310d300b06035504031304736f766130820222300d06092a864886f70d01010105000382020f003082020a028202010086ef94b5aacf2ba4f38c875f4194b44f5644392e3715575d7c92828577e692c352b567172823851c8c72347fbc9d99684cd7ca3e1db3e4cca126382c53f2a5869fb4c19bdec989b2930501af3e758ff40588915fe96b10076ce3346a193a0277d79e83e30fd8657c20e35260dd085aa32eac7c4b85786ffefbf1555cafe2bc928443430cdbba48cfbe701e12ae86e676477932730d4fc7c00af820aef85038a5b4df084cf6470d110dc4c49ea1b749b80b34709d199b3db516b223625c5de4501e861f7d261b3838f8f616aa78831d618d41d25872dc810c9b2087b5a9e146ca95be740316dcdbcb77314e23ab87d4487913b800b1113c0603ea2294188b71d3e49875df097b56f9151211fc6832f9790c5c83d17481f14ad37915fd164f4fd713f6732a15f4245714b84cd665bdbd085660ea33ad7d7095dcc414f09e3903604a40facc2314a115c0045bb50e9df38efb57e1b8e7cc105f340a26eeb46aba0fa6672953eee7f1f92dcb408e561909bbd4bdf4a4948c4d57c467d21aa238c34ba43be050398be963191fa2b49828bc1e4eeed224b40dbe9dc3e570890a71a974a2f4527edb1b07105071755105edcb2af2f269facfb89180903a572a99b46456e80d4a01685a80b233278805f2c876678e731f4ec4f52075aeef6b2b023efbb8a3637ef507c4c37c27e428152ec1817fcba640ad601cb09f72f0fbe2d274a2410203010001a321301f301d0603551d0e04160414c28bf33dd5a9a17338e5b1d1a6edd8c7d141ed0b300d06092a864886f70d01010b0500038202010084e20458b2aafd7fc27146b0986f9324f4260f244920417a77c9bf15e2e2d22d2725bdd8093ec261c3779c3ca03312516506f9410075b90595b41345956d8eb2786fb5994f195611382c2b99dba13381b0100a30bc9e6e47248bf4325e2f6eec9d789216dc7536e753bf1f4be603d9fa2e6f5e192b4eb988b8cdb0bb1e8668a9225426f7d4636479f73ed24ad1d2657c31e63c93d9679b9080171b3bd1bf10a3b92b80bd790fbf62d3644900cd08eae8b9bf9c2567be98dc8cdd2ae19a8d57a3e3e2de899f81f1279f578989e6af906f80c8c2b67651730ee7e568c1af5bcb845b6d685dc55332a9984aeceaea3b7e883447edf1c76b155d95253e39b9710eaa22efa6c81468829702b5dce7126538f3ca70c2f0ad9a5795435fdb1f715f20d60359ef9a9926c7050116e802df651727447848827815f70bd82af3cedd08783156102d2d8ce995c4c43b8e47e91a3e6927f3505a5d395e6bebb84542c570903eeab4382a1c2151f1471c7a06a34dc4d268d8fa72e93bdcd2dccc4302ecac47b9e7e3d8bc9b46d21cd097874a24d529548018dc190ff568c6aa428f0a5eedff1a347730931c74f19277538e49647a4ad7254f4c1ec7d4da12cce9e1fad9607534e66ab40a56b473d9d7e3d563fd03cad2052bad365c5a29f8ae54f09b60dbca3ea768d7767cbe1c133ca08ce725c1c1370f4aab8e5b6e286f52dc0be8d0982b5a" timestamp="1480431575" url="https://MyFirstFDroidRepo.org/fdroid/repo" version="21">
+	<repo name="My First F-Droid Repo Demo" icon="fdroid-icon.png" url="https://MyFirstFDroidRepo.org/fdroid/repo" version="21" timestamp="1480431575" pubkey="308204e1308202c9a003020102020434597643300d06092a864886f70d01010b050030213110300e060355040b1307462d44726f6964310d300b06035504031304736f7661301e170d3136303931333230313930395a170d3434303133303230313930395a30213110300e060355040b1307462d44726f6964310d300b06035504031304736f766130820222300d06092a864886f70d01010105000382020f003082020a028202010086ef94b5aacf2ba4f38c875f4194b44f5644392e3715575d7c92828577e692c352b567172823851c8c72347fbc9d99684cd7ca3e1db3e4cca126382c53f2a5869fb4c19bdec989b2930501af3e758ff40588915fe96b10076ce3346a193a0277d79e83e30fd8657c20e35260dd085aa32eac7c4b85786ffefbf1555cafe2bc928443430cdbba48cfbe701e12ae86e676477932730d4fc7c00af820aef85038a5b4df084cf6470d110dc4c49ea1b749b80b34709d199b3db516b223625c5de4501e861f7d261b3838f8f616aa78831d618d41d25872dc810c9b2087b5a9e146ca95be740316dcdbcb77314e23ab87d4487913b800b1113c0603ea2294188b71d3e49875df097b56f9151211fc6832f9790c5c83d17481f14ad37915fd164f4fd713f6732a15f4245714b84cd665bdbd085660ea33ad7d7095dcc414f09e3903604a40facc2314a115c0045bb50e9df38efb57e1b8e7cc105f340a26eeb46aba0fa6672953eee7f1f92dcb408e561909bbd4bdf4a4948c4d57c467d21aa238c34ba43be050398be963191fa2b49828bc1e4eeed224b40dbe9dc3e570890a71a974a2f4527edb1b07105071755105edcb2af2f269facfb89180903a572a99b46456e80d4a01685a80b233278805f2c876678e731f4ec4f52075aeef6b2b023efbb8a3637ef507c4c37c27e428152ec1817fcba640ad601cb09f72f0fbe2d274a2410203010001a321301f301d0603551d0e04160414c28bf33dd5a9a17338e5b1d1a6edd8c7d141ed0b300d06092a864886f70d01010b0500038202010084e20458b2aafd7fc27146b0986f9324f4260f244920417a77c9bf15e2e2d22d2725bdd8093ec261c3779c3ca03312516506f9410075b90595b41345956d8eb2786fb5994f195611382c2b99dba13381b0100a30bc9e6e47248bf4325e2f6eec9d789216dc7536e753bf1f4be603d9fa2e6f5e192b4eb988b8cdb0bb1e8668a9225426f7d4636479f73ed24ad1d2657c31e63c93d9679b9080171b3bd1bf10a3b92b80bd790fbf62d3644900cd08eae8b9bf9c2567be98dc8cdd2ae19a8d57a3e3e2de899f81f1279f578989e6af906f80c8c2b67651730ee7e568c1af5bcb845b6d685dc55332a9984aeceaea3b7e883447edf1c76b155d95253e39b9710eaa22efa6c81468829702b5dce7126538f3ca70c2f0ad9a5795435fdb1f715f20d60359ef9a9926c7050116e802df651727447848827815f70bd82af3cedd08783156102d2d8ce995c4c43b8e47e91a3e6927f3505a5d395e6bebb84542c570903eeab4382a1c2151f1471c7a06a34dc4d268d8fa72e93bdcd2dccc4302ecac47b9e7e3d8bc9b46d21cd097874a24d529548018dc190ff568c6aa428f0a5eedff1a347730931c74f19277538e49647a4ad7254f4c1ec7d4da12cce9e1fad9607534e66ab40a56b473d9d7e3d563fd03cad2052bad365c5a29f8ae54f09b60dbca3ea768d7767cbe1c133ca08ce725c1c1370f4aab8e5b6e286f52dc0be8d0982b5a">
 		<description>This is a repository of apps to be used with F-Droid. Applications in this repository are either official binaries built by the original application developers, or are binaries built from source by the admin of f-droid.org using the tools on https://gitlab.com/u/fdroid. </description>
 		<mirror>http://foobarfoobarfoobar.onion/fdroid/repo</mirror>
 		<mirror>https://foo.bar/fdroid/repo</mirror>
@@ -94,9 +94,9 @@
 			<added>2017-12-22</added>
 			<sig>056c9f1554c40ba59a2103009c82b420</sig>
 			<permissions>ACCESS_NETWORK_STATE,ACCESS_WIFI_STATE,CHANGE_WIFI_MULTICAST_STATE,INTERNET,READ_EXTERNAL_STORAGE,WRITE_EXTERNAL_STORAGE</permissions>
-			<uses-permission maxSdkVersion="18" name="android.permission.READ_EXTERNAL_STORAGE"/>
-			<uses-permission maxSdkVersion="18" name="android.permission.WRITE_EXTERNAL_STORAGE"/>
-			<uses-permission-sdk-23 maxSdkVersion="27" name="android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS"/>
+			<uses-permission name="android.permission.READ_EXTERNAL_STORAGE" maxSdkVersion="18"/>
+			<uses-permission name="android.permission.WRITE_EXTERNAL_STORAGE" maxSdkVersion="18"/>
+			<uses-permission-sdk-23 name="android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS" maxSdkVersion="27"/>
 		</package>
 	</application>
 	<application id="fake.ota.update">
@@ -182,9 +182,9 @@
 			<added>2013-12-31</added>
 			<sig>eb41d4d6082bb3e81c3d58dbf7fc7332</sig>
 			<permissions>ACCESS_NETWORK_STATE,ACCESS_WIFI_STATE,BLUETOOTH,BLUETOOTH_ADMIN,CHANGE_NETWORK_STATE,CHANGE_WIFI_MULTICAST_STATE,CHANGE_WIFI_STATE,INTERNET,NFC,RECEIVE_BOOT_COMPLETED</permissions>
-			<uses-permission maxSdkVersion="22" name="android.permission.ACCESS_NETWORK_STATE"/>
-			<uses-permission maxSdkVersion="18" name="android.permission.BLUETOOTH_ADMIN"/>
-			<uses-permission-sdk-23 maxSdkVersion="25" name="android.permission.WRITE_SETTINGS"/>
+			<uses-permission name="android.permission.ACCESS_NETWORK_STATE" maxSdkVersion="22"/>
+			<uses-permission name="android.permission.BLUETOOTH_ADMIN" maxSdkVersion="18"/>
+			<uses-permission-sdk-23 name="android.permission.WRITE_SETTINGS" maxSdkVersion="25"/>
 		</package>
 	</application>
 	<application id="obb.main.twoversions">
```
2020-04-15 21:53:39 +02:00
Jochen Sprickerhof
86beac22e2 Use libarchive instead of the Python implementation 2020-04-15 18:27:13 +00:00
Hans-Christoph Steiner
e85573d0e1 Merge branch 'git-push-logging' into 'master'
Show debug messages to give clue why pushing git mirror failed

See merge request fdroid/fdroidserver!731
2020-04-15 12:56:04 +00:00
Hans-Christoph Steiner
f39732987e Merge branch 'patch-1' into 'master'
Import - sudo refresh repos too and use the newer npm

See merge request fdroid/fdroidserver!730
2020-04-15 12:53:56 +00:00
Izzy
46a464a74b
get_app_from_url: add codeberg.org as git url 2020-04-15 09:13:08 +02:00
Torsten Grote
7d69f04b94
Show debug messages to give clue why pushing git mirror failed 2020-04-10 13:34:46 -03:00
Licaon_Kter
463f6823de Import - sudo refresh repos too and use the newer npm 2020-04-08 10:25:53 +00:00
Hans-Christoph Steiner
138bc73668
import: --omit-disable option to easily work with generated files 2020-03-11 22:13:47 +01:00
Hans-Christoph Steiner
1bb9cf43e1
import: ensure gradle: yes is added for detected Gradle builds 2020-03-11 22:13:45 +01:00
Hans-Christoph Steiner
a1e17d2b2c
deploy: retry virustotal 30 seconds after rate limiting, and log 2020-03-11 22:13:42 +01:00
Hans-Christoph Steiner
b7901952a1 deploy: make androidobservatory and virustotal functions reusable
This should not change the logic at all, just make the loop runs into
standalone functions.
2020-03-11 22:13:39 +01:00
Hans-Christoph Steiner
733e7be1b3 set F-Droid HTTP Headers globally 2020-03-11 13:41:13 +01:00
Hans-Christoph Steiner
ab2291475b import: mv reusable functions to common.py to avoid import_proxy.py
import is a strict keyword in Python, so it is not possible to import a
module called 'import', even with things like:

* import fdroidserver.import
* from fdroidserver import import
2020-03-11 13:41:13 +01:00
Marcus Hoffmann
56ee7fe765 opencollecive: fix allowed characters
The url slug used by opencollective allows case variations, they all
lead back to the canonical lowercase variant though.

Apart from ascii characters and numbers only - and _ are allowed.
2020-03-10 16:26:34 +01:00
Marcus
9d24f2e4a7 add opencollective metadata and index field 2020-03-10 14:56:03 +00:00
fuwa
2d63ab66f5 libvirt: various fixes related to the makebuildserver and fdroid build commands 2020-03-09 15:06:19 +00:00
Torsten Grote
14208a6761
Add stats support to nightly repo
This will keep track of when APKs were added
and show these dates properly in the client
instead of always resetting all dates to the latest update.
2020-03-02 14:17:42 -03:00
Hans-Christoph Steiner
b5bce9f178
build: do not write/rsync status JSON when running --on-server
!717 !716
2020-02-20 16:40:19 +01:00
Hans-Christoph Steiner
b6e6ce87cb
build: has no pretty option (closes #742) 2020-02-19 20:29:54 +01:00
Hans-Christoph Steiner
21f5bd73a7
build: fix logging failed builds to the status JSON (closes #741) 2020-02-19 20:29:51 +01:00
Hans-Christoph Steiner
bf6004b08e
rsync status JSON as first and last steps of command runs 2020-02-19 14:50:35 +01:00
Hans-Christoph Steiner
c19ef45fd5
build: write status JSON 2020-02-19 14:50:32 +01:00
Hans-Christoph Steiner
55935d9a41
signindex: write status JSON 2020-02-19 14:50:30 +01:00
Hans-Christoph Steiner
c965e7d969
deploy: write status JSON 2020-02-19 14:50:29 +01:00
Hans-Christoph Steiner
ddbdff83bd
gpgsign: write status JSON 2020-02-19 14:50:26 +01:00
Hans-Christoph Steiner
4bb590b6e5
publish: write status JSON 2020-02-19 14:50:24 +01:00
Hans-Christoph Steiner
e8a6d0f813
checkupdates: write status JSON 2020-02-18 23:45:52 +01:00
Hans-Christoph Steiner
d16478b10b
update: write status in JSON repo file, using new internal API 2020-02-18 23:45:48 +01:00
Hans-Christoph Steiner
5459a461db
common.get_head_commit_id() to get string commit ID from HEAD 2020-02-18 23:16:53 +01:00
Hans-Christoph Steiner
fafaa8f81f
standardize on path.endswith() for file extension checks
Its already widely used, this just removes the exceptions.  It is also more
Pythonic
2020-02-18 23:16:51 +01:00
Hans-Christoph Steiner
202602937d
import: ignore results if build/appid dir already exists 2020-02-18 23:16:49 +01:00
Hans-Christoph Steiner
32a29b3304
import: get git commit ID from git repo, e.g. master instead of '?' 2020-02-18 23:16:44 +01:00