Commit graph

2242 commits

Author SHA1 Message Date
Marcus Hoffmann
5ebde251b1 checkupdates: use html.unescape instead of HTMLParser.unescape
HTMLParser.unescape is only an internal method and deprecated.
This requires Pyhton 3.4.
2017-09-13 01:42:15 +02:00
Marcus Hoffmann
921524081c checkupdates: fix google play check method
This has never worked with python3 and was also not properly adapted to
apps as objects. So we fix both of these issues.
2017-09-13 01:40:09 +02:00
Hans-Christoph Steiner
253bd92df3 Merge branch 'refuse-publishing-duplicate-apks' into 'master'
proper error message when publishing with the same app in repo and unsigned

See merge request !329
2017-09-12 15:33:51 +00:00
Marcus Hoffmann
5f5d3ea896 scanner: promote jar and aar files present in the repo to errors
This makes sure we we don't include any prebuilt code in the build process.
2017-09-11 17:05:24 +02:00
Marcus Hoffmann
15c4492625 remove gradle-wrapper.jar during scan
We make sure to not use the gradle wrapper anyway so this should have no
impact.
2017-09-11 17:05:21 +02:00
Marcus Hoffmann
af2eb2bd50 scanner: warn on aar files 2017-09-11 17:05:16 +02:00
Marcus Hoffmann
5d9eb9b295 scanner: rename variables, use os.path.relpath
* rename fd -> path_in_build_dir
* rename fp -> filepath
* use os.path.reldir instead of string manipulation
2017-09-11 16:49:39 +02:00
Marcus Hoffmann
1ee3f9aad5 scanner: more descriptive variable names from os.walk
This uses the names from python documentation of os.walk().
2017-09-11 16:49:30 +02:00
Michael Pöhn
328c1ad0c7 proper error message when publishing with the same app in repo and unsigned 2017-09-07 12:19:48 +02:00
Hans-Christoph Steiner
3aee9b3f11 Merge branch 'close-subprocess-file-handles' into 'master'
make sure file-streams of subprocesses get closed

See merge request !328
2017-09-07 09:49:51 +00:00
Michael Pöhn
a718c75b05 make sure file-streams of subprocesses get closed 2017-09-07 02:36:58 +02:00
Michael Pöhn
3e6dfacf6c added signatures subcommand 2017-09-07 00:03:35 +02:00
Marcus Hoffmann
b148b82f83 add comments to gradle output directories 2017-08-26 17:03:13 +02:00
Marcus Hoffmann
b114b44020 handle gradle-plugin 3.0 output apk location
This commit adds support for new gradle plugin 3.0 output directories.
The new structure looks like this:

build/outputs/apk/<flavour1><Flavour2>/release/*.apk

Note the capitalization on the different flavour components.
So if we build a specific flavour combination we add this combination to
the path where we look for an output .apk.

Closes #363

Based on !320 by: Michel Le Bihan <michel@lebihan.pl>
Signed-off-by: Marcus Hoffmann <bubuiic@aol.com>
2017-08-26 16:03:31 +02:00
Marcus Hoffmann
a9a19527c7 metadata: also read .fdroid.txt metadata
This is documented as working here:
https://f-droid.org/docs/Building_Applications/
(second to last paragraph)

fixes #364
2017-08-22 00:03:43 +02:00
relan
6f05cf85f9 makebuildserver: upgrade NDK r15b to r15c 2017-08-11 10:19:40 +03:00
Hans-Christoph Steiner
c67f8e349c metadata: handle empty YAML files without crashing
!311
2017-07-26 18:21:28 -07:00
Hans-Christoph Steiner
7bd171480f update: test metadata creation using internal and external templates
There is a hardcoded template in update.py, and there is also the
possibility for the user to create a template.yml.  This tests both of them
and cleans up the related code a bit.

#352 !310
2017-07-26 15:34:13 -07:00
Izzy
cbd420e490 preserve order and formatting on creating Metadata from Yaml template (closes #352) 2017-07-26 09:22:01 +02:00
Izzy
b787ec9e34 Updating syntax check for Flattr (see #284) 2017-07-20 20:26:25 +02:00
Hans-Christoph Steiner
c1e9379f67 remove errant trailing slash added in 7613c18dd8 2017-07-20 00:00:08 +02:00
Hans-Christoph Steiner
0a953814e2 update: always include name/summary/desc in index.xml if available
With the new localization support, the name/summary/description in the
metadata file becomes the global override.  So most apps are not going to
have those fields present in their metadata file.  This fixes the index.xml
generation to fall back to the localized versions of those fields when they
are not set in the metadata field.

https://forum.f-droid.org/t/what-has-happend-to-osmand
2017-07-19 15:07:50 +02:00
Hans-Christoph Steiner
7613c18dd8 verify: if downloading from /repo/ fails, try /archive/
The Builds entries in metadata/ files do not easily say whether a
given APK is in the repo/ or the archive/.  So it should also try to
download the official APK from the archive/ when verifying.
2017-07-19 15:07:50 +02:00
Hans-Christoph Steiner
67d98c5a36 keep .apk file ext when diffoscope'ing Binaries: 2017-07-19 15:07:50 +02:00
Hans-Christoph Steiner
2ecaf6ef8d rewritemeta: only print file type if its changing 2017-07-19 15:07:50 +02:00
Hans-Christoph Steiner
1f7f9d403c server: report errors pushing to git mirrors
This makes `fdroid server update` fail if pushing to one of the git mirrors
fails.  This is what happens if the other methods fail, e.g. rsync or S3.

closes #347
2017-07-19 15:07:50 +02:00
Hans-Christoph Steiner
93caf27319 server: include gitlab raw URLs as git mirrors
gitlab serves raw files from a CDN, so its appropriate to use the raw URL.
@pserwylo @grote and I discussed it and found a reference, but I can't find
that reference now.

Since the client will try the next mirror if one fails, it makes sense to
include both the gitlab raw and gitlab pages URLs to the mirror.  The
gitlab pages deploy process is still a bit flaky anyway.
2017-07-19 15:07:50 +02:00
Hans-Christoph Steiner
6d8e916491 server: only rm git mirror if the git history is getting too large
git hosts like github, gitlab, bitbucket usually allow 1 gig repos. This
changes the git mirroring behavior to keep the history until the repo hits
1 gig.  Keeping history makes updates a lot faster, since the whole repo
does not need to be pushed on each update.
2017-07-19 13:03:17 +02:00
Hans-Christoph Steiner
d8954fc033 update: force checkout .gitlab-ci.yml when updating git mirrors
closes #309
2017-07-17 11:38:15 +02:00
Hans-Christoph Steiner
523b5f4777 server: smooth out btlog transfer for offline signing setups
It turns out it is error prone to `git push` to a non-bare git repo. For
the offline signing machine, the git remote needs to be a regular git repo
in a directory on a thumbdrive so that once the thumbdrive is plugged into
an online machine, that git repo can be transferred to the online machine.
2017-07-17 11:38:15 +02:00
Izzy
54a646bd93 have fallback for XML icons also consider res/mipmap*
several apps (e.g.
[FastHub](http://apt.qumran.org/fdroid/index/apk/com.fastaccess.github)
and [Monety](http://apt.qumran.org/fdroid/index/apk/open.currency)) have
their ic_launcher.png files not in res/drawable*dpi/, but in
res/mipmap*/ -- so the regex has been adjusted by this patch.

Additionally: if the only icon for a given resolution was an XML without
existing fallback-PNG, it should be considered "non existent" (ie.
"empty_density").
2017-07-17 00:54:33 +02:00
Hans-Christoph Steiner
830e788401 Merge branch 'metadata' into 'master'
--create-metadata: use yaml.dump() and (if exists) template.yml

See merge request !305
2017-07-10 09:08:45 +00:00
Izzy
dd69490821 --create-metadata: only set default empty values if not using template.py 2017-07-08 14:41:19 +02:00
Izzy
4d375a585d enable user to have presets for metadata by using a template.yml (see #345) 2017-07-08 14:21:49 +02:00
Izzy
732de91aa7 make --create-metadata use yaml.dump instead of ruamel (solves #345) 2017-07-08 14:07:11 +02:00
Hans-Christoph Steiner
25f96e1911 ensure that mirror URLs always include the repodir
Since the mirror URLs are per repo section (repo/archive), the mirror URLs
must include the repodir at the end.  This was missing for servergitmirrors

found by @cde when working on fdroidclient#35
2017-07-07 15:50:11 +02:00
mimi89999
a427bb7b56
Also search for apk in build/outputs/apk/release 2017-07-07 12:21:09 +02:00
Hans-Christoph Steiner
b0a8d4163d Merge branch 'metadata' into 'master'
--create-metadata: make sure apk[name] is not empty

See merge request !302
2017-07-06 22:49:26 +00:00
Izzy
8ea06ce83e --create-metadata: make sure apk[name] is not empty 2017-07-07 00:18:08 +02:00
Hans-Christoph Steiner
dfb07808d3 support configing buildserver VM per-build with sudo=
This adds the 'sudo' build field, which is just a script that is run as
root.  For more info, see the issue that this closes:

refs #318
closes #317
2017-07-06 17:29:23 +02:00
Hans-Christoph Steiner
abe422fd7a handle App instance with no Builds when writing YAML 2017-07-06 16:00:44 +02:00
Hans-Christoph Steiner
3e6b7062b6 update: remove ruamel requirement, and improve '--create-metadata'
If ruamel.yaml is not available, this will fallback to using PyYAML. This
also adds some blank fields to the newly created template to make it easy
for human editors to fill in.

closes #343
2017-07-06 13:31:42 +02:00
Hans-Christoph Steiner
4a15208b84 update: update openssl KnownVuln scan to handle all recent versions
Thanks to @bubu for reporting!
2017-07-06 13:31:42 +02:00
Hans-Christoph Steiner
10a607ede3 update: find PNG when recommended icon is an XML file
APKs can now use XML files for vector graphics like the app icon. `aapt`
returns the XML file by default, and perhaps also androguard.  This
checks if the icon is an XML file, and if so, it tries to find a PNG in
the APK with the same name and density to use instead

closes #322

This should also ultimately make the XML file available as an icon source
as well fdroidclient#1091
2017-07-06 12:25:48 +02:00
Hans-Christoph Steiner
c14cbeb0a5 update: normalize var name to apkzip 2017-07-06 12:25:48 +02:00
Hans-Christoph Steiner
df99c85ca6 support manually adding per-build antiFeatures in metadata
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
2017-07-06 12:25:48 +02:00
Michael Pöhn
20afa0e861 lint: check all build fields against accepted list
This makes sure there are no typoed or wrong build fields in all metadata
files.
2017-07-06 12:25:48 +02:00
Hans-Christoph Steiner
b34d5c83b9 Merge branch '343-ruamel-yaml-version-check' into 'master'
check version of ruamel.yaml and raise according error message

See merge request !301
2017-07-06 08:10:14 +00:00
Michael Pöhn
4ca57ebfbb check version of ruamel.yaml and raise according error message 2017-07-06 09:07:54 +02:00
Hans-Christoph Steiner
ebef1ae57d Merge branch 'antifeature' into 'master'
added new AntiFeature: ApplicationDebuggable

See merge request !298
2017-07-05 20:34:35 +00:00