Commit graph

433 commits

Author SHA1 Message Date
Hans-Christoph Steiner
2659312a7c update: support working with old versions of PIL/Pillow
Image.close() was added in Pillow 2.4 or so.
2017-12-20 23:46:37 +01:00
Michael Pöhn
c17aeb5231 fix handling unreadable images in update.extract_apk_icons 2017-12-16 22:06:20 +01:00
Hans-Christoph Steiner
42522c23c9 update: do not crash if AndroidManifest.xml in APK has invalid date
This crash actually blocked a Janus exploit APK from being added to the
repo, but crashing isn't really the appropriate way to do that.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
8f45796ecb update: close unclosed Image instance 2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
387eebc4d6 update: strip all metadata from PNGs
This strips metadata and optimizes the compression of all PNGs copied
from the app's source repo as well as all the icons extracted from the
APKs.  There have been exploits delivered via image metadata, and
F-Droid isn't using it all, so its best to just remove it.

This unfortunately uncompresses and recompresses the files.  Luckily,
that's a lossless procedure with PNGs, and we might end up with
smaller files.  The only tool I could find that strips without
changing the image data is exiftool, but that is written in Perl.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
67b9514c5a update: strip EXIF data from all JPEGs
EXIF data can be abused to exploit systems a lot easier than the JPEG image
data can.  The F-Droid ecosystem does not use the EXIF data, so keep things
safe and strip it all away.  There is a chance that some images might rely
on the rotation to be set by EXIF, but I think having a safe system is more
important.

If needed, only the rotation data could be saved.  But that then makes it
hard to tell which images have been stripped.  This way, if there is no
EXIF, it has been stripped.  And if there is EXIF data, then it is suspect.

https://securityaffairs.co/wordpress/51043/mobile-2/android-cve-2016-3862-flaw.html
https://threatpost.com/google-shuts-down-potentially-massive-android-bug/120393/
https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-exif-headers.html

The big downside of this is that it decompresses and recompresses the
image data.  That should be replaced by a technique from jhead,
exiftool, ObscuraCam, etc. that only strips the metadata.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
bde0558d82 update: reject APKs with invalid file sig, probably Janus exploits
This just checks the first four bytes of the APK file, aka the "file
signature", to make sure it is the ZIP signature and not the DEX signature.
This was checked against the test APK, and I ran it against some known
malware and all of f-droid.org to make sure it works.

All valid ZIP files (therefore APK files) should start with the ZIP
Local File Header of four bytes.

https://www.guardsquare.com/en/blog/new-android-vulnerability-allows-attackers-modify-apps-without-affecting-their-signatures
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
5ce950e748 update: print warnings for all KnownVulns found
Some baby steps towards making the KnownVuln stuff more visible.
2017-12-14 16:57:22 +01:00
Andrea Scarpino
19a83b6219 Add Liberapay support 2017-12-12 11:53:31 +01:00
Hans-Christoph Steiner
5d54446efc update: do not replace Name/Summary from template unless blank
`fdroid nightly` needs this change so it can set the Summary using the
template.
2017-12-07 22:39:32 +01:00
tobiasKaminsky
d5ecb34df7
add flavour to metadata 2017-12-01 15:25:00 +01:00
Hans-Christoph Steiner
d46d9574b4 update: use KnownApks dates to check system clock on offline machines
KnownApks provides a reliable source of a relatively recent date.
2017-11-29 21:06:02 +01:00
Hans-Christoph Steiner
9ade09349b update: improve help strings 2017-11-08 14:05:17 +01:00
Hans-Christoph Steiner
7be46d53c7 sort Anti-Features before writing them to the wiki
This avoids silly updates to the wiki where the sort order of the Anti-
Features is the only change:

−{{AntiFeature|KnownVuln}}
 {{AntiFeature|DisabledAlgorithm}}
+{{AntiFeature|KnownVuln}}

https://f-droid.org/wiki/index.php?title=info.guardianproject.browser&curid=517&diff=189655&oldid=188037
2017-10-27 09:15:22 +02:00
Ciaran Gultnieks
505d2c1325 update bombs out when no icons (closes #402) 2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner
ac69d5b17d update: make all strings translatable 2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner
ffc91e301a make _ always be the gettext function, nothing else
This avoids hard bugs where the _() function gets overidden by a str or
something else.
2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner
927104a4e3 update: make strings translatable 2017-10-19 23:16:58 +02:00
Hans-Christoph Steiner
460c4185bb update: support Fastlane's official location for screenshots
fastlane/android/metadata/locale/images/*Screenshots

https://github.com/fastlane/fastlane/blob/2.61.0/screengrab/lib/screengrab/runner.rb#L324

closes #357
2017-10-13 14:58:15 +02:00
Hans-Christoph Steiner
a4169484fd update: fix detection of Triple-T screenshots
* https://github.com/westnordost/StreetComplete/issues/489
* https://forum.f-droid.org/t/screenshots-dont-show-up-in-client-using-gradle-play-publisher/881/2

#357
2017-10-13 14:58:15 +02:00
Michael Pöhn
290b9050e0 convert apk timestamp from filesystem to utc before storing 2017-09-26 14:13:08 +02:00
Michael Pöhn
6fc968f7cd choose best apk version vor index v0 2017-09-26 14:11:09 +02:00
Michael Pöhn
6930edf889 sort index-v1; publish now creates and stores a list of signature fingerprints 2017-09-26 14:11:09 +02:00
Michael Pöhn
6a4a78429f add signer to index v1 2017-09-26 14:11:09 +02:00
Hans-Christoph Steiner
176f539647 allow spaces in filenames
This fixes all the bugs I could find that prevented fdroid from
handling files with spaces in them.  This is more important now that
fdroid supports random media files, and Repomaker
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
6c7b987239 fix string formats that are ambiguous for translators 2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner
53e4ec47a7 English source string corrections
From @monolifed <monolifed@gmail.com>
2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner
278d67d960 implement gettext localization
This allows all the text to be localized via Weblate.  This is a quick
overview of all the strings, but there are certainly some that were left
out.

closes #342
2017-09-15 11:39:00 +02:00
Hans-Christoph Steiner
7bd171480f update: test metadata creation using internal and external templates
There is a hardcoded template in update.py, and there is also the
possibility for the user to create a template.yml.  This tests both of them
and cleans up the related code a bit.

#352 !310
2017-07-26 15:34:13 -07:00
Izzy
cbd420e490 preserve order and formatting on creating Metadata from Yaml template (closes #352) 2017-07-26 09:22:01 +02:00
Izzy
54a646bd93 have fallback for XML icons also consider res/mipmap*
several apps (e.g.
[FastHub](http://apt.qumran.org/fdroid/index/apk/com.fastaccess.github)
and [Monety](http://apt.qumran.org/fdroid/index/apk/open.currency)) have
their ic_launcher.png files not in res/drawable*dpi/, but in
res/mipmap*/ -- so the regex has been adjusted by this patch.

Additionally: if the only icon for a given resolution was an XML without
existing fallback-PNG, it should be considered "non existent" (ie.
"empty_density").
2017-07-17 00:54:33 +02:00
Izzy
dd69490821 --create-metadata: only set default empty values if not using template.py 2017-07-08 14:41:19 +02:00
Izzy
4d375a585d enable user to have presets for metadata by using a template.yml (see #345) 2017-07-08 14:21:49 +02:00
Izzy
732de91aa7 make --create-metadata use yaml.dump instead of ruamel (solves #345) 2017-07-08 14:07:11 +02:00
Izzy
8ea06ce83e --create-metadata: make sure apk[name] is not empty 2017-07-07 00:18:08 +02:00
Hans-Christoph Steiner
3e6b7062b6 update: remove ruamel requirement, and improve '--create-metadata'
If ruamel.yaml is not available, this will fallback to using PyYAML. This
also adds some blank fields to the newly created template to make it easy
for human editors to fill in.

closes #343
2017-07-06 13:31:42 +02:00
Hans-Christoph Steiner
4a15208b84 update: update openssl KnownVuln scan to handle all recent versions
Thanks to @bubu for reporting!
2017-07-06 13:31:42 +02:00
Hans-Christoph Steiner
10a607ede3 update: find PNG when recommended icon is an XML file
APKs can now use XML files for vector graphics like the app icon. `aapt`
returns the XML file by default, and perhaps also androguard.  This
checks if the icon is an XML file, and if so, it tries to find a PNG in
the APK with the same name and density to use instead

closes #322

This should also ultimately make the XML file available as an icon source
as well fdroidclient#1091
2017-07-06 12:25:48 +02:00
Hans-Christoph Steiner
c14cbeb0a5 update: normalize var name to apkzip 2017-07-06 12:25:48 +02:00
Hans-Christoph Steiner
df99c85ca6 support manually adding per-build antiFeatures in metadata
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
2017-07-06 12:25:48 +02:00
Izzy
2c8bacde42 create_metadata: do not skip APK files having no name 2017-07-05 20:31:21 +00:00
Hans-Christoph Steiner
3cb936ae92 fdroid update --create-metadata now outputs YAML format
As part of the push towards using YAML as the main metadata format, this
makes the blank template be a .yml file.
2017-07-04 11:51:08 +02:00
Torsten Grote
0bbe0c5db0
Break up the scan_apk() method and rename it to process_apk() 2017-07-03 18:37:01 -03:00
Hans-Christoph Steiner
56ee5de2bd update: invalidate cache if allow_disabled_algorithms changes
Since the cache contains implicitly the result of the jarsigner verify,
if the allow_disabled_algorithms config changes, then the apkcache is
invalid.
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
e75cabfe77 update: write cache file if anything has changed it
This fixes a bug introduced in 04db6870 where cachechanged for scan_apks()
was set only by the last call to scan_apk().
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
39fd664778 update: create 'archive/' if needed when moving APKs
Normally, just 'repo/' is created by default, e.g. `fdroid init`. If APKs
are dumped into 'repo/', then have invalid signatures, then they'll be
automatically moved to 'archive/', which therefore needs to exist.
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
746d4bd4cf update: allow_disabled_algorithms option to keep MD5 sigs in repo
The new policy is to move APKs with invalid signatures to the archive,
and only add those APKs to the archive's index if they have valid MD5
signatures.

closes #323
closes #292
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
b7260ea854 update: allow deprecated signatures only in the archive
In April 2017, Oracle's jarsigner and Google's apksigner both switched to
considering any APK signature that uses MD5 as unsigned.  Any old build
is likely to have a MD5 signature.  This sets up the archive as the only
place where these "disabled algorithms" are allowed in the repo, and
marks any APK signed by a "disabled algorithm" as having a "known
vulnerability"

This also now automatically moves APKs with invalid signatures to the
archive section.

#323
2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
0047f19d56 update: move duplicated code into move_apk_between_sections() 2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
e1492148fa fix "Archive Policy:" field, APKs can move in/out of archive
The original logic was checking keepversions against the len() of ALL the
APKs in the repo/archive.  The correct thing is to check against the
number of APKs available for the given packageName/appid.

closes #166
2017-06-28 23:53:42 +02:00