These entries are hardcoded as a single line in all the app stores, so
newlines should be stripped to get the data simple to use. This is in
contrast with the on-disk format for Fastlane and Triple-T, which includes
a newline in the title.txt and short_description.txt files. I think all
files in those systems are normalized to end in a newline.
6d0b1bbe6fae0909683f2c6a154515bc4bfcb674 didn't handle the
allow_disabled_algorithm case at all, so we add it back.
This additionally fixes a (previously existing) bug where setting
allow_disabled_algorithms to True didn't move apks back from archive to
repo. Introduce a new test for this.
The disabled_algorithm archiving logic is still all over the place so
ideally that needs a future refactor.
We want to keep the stable version (CV) of an app in /repo with highest
priority and only move it to /archive when ArchivePolicy is set to 0.
For this to work we need two changes here:
1) when sorting apks by version code we insert the apk corresponding to
the CVC of the app afterwards with highest priority. So when walking the
list of apks afterwards the CVC apk is always kept first.
2) Instead of the two pass algorithm of moving things back and forth to
the archive we instead figure out where each apk of an app goes in first
before actually moving them into the right place.
Fixes: fdroid/fdroidserver#385
Python PIL is not so tolerant, so bad EXIF causes crashes:
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2088, in main
insert_localized_app_metadata(apps)
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 978, in insert_localized_app_metadata
_strip_and_copy_image(os.path.join(root, f), destdir)
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 754, in _strip_and_copy_image
in_image = Image.open(fp)
File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2687, in open
% (filename if filename else fp))
OSError: cannot identify image file <_io.BufferedReader name='build/org.sw24softwares.starkeverben/fastlane/metadata/android/en-US/images/featureGraphic.png'>
Using a filename based on the hash of the contents means that the caching
algorithms for fdroidclient and browsers can safely cache the file forever
using the filename, since this guarantees that the contents will never
change for a given filename.
This does not cover screenshots, only icon.png, featureGraphic.png,
tvBanner.png, and promoGraphic.png.
fdroidserver#689
fdroid-website!453
This was done with much help from @uniqx. This is the first level of
supporting APK Signatures v1, v2, and v3. This is enough to include
APKs with any combo of v1/v2/v3 signatures. For this to work at all,
apksigner and androguard 3.3.3+ must be installed.
closes#399
androguard's v2 signature methods return DER-encoded certificates. The DER-
encoded certificates are parsed from JAR Signature's raw "Signature Block
File".
Also, update the documentation comment to be absolutely clear.
The function used to copy images from any locale directory starting with at least two lowercase letters correctly, but only add those to the app metadata that started with two letters plus one big letter or a "-". The same problem did not exist for adding texts to the metadata.
build-tools 28.0.3 added a new field in the end 'compileSdkVersionCodename=', which also accidentally ends with the string 'name='.
The purpose of this regex was to catch the 'packageName' field, which is in ht eaapt ouput the exact ' name=', therefore added whe non-caracter \W prefix match.
sample aapt output (28.0.3):
package: name='com.a.b.app' versionCode='1' versionName='1.0' compileSdkVersion='28' compileSdkVersionCodename='9'
previously, regex was catching second occurence, so '9'
See merge request !582
Fastlane/Triple-T app store metadata files must be in UTF-8. Before this
would crash if they were not. This changes the handling to just replace
the non-UTF8 chars with a �. Here's the stacktrace:
CRITICAL: Unknown exception found!
Traceback (most recent call last):
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 164, in <module>
main()
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 159, in main
raise e
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroid", line 138, in main
mod.main()
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2010, in main
insert_localized_app_metadata(apps)
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 885, in insert_localized_app_metadata
os.path.join(root, f))
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 700, in _set_localized_text_entry
text = fp.read()[:limit]
File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 94: invalid start byte
699b3e4c69 got it wrong for targetSdkVersion.
Also, one confusing thing is that aapt outputs "sdkVersion: '3'" for
com.politedroid_3.apk but no "sdkVersion:" for no.min.target.sdk_987.apk.
F-Droid never really supported running on android-1 or android-2, so it
seems pointless to debug support for them.
Android Application IDs must be valid Java Package Names. While the build
tools likely validate the Application ID, it is possible to manually create
a malicious APK.
pickle can serialize executable code, while JSON is only ever pure data.
The APK cache is only ever pure data, so no need for the security risks of
pickle. For example, if some malicious thing gets write access on the
`fdroid update` machine, it can write out a custom tmp/apkcache which would
then be executed. That is not possible with JSON.
This does just ignore any existing cache and rebuilds from scratch. That is
so we don't need to maintain pickle anywhere, and to ensure there are no
glitches from a conversion from pickle to JSON.
closes#163
Sometimes androguard returns the XMLNS as entirely empty, which would make
it an invalid APK since normally the 'android' name is mapped to the
'http://schemas.amazon.com/apk/res/android' value. Occasionally, a
different key is used.
closesfdroid/fdroidserver#515
