This lets index-v1 be parsed directly into class instances because the
field/instance var names match exactly. The original index v0 element
must retain the 'lastupdated' name for backwards compatibility.
Python is heavily based on its core data types, and dict is one of the more
important ones. Even classes are basically a wrapper around a dict. This
converts metadata.App to be a subclass of dict so it can behave like a dict
when being dumped and loaded. This makes its drastically easier to use
different data formats for build metadata and for sending data to the
client. This approach will ultimately mean we no longer have to maintain
custom parsing and dumping code.
This also means then that the YAML/JSON field names will not have spaces in
them, and they will match exactly what it used as the dict keys once the
data is parsed, as well as matching exactly the instance attribute names:
* CurrentVersion: 1.2.6
* app['CurrentVersion'] == '1.2.6'
* app.CurrentVersion == '1.2.6'
Inspired by:
https://goodcode.io/articles/python-dict-object/
This is to clean up the code for the introduction of a new index format.
This also makes it so that only repo/categories.txt is generated, and not
archive/categories.txt.
Currently, the only thing that I can find that reads categories.txt is the
wordpress plugin, e.g. wp-fdroid. And it only reads repo/categories.txt
not archive/categories.txt.
This scans all APKs for old versions of OpenSSL libraries that are known to
be vulnerable to issues, or fully unsupported.
This really should be implemented as a per-APK AntiFeature, so that it can
apply to any version that is vulnerable. Since AntiFeatures are currently
only per-App, this instead sets the AntiFeature only if the latest APK is
vulnerable.
Google also enforces this:
https://support.google.com/faqs/answer/6376725?hl=en
apk['antiFeatures'] has the first letter small, since all build fields
start with a lowercase letter. app.AntiFeatures has the first
uppercase since all App fields are that way.
This makes sure there is a GPG signature on any file that is included in
the repo, including APKs, OBB, source tarballs, media files, OTA update
ZIPs, etc. Having a GPG signature is more important on non-APK files since
they mostly do not have any signature mechanism of their own.
This also adds basic tests of adding non-APK/OBB files to a repo with
`fdroid update`.
closes#232
This makes it so that the final build product can be specified in output=
and it'll work no matter if its an APK or not. This was developed around
the case of building the OTA update.zip for the Privileged Extension. It
should work for any build process in theory but it has not yet been tested.
https://gitlab.com/fdroid/privileged-extension/issues/9
This adds the most basic level of support for including arbitrary files in
an F-Droid repository. This is useful for things like including videos,
ebooks, update.zip files for ROM updates, and more. The aim is to have
this as generic as possible to keep it flexible for unforeseen uses.
Code-wise, this is really just a first effort. This area of code has not
been touched in a very long time, and the repo parsing is done in a giant
function that is not easy to break apart. It should be broken up to more
cleanly support arbitrary files.
Also remove the TODO line, we've decided to keep the old permission
format for now, at least until there is a major overhaul of the index
data format. And the issue tracker the proper place for TODOs.
In many cases, there are times where metadata errors need to be ignored, or
at least not stop the command from running. For example, there will
inevitably be new metadata fields added, in which case a packaged version
of fdroidserver will throw errors on each one. This adds a standard -W
flag to customize the response: ignore, default, or error.
* by default, the errors are still errors
* `fdroid readmeta -W` will just print errors
* `fdroid readmeta -Wignore` will not even print errors
https://gitlab.com/fdroid/fdroidserver/issues/150
urllib.parse.urljoin() will strip off the last path segment before joining
if that last path segment does not end with /. That's a "feature" to make
it easy to replace file names. Here it was stripping off the essential
'fdroid' segment, making URLs like:
https://foo.com/repo
when they should be
https://foo.com/fdroid/repo
It is now possible for the server operator to specify lists of apps that
must be installed or deleted on the client (aka "push installs). If
the user has opted in, or the device is already setup to respond to
these requests, then fdroidclient will automatically install/delete
the packageNames listed. This is protected by the same signing key
as the app index metadata.
It generates single XML elements with the data set in the attributes. This
keeps the XML compact and easily extensible, e.g. for adding versionCode,
signingKey, etc as attributes:
<install packageName="com.fsck.k9"/>
<install packageName="at.bitfire.davdroid"/>
<delete packageName="com.facebook.orca"/>
Copyright: 2016 Blue Jay Wireless
Signed-off-by: Hans-Christoph Steiner <hans@eds.org>
closes#177
Parse maxSdkVersion and -23 permission tags
Parse maxSdkVersion and uses-permission-sdk-23 tags and insert them into the index.
AndroidManifest:
```xml
<uses-permission android:name="android.permission.READ_CALENDAR" />
<uses-permission
android:name="android.permission.WRITE_EXTERNAL_STORAGE"
android:maxSdkVersion="18" />
<uses-permission-sdk-23 android:name="android.permission.ACCESS_FINE_LOCATION" />
```
new index format:
```xml
<package>
<version>1.0</version>
<versioncode>1</versioncode>
<apkname>maxmin.apk</apkname>
<hash type="sha256">526eb6d643050c3fae42fb6c001c704006046db52e98998b21f7646ecae3dae5
</hash>
<sig>893fc1f22301c902d05c938b29d21648</sig>
<size>8681</size>
<sdkver>14</sdkver>
<targetSdkVersion>24</targetSdkVersion>
<added>2016-08-01</added>
<permissions>READ_EXTERNAL_STORAGE,READ_CALENDAR,WRITE_EXTERNAL_STORAGE</permissions>
<uses-permission name="android.permission.WRITE_EXTERNAL_STORAGE"
maxSdkVersion="18" />
<uses-permission name="android.permission.READ_CALENDAR" />
<uses-permission name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission-sdk-23 name="android.permission.ACCESS_FINE_LOCATION" />
</package>
```
old format:
```xml
<permissions>READ_CALENDAR,ACCESS_FINE_LOCATION,WRITE_EXTERNAL_STORAGE</permissions>
```
This generates both formats for backward compatibility with old clients.
Also increase version of index format.
This is a proposal for the issues discussed in https://gitlab.com/fdroid/fdroidclient/issues/704
@eighthave @mvdan
See merge request !150
Parse maxSdkVersion and uses-permission-sdk-23
tags and insert them into the index, e.g.,
<package>
<permissions>READ_EXTERNAL_STORAGE,READ_CALENDAR,WRITE_EXTERNAL_STORAGE</permissions>
<uses-permission name="android.permission.WRITE_EXTERNAL_STORAGE"
maxSdkVersion="18" />
<uses-permission name="android.permission.READ_CALENDAR" />
<uses-permission name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission-sdk-23 name="android.permission.ACCESS_FINE_LOCATION" />
</package>
Also increase version of index format
Pass a date from APK to KnownApks.recordapk()
... if --use-date-from-apks option is used.
Essentially, it just expands influence of `--use-date-from-apks` option to `stats/known_apks.txt`.
See merge request !141
OBB files provide a mechanism to deliver large data blobs that do not need
to be part of the APK. For example, a game's assets do not need to change
often, so they can be shipped as an OBB, then APK updates do not need to
include all those assets for each update.
https://developer.android.com/google/play/expansion-files.html
The ZIP format has no official encoding :-| so we have to do hacks. The
zipfile devs couldn't even sort this out:
https://bugs.python.org/issue10614closes#167
This stops these errors:
fdroid/fdroidserver/fdroidserver/update.py:744: ResourceWarning: unclosed
file <_io.BufferedReader
name='repo/icons-320/info.guardianproject.urzip.100.png'>
fdroid/fdroidserver/fdroidserver/update.py:721: DeprecationWarning: The
'warn' function is deprecated, use 'warning' instead
When the metadata changes, different things will be stored about each APK.
So invalidate the cached info parsed from APKs if the cache's metadata
version does not match the metadata version of the currently running tools.
This makes UTF-8 the sole supported encoding for F-Droid's files. This is
mostly codifying the already existing practice for config.py and index.xml.
The other files where always just ASCII before.
* config.py
* metadata/*.txt
* known_apks.txt
* categories.txt
* latestapps.txt
* latestapps.dat
* index.xml
Note: this does not change the read/write encoding of stats files. That is
still ASCII.
