For a bit repo like f-droid.org, it makes sense to standardize on a single
format for metadata files. This adds support for enforcing a single data
format, or a reduced set of data formats. So f-droid.org would run like
this if it changed to YAML:
accepted_formats = ['txt', 'yaml']
Then once everything was converted to YAML, it could look like this:
accepted_formats = ['yaml']
In order to prevent confusion caused by multiple metadata files for a given
app, fdroid will exit with an error if it finds any app metadata file with
the same package ID as one that has already been parsed.
This puts process of setting up the defaults for the internal dict
that represents a parsed app into a single method that is reused for all
metadata formats.
YAML is a format that is quite similar to the .txt format, but is a
widespread standard that has editing modes in popular editors. It is also
easily parsable in python.
The .pickle for testing is a lightly edited version of the real metadata
for org.videolan.vlc:
* comments were removed
This puts all of the needed post parsing checks on the metadata into a
single method that is used by all parsing methods (.txt, JSON, XML, YAML).
This provides the single place to normalize the internal representation of
the metadata.
It would be good to also change the internal representation to use more Python
bool/int types so that less post parsing is needed for JSON, XML, and YAML.
The SMSSecure test .pickle was changed to account for the use of lstrip()
and rstrip() on all 'script' types.
This also changes the example JSON to use ints for versionCodes
No need for a specific metadata type for 'Requires Root'. Just use 'bool',
then convert on output. This keeps the internal representation consistent
since all boolean-ish values will be Python bools. This in turn makes
parsing metadata in various formats easier. This also makes setting
booleans more tolerant since all these values are valid:
Yes yes No no True true False false
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code. XML is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.
The .pickle for testing is a lightly edited version of the real metadata
for net.osmand.plus:
* comments were removed
* "NonFreeNet" was added as an AntiFeature
The AntiFeatures metadata is a comma-separated list of tags, like
Categories, so it should also be stored internally as a list. This makes
parsing XML and JSON easier.
The test cases' .pickle files look like they change a lot, but they really
don't, its only the change of default AntiFeatures value from None to []
The only data produced by parsing metadata that was stored as a tuple was
the comments' key/values. Everywhere else, lists are used. This changes
the comments to also only use lists to keep the data consistent internally.
That makes it vastly easier to produce the exact same metadata dict when
parsing .txt, JSON, XML, etc.
fdroidserver currently uses plain python 2.x strings rather than unicode
strings, so the parsed JSON should produce the same format as the parsed
.txt metadata.
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code. JSON is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.
config option to enable per-app repos for nightly builds
For Guardian Project, we've been running an fdroid repo for the nightly builds for each of our apps: https://dev.guardianproject.info/debug This is built using a big, hacked up shell script: [update-debug-fdroid-repo](https://github.com/guardianproject/fdroid-repo-tools/blob/master/update-debug-fdroid-repo). It has proven very useful to us to be able to subscribe to the nightly build for a single app, so this the first step of porting that horrid shell script to `fdroidserver`.
This also helps make the fdroidserver tool suite the single set of tools for all types of builds and releases. That will hopefully drive more free software developers to make f-droid.org the core channel for official releases.
See merge request !66
This keeps memory usage low because it only has to read a chunk at a time
into memory while before it read the whole file into memory before
uploading it.
This also seems to handle setting the permissions ACL better.
For devs that want to build and distribute nightly builds of their apps
using the fdroid tools. The core idea here is to make the fdroidserver
tool suite the single set of tools for all types of builds and releases.
That will hopefully drive more free software developers to make f-droid.org
an core channel for official releases.
