Commit graph

251 commits

Author SHA1 Message Date
Daniel Martí
1987ee7f54 Remove gradle=main
It was only supported in txt metadata, and was unused in fdroiddata
anyway.
2015-12-03 10:56:59 +01:00
Daniel Martí
5e0bc7f2c3 Properly close all metadata files once read 2015-12-03 10:51:27 +01:00
Daniel Martí
96f1fc5355 Never use exit/log in metadata 2015-12-02 22:59:58 +01:00
Daniel Martí
b9ac6fb69a Speed up metadata reading
Total time for `fdroid readmeta` went down from ~1.6s to ~1.4s on my
machine.
2015-12-02 22:38:40 +01:00
Daniel Martí
528e45d22b Simplify post_metadata_parse usage 2015-12-02 22:29:31 +01:00
Daniel Martí
c975531d87 metadata: properly store nums as strs and bools as bools 2015-12-02 20:49:42 +01:00
Daniel Martí
71a5eac494 Pre-compile more regexes 2015-12-02 20:40:30 +01:00
Daniel Martí
a566ee4475 Fix default update= on ant builds 2015-12-02 20:16:25 +01:00
Daniel Martí
bf8518ee8f Rework build into a class
This simplifies usage, goes from

    build['flag']
to
    build.flag

Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.

As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.

While at it, unify "build", "thisbuild", "info", "thisinfo", etc into
just "build".
2015-12-02 20:03:11 +01:00
Daniel Martí
6e87a8c45a Add documentation to new App class 2015-11-28 17:19:59 +01:00
Daniel Martí
ab614ab442 Rework app into a class
This simplifies usage, goes from

	app['Foo']
to
	app.Foo

Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.

As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.
2015-11-28 17:11:05 +01:00
Daniel Martí
de12cfdbe1 Handle duplicate apps in a much cleaner way
Don't log and exit in an inner metadata function. Handle it at a higher
level and do a proper exception. This also avoids unnecessary passing of
apps all around.
2015-11-28 14:04:21 +01:00
Daniel Martí
48645072ee extension simplifications 2015-10-04 18:01:32 +02:00
Daniel Martí
53ed335d4b Final touches to yaml writing
Now it writes and reads properly.
2015-10-04 07:56:32 -07:00
Daniel Martí
ac37e3084e More improvements to yaml writing 2015-10-04 07:55:55 -07:00
Daniel Martí
9c5b5f7b60 Don't leave an empty line at the end 2015-10-04 07:55:42 -07:00
Daniel Martí
1aa891c4fd Add basic yaml metadata writing 2015-10-04 07:55:42 -07:00
Daniel Martí
082e275aa1 Rename write_metadata to specify txt 2015-10-03 23:13:44 -07:00
Daniel Martí
885a3a73f1 Simplify format parsing logic 2015-10-03 23:00:18 -07:00
Daniel Martí
911994fc99 Make write_metadata take a writer
This will let rewritemeta report format issues without writing to disk.
2015-09-24 22:38:33 -07:00
Daniel Martí
e285a8ac31 Remove Dogecoin donation support
Not nearly as popular as bitcoin and litecoin. To the point where it hadn't
been used yet.

Bump index version to 14 to mark its removal.
2015-09-18 19:06:10 -07:00
Daniel Martí
d07bf0754a metadata: fix html paragraph tag regression
Spotted in fdroid/fdroidclient#422
2015-09-17 14:14:54 -07:00
Daniel Martí
fb23b2cff4 metadata: don't break words nor on hyphens 2015-09-11 22:52:50 -07:00
Daniel Martí
223db1588b metadata: allow links spanning multiple lines
This allows for nicer text wrapping.
2015-09-11 22:07:20 -07:00
Daniel Martí
3413965eaa Wrap descriptions to 80 chars 2015-09-11 21:36:18 -07:00
Daniel Martí
ade5036e23 metadata: rewrite multiline field writing
Now description text wrapping can be added.
2015-09-11 19:12:51 -07:00
Daniel Martí
cd202e59b1 Ensure that builds are always sorted when writing
This might not be true if builds are appended by e.g. checkupdates --auto.
This ensures that we sort them again before writing.
2015-09-11 18:21:07 -07:00
Hans-Christoph Steiner
c80c1bf017 import: split out URL handling into its own function
This is preparation to add other import methods, like checking if the
command was run in a currently checked out git repo.
2015-09-10 11:08:39 +02:00
Daniel Martí
de1c599127 Fix regression introduced in 2831b3e93
An empty list of antifeatures should not be written.
2015-09-05 10:10:37 -04:00
Hans-Christoph Steiner
325db90918 set "accepted formats" for metadata in config.py
For a bit repo like f-droid.org, it makes sense to standardize on a single
format for metadata files.  This adds support for enforcing a single data
format, or a reduced set of data formats.  So f-droid.org would run like
this if it changed to YAML:

  accepted_formats = ['txt', 'yaml']

Then once everything was converted to YAML, it could look like this:

  accepted_formats = ['yaml']
2015-09-01 11:53:37 +02:00
Hans-Christoph Steiner
8d6e0aebb9 exit with error if duplicate metadata file is found
In order to prevent confusion caused by multiple metadata files for a given
app, fdroid will exit with an error if it finds any app metadata file with
the same package ID as one that has already been parsed.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
6f334558df metadata: rename metafile to metadatapath when it is a path
To keep the code clear, change the code to use separate variables for the
path to the metadata file, and for the file object of the opened metadata.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
285c06d156 metadata: use unified method for getting default app metadata dict
This puts process of setting up the defaults for the internal dict
that represents a parsed app into a single method that is reused for all
metadata formats.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
32e257d742 support app metadata in YAML format
YAML is a format that is quite similar to the .txt format, but is a
widespread standard that has editing modes in popular editors.  It is also
easily parsable in python.

The .pickle for testing is a lightly edited version of the real metadata
for org.videolan.vlc:

 * comments were removed
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
0b62e7f22a set appid in get_default_app_info_list() 2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
0425788b7d consolidate all post-parse metadata checks
This puts all of the needed post parsing checks on the metadata into a
single method that is used by all parsing methods (.txt, JSON, XML, YAML).
This provides the single place to normalize the internal representation of
the metadata.

It would be good to also change the internal representation to use more Python
bool/int types so that less post parsing is needed for JSON, XML, and YAML.

The SMSSecure test .pickle was changed to account for the use of lstrip()
and rstrip() on all 'script' types.

This also changes the example JSON to use ints for versionCodes
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
014df8426f eliminate Boolean metadata type, only 'bool' is needed
No need for a specific metadata type for 'Requires Root'.  Just use 'bool',
then convert on output.  This keeps the internal representation consistent
since all boolean-ish values will be Python bools.  This in turn makes
parsing metadata in various formats easier.  This also makes setting
booleans more tolerant since all these values are valid:

 Yes yes No no True true False false
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
ab145de6bc support app metadata in XML format
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code.  XML is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.

The .pickle for testing is a lightly edited version of the real metadata
for net.osmand.plus:

* comments were removed
* "NonFreeNet" was added as an AntiFeature
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
2831b3e93f convert internal representation of AntiFeatures to list
The AntiFeatures metadata is a comma-separated list of tags, like
Categories, so it should also be stored internally as a list.  This makes
parsing XML and JSON easier.

The test cases' .pickle files look like they change a lot, but they really
don't, its only the change of default AntiFeatures value from None to []
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
cc29d39f0a metadata: store comments in lists since lists are used everywhere else
The only data produced by parsing metadata that was stored as a tuple was
the comments' key/values.  Everywhere else, lists are used.  This changes
the comments to also only use lists to keep the data consistent internally.
That makes it vastly easier to produce the exact same metadata dict when
parsing .txt, JSON, XML, etc.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
61f6be1a7b in JSON metadata, convert unicode to str to match the internal format
fdroidserver currently uses plain python 2.x strings rather than unicode
strings, so the parsed JSON should produce the same format as the parsed
.txt metadata.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
eeb8ba02b0 move metadata post-parse processing to reusable function 2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
84c9777e9e support app metadata in JSON format
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code.  JSON is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.
2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
2b72634120 use common method to set up defaults in metadata dict
This is needed for the upcoming metadata formats: JSON, XML, etc.
2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
141e0e24c7 metadata: rename parse_metadata() to parse_txt_metadata()
This makes the code clear as support for other metadata formats is added.
2015-09-01 11:39:50 +02:00
Daniel Martí
e0870f6e76 metadata: error on url titles that aren't 2015-08-28 15:48:53 -07:00
Daniel Martí
ecd0e07e93 Revert "metadata: complain if link url isn't a url"
I already tried this once. It had to be reverted because Maintainer Notes
often don't obey this. I had forgotten.

This reverts commit 97ba94903e.
2015-08-28 14:10:05 -07:00
Daniel Martí
97ba94903e metadata: complain if link url isn't a url 2015-08-28 11:08:59 -07:00
Daniel Martí
e99ae53538 metadata: fix harmless typo 2015-08-27 20:41:18 -07:00
Daniel Martí
06c94b3a44 Add support for gradle properties via gradleprops 2015-08-24 15:54:05 -07:00