Commit graph

251 commits

Author SHA1 Message Date
Hans-Christoph Steiner
255932453c ensure _ is used only for gettext strings
_ is often used for variables that are ignored, this makes that a bit more
explicit
!338
2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner
278d67d960 implement gettext localization
This allows all the text to be localized via Weblate.  This is a quick
overview of all the strings, but there are certainly some that were left
out.

closes #342
2017-09-15 11:39:00 +02:00
Marcus Hoffmann
a9a19527c7 metadata: also read .fdroid.txt metadata
This is documented as working here:
https://f-droid.org/docs/Building_Applications/
(second to last paragraph)

fixes #364
2017-08-22 00:03:43 +02:00
Hans-Christoph Steiner
c67f8e349c metadata: handle empty YAML files without crashing
!311
2017-07-26 18:21:28 -07:00
Izzy
b787ec9e34 Updating syntax check for Flattr (see #284) 2017-07-20 20:26:25 +02:00
Hans-Christoph Steiner
dfb07808d3 support configing buildserver VM per-build with sudo=
This adds the 'sudo' build field, which is just a script that is run as
root.  For more info, see the issue that this closes:

refs #318
closes #317
2017-07-06 17:29:23 +02:00
Hans-Christoph Steiner
abe422fd7a handle App instance with no Builds when writing YAML 2017-07-06 16:00:44 +02:00
Hans-Christoph Steiner
df99c85ca6 support manually adding per-build antiFeatures in metadata
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
2017-07-06 12:25:48 +02:00
Hans-Christoph Steiner
b34d5c83b9 Merge branch '343-ruamel-yaml-version-check' into 'master'
check version of ruamel.yaml and raise according error message

See merge request !301
2017-07-06 08:10:14 +00:00
Michael Pöhn
4ca57ebfbb check version of ruamel.yaml and raise according error message 2017-07-06 09:07:54 +02:00
Izzy
4d9c60d352 added new AntiFeature: ApplicationDebuggable
Useful e.g. for "test repositories" to indicate an app was compiled with
"application-debuggable"
2017-07-04 21:15:04 +02:00
Michael Pöhn
1b362c2183 correct yaml-rewriting for buildozer flag 2017-07-04 13:27:45 +02:00
Michael Pöhn
0980fb4df8 import ruamel.ymal only when re-writing yaml metadata 2017-07-04 11:51:08 +02:00
Michael Pöhn
19d422da86 rewritemeta yaml: fixed boolen mapping for build flags 2017-07-04 11:51:08 +02:00
Michael Pöhn
056a28f897 fix indentation 2017-07-04 11:51:08 +02:00
Michael Pöhn
75ebb6e8cc fix building with yml metadata 2017-07-04 11:51:08 +02:00
Michael Pöhn
7ab197be2e completed ordered yaml field list 2017-07-04 11:51:08 +02:00
Michael Pöhn
f1e0089931 prettify write_yaml metadata 2017-07-04 11:51:08 +02:00
Michael Pöhn
7aa92abebc yaml rewrite version code as int 2017-07-04 11:51:08 +02:00
Michael Pöhn
f9d4007ea6 rewrite metadata builds list to yaml 2017-07-04 11:51:08 +02:00
Michael Pöhn
fbd5e8772c yaml metadata rewrite: correct blank line 2017-07-04 11:51:08 +02:00
Michael Pöhn
735b7d0f33 fixed pyflakes/pep warings 2017-07-04 11:51:08 +02:00
Michael Pöhn
0f90ab9aac rewrite to yaml works for app data now (builds still missing) 2017-07-04 11:51:08 +02:00
lb@lb520
230286786f Added a method to build python/kivy projects using buildozer. 2017-06-20 14:16:31 +02:00
Hans-Christoph Steiner
30ff777897 rewritemeta: do not include empty Summary: or Description:
Since the Summary: and Description: in the metadata file has the highest
priority of all the localized texts, adding blank versions means that
apps would always have blank Summary and Description even if the app has
those fields in the localized sections of fdroiddata and/or in the app's
source repo itself.

fdroiddata!2262
2017-06-07 22:10:25 +02:00
Torsten Grote
1fcd8e63a3
Replace sys.exit() in non-main functions by exceptions
Also move all exceptions into one module
2017-05-22 16:51:11 -03:00
Hans-Christoph Steiner
84bb41a91f metadata: switch from deprecated cgi.escape to html.escape
cgi.escape is deprecated in Python 3.x and has security issues:
https://bugs.python.org/issue26398

html.escape() differs from cgi.escape() by its defaults to quote=True:

 s = html.escape( """& < " ' >""" )   # s = '&amp; &lt; &quot; &#x27; &gt;'
2017-05-18 16:13:17 +02:00
Hans-Christoph Steiner
4f758f5c78 fix white space trimming when writing .txt format
Some of the internal data format changes somehow broke the white space
trimming when writing out .txt.

closes #285
2017-05-02 14:55:23 +02:00
Hans-Christoph Steiner
0d00705aed use pyyaml for rewriting .yml metadata files
This replaces the broken, custom code with the standard YAML lib.

In rewritemeta, do not call app.metadatapath since it will be deleted when
the dict is cleaned up for outputing. metadatapath is only used internally
and should not be written out.

closes #169
refs #290
2017-05-02 14:55:23 +02:00
Hans-Christoph Steiner
cb49f57c06 support "Author Web Site" as metadata field
Fastlane Supply, Triple-T Gradle Play Publisher, and many app stores
include the possibility to specify a website for the author, as distinct
from the website for the app.

closes #204
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner
7d4176dd50 metadata: don't crash if metadata includes blank Categories list
If someone includes just 'Categories:', it shouldn't crash.
2017-04-03 20:18:54 +02:00
Hans-Christoph Steiner
713d9195c3 ensure that app.Categories is always a list of strings
This just makes it easier for people writing build recipes.  Rewriting will
output a list of strings as well.

The test index.xml and categories.txt are updated to include the new number
categories, and the changed CurrentVersionCode to 2147483647 (MAX_VALUE)
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
3afd6ca684 only set 'No description available' in the old index format
The new index format aims to be a direct representation of the data, then
the clients/website will handle the rest.
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
4e8e297944 metadata: port .fdroid.yml handling to App dict subclass
This got overlooked in b7fc7f2228, then
caught in the ./makebuildserver tests, where it builds Checkey.
2017-02-28 12:08:04 +01:00
Hans-Christoph Steiner
8b85c860a6 rename lastupdated to lastUpdated to match fdroid client
This lets index-v1 be parsed directly into class instances because the
field/instance var names match exactly.  The original index v0 element
must retain the 'lastupdated' name for backwards compatibility.
2017-02-28 12:08:04 +01:00
Hans-Christoph Steiner
7a25dbc081 rewritemeta: fix silly bug of using == for =
This was introduced in c0bc3afda9
2017-02-24 13:36:41 +01:00
Hans-Christoph Steiner
593613043e normalize Build TYPE_STRING data based on .txt
Unfortunately, things like versionCode must be strings.  That should be
converted to be ints throughout.
2017-02-24 11:01:01 +01:00
Hans-Christoph Steiner
c178cfb843 normalize Build TYPE_LIST data based on .txt
In the future, we should have better internal datatypes for this stuff,
i.e. instead of gradle: ['yes'] for True, actually use a boolean.  For now,
make the YAML and JSON metadata produce the same internal data as .txt.
2017-02-24 11:01:01 +01:00
Hans-Christoph Steiner
e0f39a7e7b rename Build fields: version -> versionName, vercode -> versionCode
Since the YAML/JSON/etc. field names are now exactly the same as the field
names used in the internal dict in the Build class, this is a global rename

This keeps with the standard names used in Android:
https://developer.android.com/guide/topics/manifest/manifest-element.html
2017-02-24 11:01:01 +01:00
Hans-Christoph Steiner
c0bc3afda9 convert metadata.Build to a subclass of dict
Like with the App class in the commit before, this makes it a lot
easier to work with this data when converting between the internal
formats and external formats like YAML, JSON, MsgPack, protobuf, etc.

The one unfortunate thing here is Build.update. It becomes
dict.update(), which is a method not an attribute.
build.get('update') or build['update'] could be used, but that would
be oddly inconsistent. So instead the field is renamed to
'androidupdate', except for in the .txt v0 metadata files. This better
describes what field does anyway, since it runs `android update`.

Build.update is only referenced in two places right next to each other
for the ant builds, so this change still seems worthwhile.
2017-02-24 11:01:01 +01:00
Hans-Christoph Steiner
b7fc7f2228 convert App to subclass of dict to support parsing/dumping libs
Python is heavily based on its core data types, and dict is one of the more
important ones.  Even classes are basically a wrapper around a dict. This
converts metadata.App to be a subclass of dict so it can behave like a dict
when being dumped and loaded.  This makes its drastically easier to use
different data formats for build metadata and for sending data to the
client.  This approach will ultimately mean we no longer have to maintain
custom parsing and dumping code.

This also means then that the YAML/JSON field names will not have spaces in
them, and they will match exactly what it used as the dict keys once the
data is parsed, as well as matching exactly the instance attribute names:

* CurrentVersion: 1.2.6
* app['CurrentVersion'] == '1.2.6'
* app.CurrentVersion == '1.2.6'

Inspired by:
https://goodcode.io/articles/python-dict-object/
2017-02-24 11:01:01 +01:00
Hans-Christoph Steiner
4625651192 make Build class act more like a dict
This makes it a lot easier to work with Build instances with parsing and
dumping libraries, since they expect only core Python types (dict, list,
tuple, str, etc)
2017-02-24 11:01:01 +01:00
Hans-Christoph Steiner
948689281a ensure fdroiddata metadata file overrides .fdroid.yml in source
If the already parsed App instance from metadata/*.* has a field, then the
value coming from .fdroid.yml should not override it.
2017-01-11 22:42:41 +01:00
Hans-Christoph Steiner
e37be9b063 fix fdroid build with non-git repos
HEAD is really only in git.  This was introduced in
a4e4310803
2016-12-05 20:19:56 +01:00
Hans-Christoph Steiner
ce3efe4168 remove support for XML app metadata, its broken
JSON and YAML are very closely related, so supporting both of them is
basically almost no extra work.  Both are also closely related to how
Python works with dicts and pickles. XML is a very different beast, and its
not popular for this kind of thing anyway, so just purge it.
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
b91cdffe17 App.get_last_build() method to replace duplicated code
Look @mvdan, I added a method to the App class!
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
3db2c9869c do proper checking of versionCode value
versionCode is defined as a Java Integer, so any value between -2147483648
(Integer.MIN_VALUE) and 2147483647 (Integer.MIN_VALUE) is valid, including
0.

https://developer.android.com/guide/topics/manifest/manifest-element.html#vcode
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
98297278bd check all APKs for old versions of OpenSSL
This scans all APKs for old versions of OpenSSL libraries that are known to
be vulnerable to issues, or fully unsupported.

This really should be implemented as a per-APK AntiFeature, so that it can
apply to any version that is vulnerable.  Since AntiFeatures are currently
only per-App, this instead sets the AntiFeature only if the latest APK is
vulnerable.

Google also enforces this:
https://support.google.com/faqs/answer/6376725?hl=en

apk['antiFeatures'] has the first letter small, since all build fields
start with a lowercase letter.  app.AntiFeatures has the first
uppercase since all App fields are that way.
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
a4e4310803 allow metadata to be embedded in source repos via .fdroid.yml
This allows a source repo to include a complete metadata file so that it
can be built directly in place using `fdroid build`.  If that app is then
included in fdroiddata, it will first load the source repo type and URL
from fdroiddata, then read .fdroid.yml if it exists, then include the rest
of the metadata as specified in fdroiddata, so that fdroiddata has
precedence over the metadata in the source code.

This lets `fdroid build` apps without having a whole fdroiddata setup, but
instead just directly in place in the source code.  This also lets devs
optionallu maintain the fdroid metadata as part of their app, rather than
in fdroiddata without loosing any control.  This should make it easier to
spread around the maintenance load.
2016-11-16 23:28:03 +01:00
Hans-Christoph Steiner
f8dca60a20 convert comments above functions to python docstrings
This is how to write per-function comments.
https://www.python.org/dev/peps/pep-0257/
2016-11-16 23:28:03 +01:00