From 99c39602ef29466441ef9bd6a1e0955de239f108 Mon Sep 17 00:00:00 2001 From: Ciaran Gultnieks Date: Tue, 10 Jan 2012 13:20:27 +0000 Subject: [PATCH 1/8] Metadata: new ShareMyPosition, and aCal current version update --- metadata/com.morphoss.acal.txt | 4 ++-- metadata/net.sylvek.sharemyposition.txt | 9 +++++---- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/metadata/com.morphoss.acal.txt b/metadata/com.morphoss.acal.txt index 81b42fa9..0ded171e 100644 --- a/metadata/com.morphoss.acal.txt +++ b/metadata/com.morphoss.acal.txt @@ -27,5 +27,5 @@ Build Version:1.32,34,r1.32,target=android-8 Build Version:1.33,35,r1.33,target=android-8 Build Version:1.34,36,r1.34,target=android-8 -Market Version:1.33 -Market Version Code:35 +Market Version:1.34 +Market Version Code:36 diff --git a/metadata/net.sylvek.sharemyposition.txt b/metadata/net.sylvek.sharemyposition.txt index 01a010b9..fcb8cc34 100644 --- a/metadata/net.sylvek.sharemyposition.txt +++ b/metadata/net.sylvek.sharemyposition.txt @@ -14,13 +14,14 @@ The recipient is sent a link to a web service. (The source for the web service is in the same repository as the client). . -Repo Type:svn -Repo:http://sharemyposition.googlecode.com/svn/trunk/ +Repo Type:git-svn #Note - deliberately pulling from trunk, not trunk/ShareMyPosition-android, #even though that would be more efficient, in order to preserve the #corresponding source for the web service in our source tarball. +Repo:http://sharemyposition.googlecode.com/svn/trunk/ Build Version:1.0.11,16,64,subdir=ShareMyPosition-android +Build Version:1.1.0-beta3,20,70,subdir=ShareMyPosition-android,target=android-11,prebuild=mv lib libs -Market Version:1.1.0-beta1 -Market Version Code:18 +Market Version:1.1.0-beta3 +Market Version Code:20 From 1458cfb04f078aaa338f965a921a767c046fe050 Mon Sep 17 00:00:00 2001 From: Nicolas Raoul Date: Tue, 10 Jan 2012 21:59:09 +0900 Subject: [PATCH 2/8] Ignore logs directory --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index ccbb9b7d..c8ac9607 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ config.py repo/ +logs/ built/ build/ build_*/ From 596b602655a1e1306aabf79ce3c1ade225641437 Mon Sep 17 00:00:00 2001 From: Ciaran Gultnieks Date: Tue, 10 Jan 2012 15:24:02 +0000 Subject: [PATCH 3/8] Metadata: some formatting fixes --- metadata/An.stop.txt | 3 ++- metadata/com.funambol.androidsync.txt | 2 +- metadata/com.jadn.cc.txt | 3 ++- metadata/goo.TeaTimer.txt | 5 +++-- metadata/ru.gelin.android.weather.notification.txt | 7 ++++--- 5 files changed, 12 insertions(+), 8 deletions(-) diff --git a/metadata/An.stop.txt b/metadata/An.stop.txt index b2b7c43b..5d17306f 100644 --- a/metadata/An.stop.txt +++ b/metadata/An.stop.txt @@ -5,7 +5,8 @@ Web Site:http://code.google.com/p/anstop/ Source Code:http://code.google.com/p/anstop/source/checkout Issue Tracker:http://code.google.com/p/anstop/issues/list Summary:A simple stopwatch -Description:A simple stopwatch, that also supports lap timing and a countdown +Description: +A simple stopwatch, that also supports lap timing and a countdown timer. . diff --git a/metadata/com.funambol.androidsync.txt b/metadata/com.funambol.androidsync.txt index 7a31a9e8..c66e7893 100644 --- a/metadata/com.funambol.androidsync.txt +++ b/metadata/com.funambol.androidsync.txt @@ -5,7 +5,7 @@ Web Site:http://www.funambol.com Source Code:https://android-client.forge.funambol.org/source/browse/android-client/ Issue Tracker: Summary:Funambol sync client -Description:Sync Client +Description: Funambol sync client. . Repo Type:svn diff --git a/metadata/com.jadn.cc.txt b/metadata/com.jadn.cc.txt index fad0aa7e..d78ff2b0 100644 --- a/metadata/com.jadn.cc.txt +++ b/metadata/com.jadn.cc.txt @@ -5,7 +5,8 @@ Source Code:http://github.com/bherrmann7/Car-Cast/ Issue Tracker:https://github.com/bherrmann7/Car-Cast/issues Donate:https://market.android.com/details?id=com.jadn.ccpro Summary:A podcast downloader and player. -Description:Car Cast is a simple audio podcast downloader and player. +Description: +Car Cast is a simple audio podcast downloader and player. Optimized for use in a daily commute, it features big buttons, large text, remembers last played location. 1. Subscribe to podcasts diff --git a/metadata/goo.TeaTimer.txt b/metadata/goo.TeaTimer.txt index 5f704e2e..1dd09158 100644 --- a/metadata/goo.TeaTimer.txt +++ b/metadata/goo.TeaTimer.txt @@ -4,8 +4,9 @@ Category:Office Web Site:http://solidsushi.com/lab/teatimer Source Code:http://github.com/ralphleon/TeaTimer Issue Tracker:https://github.com/ralphleon/TeaTimer/issues -Summary:A Simple Tea Timer for Android -Description:A straight-forward tea timer +Summary:Simple Tea Timer +Description: +A straight-forward tea timer . Repo Type:git diff --git a/metadata/ru.gelin.android.weather.notification.txt b/metadata/ru.gelin.android.weather.notification.txt index b7689b37..326f9c63 100644 --- a/metadata/ru.gelin.android.weather.notification.txt +++ b/metadata/ru.gelin.android.weather.notification.txt @@ -3,9 +3,10 @@ Category:Internet Web Site:https://code.google.com/p/weather-notification-android/ Source Code:https://code.google.com/p/weather-notification-android/source/browse/ Issue Tracker:https://code.google.com/p/weather-notification-android/issues/list -Summary: Displays the air temperature and other weather conditions in the Android notification bar. -Description:Simple application which displays the air temperature -and other weather conditions in the Android notification bar. +Summary:Weather info in notification bar +Description: +Simple application which displays the air temperature +and other weather conditions in the notification bar. The air temperature is always visible like a system clock. . From 9453071e882bc55ff1d58ac4cd8136ebcb3d8b5b Mon Sep 17 00:00:00 2001 From: Ciaran Gultnieks Date: Tue, 10 Jan 2012 15:24:21 +0000 Subject: [PATCH 4/8] Metadata: updated Wordpress version info --- metadata/org.wordpress.android.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/metadata/org.wordpress.android.txt b/metadata/org.wordpress.android.txt index 47471fe8..0e181087 100644 --- a/metadata/org.wordpress.android.txt +++ b/metadata/org.wordpress.android.txt @@ -21,5 +21,5 @@ Repo:http://android.svn.wordpress.org/trunk/ Build Version:1.3.9,31,202,prebuild=mkdir libs && mv *.jar libs && sed -i "s@checkStats(accounts.size());@// MY PRIVACY > YOUR STATS@" src/org/wordpress/android/wpAndroid.java,encoding=utf-8 Build Version:1.4.1,33,228,prebuild=mkdir libs && mv *.jar libs && sed -i "s@checkStats(accounts.size());@// MY PRIVACY > YOUR STATS@" src/org/wordpress/android/wpAndroid.java,encoding=utf-8 -Market Version:2.0.2 -Market Version Code:40 +Market Version:2.0.3 +Market Version Code:42 From 7645cc1af6b939257a46b0a4f9a2f9179f6635d7 Mon Sep 17 00:00:00 2001 From: Dominik Date: Tue, 10 Jan 2012 18:55:18 +0100 Subject: [PATCH 5/8] AdAway version 1.19 --- metadata/org.adaway.txt | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/metadata/org.adaway.txt b/metadata/org.adaway.txt index 16037a47..bf5763e2 100644 --- a/metadata/org.adaway.txt +++ b/metadata/org.adaway.txt @@ -18,6 +18,7 @@ Requires Root:yes Build Version:1.12,13,ea5378a94ee0dc1d99d2cec95fae7e6d81afb2b9,subdir=org_adaway/,buildjni=yes Build Version:1.15,16,4128e59da2eac5c2904c7c7568d298ca51e79540,subdir=org_adaway/,buildjni=yes,patch=defprop.patch Build Version:1.18,19,0b9985398b9eef7baf6aadd0dbb12002bc199d2e,subdir=org_adaway/,buildjni=yes,patch=defprop.patch +Build Version:1.19,20,ab27f4dab5f3ea5e228cfb4a6b0e1fbf53695f22,subdir=org_adaway/,buildjni=yes -Market Version:1.18 -Market Version Code:19 +Market Version:1.19 +Market Version Code:20 From c4614c7bf2d1f2a89c3c7cf34b167563644e0147 Mon Sep 17 00:00:00 2001 From: Ciaran Gultnieks Date: Tue, 10 Jan 2012 18:07:59 +0000 Subject: [PATCH 6/8] Metadata: syntax correction --- metadata/org.adaway.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metadata/org.adaway.txt b/metadata/org.adaway.txt index bf5763e2..27327b56 100644 --- a/metadata/org.adaway.txt +++ b/metadata/org.adaway.txt @@ -13,7 +13,7 @@ Blocks advertisements via the /etc/hosts file. Supports automatic updates. Repo Type:git Repo:git://gitorious.org/adaway/adaway.git -Requires Root:yes +Requires Root:Yes Build Version:1.12,13,ea5378a94ee0dc1d99d2cec95fae7e6d81afb2b9,subdir=org_adaway/,buildjni=yes Build Version:1.15,16,4128e59da2eac5c2904c7c7568d298ca51e79540,subdir=org_adaway/,buildjni=yes,patch=defprop.patch From c9c824f57ddf643e399b9328b290b0b972489b56 Mon Sep 17 00:00:00 2001 From: Ciaran Gultnieks Date: Tue, 10 Jan 2012 18:11:20 +0000 Subject: [PATCH 7/8] Metadata: adaway needs default properties patch --- metadata/org.adaway.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metadata/org.adaway.txt b/metadata/org.adaway.txt index 27327b56..67392449 100644 --- a/metadata/org.adaway.txt +++ b/metadata/org.adaway.txt @@ -18,7 +18,7 @@ Requires Root:Yes Build Version:1.12,13,ea5378a94ee0dc1d99d2cec95fae7e6d81afb2b9,subdir=org_adaway/,buildjni=yes Build Version:1.15,16,4128e59da2eac5c2904c7c7568d298ca51e79540,subdir=org_adaway/,buildjni=yes,patch=defprop.patch Build Version:1.18,19,0b9985398b9eef7baf6aadd0dbb12002bc199d2e,subdir=org_adaway/,buildjni=yes,patch=defprop.patch -Build Version:1.19,20,ab27f4dab5f3ea5e228cfb4a6b0e1fbf53695f22,subdir=org_adaway/,buildjni=yes +Build Version:1.19,20,ab27f4dab5f3ea5e228cfb4a6b0e1fbf53695f22,subdir=org_adaway/,buildjni=yes,patch=defprop.patch Market Version:1.19 Market Version Code:20 From c469f0feed29fa0237ef24247f7c7d0454d93c78 Mon Sep 17 00:00:00 2001 From: Ciaran Gultnieks Date: Tue, 10 Jan 2012 18:57:07 +0000 Subject: [PATCH 8/8] Improved metadata handling Two main points: Firstly, there is no longer a random mapping between app['key'] and a corresponding field in the metadata file - the key and field are now the same. Secondly, more information (including comments) is retrieved from the metadata, to facilitate being able to re-write it which is necessary for various support utilities. --- build.py | 8 +- checkmarket2.py | 2 +- common.py | 200 +++++++++++++++++++++++++++++------------------- scanner.py | 4 +- update.py | 60 +++++++-------- 5 files changed, 157 insertions(+), 117 deletions(-) diff --git a/build.py b/build.py index fadb7a54..b69020b5 100755 --- a/build.py +++ b/build.py @@ -74,13 +74,13 @@ if not os.path.isdir(build_dir): for app in apps: - if app['disabled']: + if app['Disabled']: print "Skipping %s: disabled" % app['id'] elif not app['builds']: print "Skipping %s: no builds specified" % app['id'] - if (app['disabled'] is None and app['repo'] != '' - and app['repotype'] != '' and (options.package is None or + if (app['Disabled'] is None and app['Repo'] != '' + and app['Repo Type'] != '' and (options.package is None or options.package == app['id']) and len(app['builds']) > 0): print "Processing " + app['id'] @@ -88,7 +88,7 @@ for app in apps: build_dir = 'build/' + app['id'] # Set up vcs interface and make sure we have the latest code... - vcs = common.getvcs(app['repotype'], app['repo'], build_dir) + vcs = common.getvcs(app['Repo Type'], app['Repo'], build_dir) refreshed_source = False diff --git a/checkmarket2.py b/checkmarket2.py index 193a63fa..f1a7d805 100755 --- a/checkmarket2.py +++ b/checkmarket2.py @@ -63,7 +63,7 @@ for app in apps: print "...couldn't find version code" elif not version: print "...couldn't find version" - elif vercode == app['marketvercode'] and version == app['marketversion']: + elif vercode == app['Market Version Code'] and version == app['Market Version']: print "...up to date" else: print '...updating to version:' + version + ' vercode:' + vercode diff --git a/common.py b/common.py index d54ed432..d1bd6170 100644 --- a/common.py +++ b/common.py @@ -1,7 +1,7 @@ # -*- coding: utf-8 -*- # # common.py - part of the FDroid server tools -# Copyright (C) 2010-11, Ciaran Gultnieks, ciaran@ciarang.com +# Copyright (C) 2010-12, Ciaran Gultnieks, ciaran@ciarang.com # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by @@ -245,15 +245,53 @@ class vcs_bzr(vcs): raise VCSException("Bzr update failed") +# Get the type expected for a given metadata field. +def metafieldtype(name): + if name == 'Description': + return 'multiline' + if name == 'Requires Root': + return 'flag' + if name == 'Build Version': + return 'build' + if name == 'Use Built': + return 'obsolete' + return 'string' + +# Parse metadata for a single application. +# +# 'metafile' - the filename to read. The package id for the application comes +# from this filename. +# +# Returns a dictionary containing all the details of the application. There are +# two major kinds of information in the dictionary. Keys beginning with capital +# letters correspond directory to identically named keys in the metadata file. +# Keys beginning with lower case letters are generated in one way or another, +# and are not found verbatim in the metadata. +# +# Known keys not originating from the metadata are: +# +# 'id' - the application's package ID +# 'builds' - a list of dictionaries containing build information +# for each defined build +# 'comments' - a list of comments from the metadata file. Each is +# a tuple of the form (field, comment) where field is +# the name of the field it preceded in the metadata +# file +# 'name' - the application's name, as displayed. This will +# always be None on return from this parser. The name +# is discovered from built APK files. +# def parse_metadata(metafile, **kw): - def parse_buildline(value): + def parse_buildline(lines): + value = "".join(lines) parts = [p.replace("\\,", ",") for p in re.split(r"(? 0: + raise MetaDataException("Unexpected text on same line as " + field + " in " + metafile.name) + elif fieldtype == 'string': + thisinfo[field] = value + elif fieldtype == 'flag': + if value == 'Yes': + thisinfo[field] = True + elif value == 'No': + thisinfo[field] = False + else: + raise MetaDataException("Expected Yes or No for " + field + " in " + metafile.name) + elif fieldtype == 'build': if value.endswith("\\"): mode = 2 - buildline = [value[:-1]] + buildlines = [value[:-1]] else: - thisinfo['builds'].append(parse_buildline(value)) - elif field == "Requires Root": - if value == "Yes": - thisinfo['requiresroot'] = True + thisinfo['builds'].append(parse_buildline([value])) + elif fieldtype == 'obsolete': + pass # Just throw it away! else: - raise MetaDataException("Unrecognised field " + field + " in " + metafile.name) - elif mode == 1: # multi-line description + raise MetaDataException("Unrecognised field type for " + field + " in " + metafile.name) + elif mode == 1: # Multiline field if line == '.': mode = 0 else: if len(line) == 0: - thisinfo['description'] += '\n\n' + thisinfo[field] += '\n\n' else: - if (not thisinfo['description'].endswith('\n') and - len(thisinfo['description']) > 0): - thisinfo['description'] += ' ' - thisinfo['description'] += line - elif mode == 2: # line continuation + if (not thisinfo[field].endswith('\n') and + len(thisinfo[field]) > 0): + thisinfo[field] += ' ' + thisinfo[field] += line + elif mode == 2: # Line continuation mode in Build Version if line.endswith("\\"): - buildline.append(line[:-1]) + buildlines.append(line[:-1]) else: - buildline.append(line) + buildlines.append(line) thisinfo['builds'].append( - parse_buildline("".join(buildline))) + parse_buildline(buildlines)) mode = 0 + if mode == 1: - raise MetaDataException("Description not terminated in " + metafile.name) - if len(thisinfo['description']) == 0: - thisinfo['description'] = 'No description available' + raise MetaDataException(field + " not terminated in " + metafile.name) + elif mode == 2: + raise MetaDataException("Unterminated continuation in " + metafile.name) + + if len(thisinfo['Description']) == 0: + thisinfo['Description'] = 'No description available' + + # Ensure all AntiFeatures are recognised... + if thisinfo['AntiFeatures']: + parts = thisinfo['AntiFeatures'].split(",") + for part in parts: + if (part != "Ads" and + part != "Tracking" and + part != "NonFreeNet" and + part != "NonFreeDep" and + part != "NonFreeAdd"): + raise MetaDataException("Unrecognised antifeature '" + part + "' in " \ + + metafile.name) + return thisinfo + def read_metadata(verbose=False): apps = [] for metafile in sorted(glob.glob(os.path.join('metadata', '*.txt'))): diff --git a/scanner.py b/scanner.py index 3291a2dc..f4d63e35 100755 --- a/scanner.py +++ b/scanner.py @@ -55,7 +55,7 @@ for app in apps: skip = False if options.package and app['id'] != options.package: skip = True - elif app['disabled']: + elif app['Disabled']: print "Skipping %s: disabled" % app['id'] skip = True elif not app['builds']: @@ -71,7 +71,7 @@ for app in apps: build_dir = 'build/' + app['id'] # Set up vcs interface and make sure we have the latest code... - vcs = common.getvcs(app['repotype'], app['repo'], build_dir) + vcs = common.getvcs(app['Repo Type'], app['Repo'], build_dir) refreshed_source = False diff --git a/update.py b/update.py index a943bb7e..b9740992 100755 --- a/update.py +++ b/update.py @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- # # update.py - part of the FDroid server tools -# Copyright (C) 2010-11, Ciaran Gultnieks, ciaran@ciarang.com +# Copyright (C) 2010-12, Ciaran Gultnieks, ciaran@ciarang.com # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU Affero General Public License as published by @@ -196,7 +196,7 @@ for app in apps: if app['name'] is None: app['name'] = app['id'] app['icon'] = '' - if app['disabled'] is None: + if app['Disabled'] is None: print "WARNING: Application " + app['id'] + " has no packages" else: if app['name'] is None: @@ -283,14 +283,14 @@ apps_nopkg = 0 for app in apps: - if app['disabled'] is None: + if app['Disabled'] is None: # Get a list of the apks for this app... gotmarketver = False apklist = [] for apk in apks: if apk['id'] == app['id']: - if str(apk['versioncode']) == app['marketvercode']: + if str(apk['versioncode']) == app['Market Version Code']: gotmarketver = True apklist.append(apk) @@ -304,22 +304,22 @@ for app in apps: addElement('id', app['id'], doc, apel) addElement('name', app['name'], doc, apel) - addElement('summary', app['summary'], doc, apel) + addElement('summary', app['Summary'], doc, apel) addElement('icon', app['icon'], doc, apel) - addElement('description', app['description'], doc, apel) - addElement('license', app['license'], doc, apel) - if 'category' in app: - addElement('category', app['category'], doc, apel) - addElement('web', app['web'], doc, apel) - addElement('source', app['source'], doc, apel) - addElement('tracker', app['tracker'], doc, apel) - if app['donate'] != None: - addElement('donate', app['donate'], doc, apel) - addElement('marketversion', app['marketversion'], doc, apel) - addElement('marketvercode', app['marketvercode'], doc, apel) - if not (app['antifeatures'] is None): - addElement('antifeatures', app['antifeatures'], doc, apel) - if app['requiresroot']: + addElement('description', app['Description'], doc, apel) + addElement('license', app['License'], doc, apel) + if 'Category' in app: + addElement('category', app['Category'], doc, apel) + addElement('web', app['Web Site'], doc, apel) + addElement('source', app['Source Code'], doc, apel) + addElement('tracker', app['Issue Tracker'], doc, apel) + if app['Donate'] != None: + addElement('donate', app['Donate'], doc, apel) + addElement('marketversion', app['Market Version'], doc, apel) + addElement('marketvercode', app['Market Version Code'], doc, apel) + if not (app['AntiFeatures'] is None): + addElement('antifeatures', app['AntiFeatures'], doc, apel) + if app['Requires Root']: addElement('requirements', 'root', doc, apel) # Sort the apk list into version order, just so the web site @@ -370,38 +370,38 @@ for app in apps: if options.buildreport: if len(app['builds']) == 0: print ("WARNING: No builds defined for " + app['id'] + - " Source: " + app['source']) + " Source: " + app['Source Code']) warnings += 1 else: - if app['marketvercode'] != '0': + if app['Market Version Code'] != '0': gotbuild = False for build in app['builds']: - if build['vercode'] == app['marketvercode']: + if build['vercode'] == app['Market Version Code']: gotbuild = True if not gotbuild: print ("WARNING: No build data for market version of " - + app['id'] + " (" + app['marketversion'] - + ") " + app['source']) + + app['id'] + " (" + app['Market Version'] + + ") " + app['Source Code']) warnings += 1 # If we don't have the market version, check if there is a build # with a commit ID starting with '!' - this means we can't build it # for some reason, and don't want hassling about it... - if not gotmarketver and app['marketvercode'] != '0': + if not gotmarketver and app['Market Version Code'] != '0': for build in app['builds']: - if build['vercode'] == app['marketvercode']: + if build['vercode'] == app['Market Version Code']: gotmarketver = True # Output a message of harassment if we don't have the market version: - if not gotmarketver and app['marketvercode'] != '0': - addr = app['source'] - print "WARNING: Don't have market version (" + app['marketversion'] + ") of " + app['name'] + if not gotmarketver and app['Market Version Code'] != '0': + addr = app['Source Code'] + print "WARNING: Don't have market version (" + app['Market Version'] + ") of " + app['name'] print " (" + app['id'] + ") " + addr warnings += 1 if options.verbose: # A bit of extra debug info, basically for diagnosing # app developer mistakes: - print " Market vercode:" + app['marketvercode'] + print " Market vercode:" + app['Market Version Code'] print " Got:" for apk in apks: if apk['id'] == app['id']: