_ is a valid character for Java package names, so it should also work in
the repo file naming scheme. This makes it so it only splits the file
name based on the last _.
This is a little omission. keys that are used in metadata/*.yml all start
with an UpperCase letter, but in fdroidserver, index-v1.json, and
fdroidclient, it is all camelCase with lowercase first letter. The keys
from the 'localized' section are currently never in metadata/*.yml, so
these keys never get downcase. This change will break fdroidclient
versions that do not also have this change, but since we're in alpha, that
should be fine.
If support for a 'localized' section is added to metadata/*.yml, then the
keys there should probably be UpperCase CamelCase to match the other keys.
glob != regexp, [A-Z-.@] means one of those chars is required in that
position, so the glob pattern was requiring a least 3 chars. Locales are
usually just two lower case letters, e.g. vi, de, ar.
This uses the "What's New" entry for the CurrentVersionCode and includes it
as the current WhatsNew metadata for the App class.
Things like fastlane supply and Google Play support a "What's New" entry
per-APK, but fdroidclient does not current use anything but the current
version of this data. Right now, it seems we probably only want to have
the latest WhatsNew in the index to save space.
In theory, we could make the WhatsNew data structure follow the structure
of fastlane/Play, but that would quite a bit of complexity for something
that might never be used.
fdroidclient#910
Since `fdroid server update` is the place where all uploads to servers
happens, it makes sense to also handle the git push for the binary
transparency log here instead of `fdroid btlog`
This was failing on environments that did not have any LANG or LC_* locale
variables set. This is a valid setup, and is common in headless setups, so
it needs to be handled.
This also adds a new pass of the test suite without the locale env vars set
so that this situation is also tests on gitlab-ci, not only gpjenkins.
The error this caused was:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-18: ordinal not in range(128)
In 6c2cf2ccdd, the names of some essential
data bits changed. If those names are not in the tmp/apkcache, then
`fdroid update` shouldn't crash but instead just ignore that cache entry.
tmp/apkcache should be deleted since the metadata version was bumped, but
I guess that does not always happen.
With the binary transparency log stored in git, it makes that more readable
and as small as possible if the index files are fully sorted. That will
reduce the differences between two copies of an index file to a bare
minimum.
If in the future we implement some kind of binary diff transfer, this will
also help there.
This is a bit different than index.jar: instead of their being index.xml
and index_unsigned.jar, the presense of index-v1.json means that there is
unsigned data. That file is then stuck into a jar and signed by the
signing process. index-v1.json is never published to the repo. It is
included in the binary transparency log, if that is enabled.
If there are graphics files that are in repo/ but there is no metadata for
the associated app, then do not include the graphics in the index. Issue a
warning about this.
This looks for standard graphics assets in two standardized locations, one
in metadata/ subdirs and another in fastlane's standard dirs in the app's
source repo. When it finds them, it copies them into the repo, where they
will then be included in the index for use by fdroidclient. The images in
the metadata/ folder of fdroiddata take precendence over the files in the
app's source repo.
So like this for including graphics in fdroiddata:
metadata/packageName/locale/filename.(png|jpg|jpeg)
for example:
metadata/org.videolan.vlc/en-US/featureGraphic.png
or
metadata/info.guardianproject.ripple/zh-CN/phoneScreenshots/screenshot1.png
Including graphics in fdroiddata would be optional. The prefered way to get
graphics into the repo would be for the files to be in the git repo in a
standard location. This fastlane layout is currently supported:
https://github.com/fastlane/fastlane/blob/1.109.0/supply/README.md#images-and-screenshots
This syncs up the field names between the fdroiddata .yml files, the keys
used in the implementation in fdroidserver, the index data format, and the
final data structures in fdroidclient. This makes it easier for devs to
follow, and makes the Jackson parsing library automatically handle
converting the data from the index file to Java instances.
This bumps the metadata version since the apkcache will have to be
discarded.
Here are the name changes:
* apkname --> apkName
* id --> packageName
* sha256 --> hash
* version --> versionName
* versioncode --> versionCode
tests/repo/index.xml was changed only to bump the metadata version
from 17 to 18.
Using datetime instances as the internal format makes it much easier to
convert between the formats needed for index.xml and index-v1. apkcache
still uses time tuples and known_apks.txt still uses the ISO date.
Python encode/decode libs work directly with dicts, so the internal dict
can just be passed directly to any of these libs (pyyaml, pyjson, msgpack,
simplejson, etc). This still generates the exact same index.xml as before.
This converts the internal format for the repo timestamp to a datetime
instance, which can be easily converted to UNIX time in seconds for XML
and UNIX time in milliseconds for the new index formats. UNIX time in
milliseconds is directly serialized into a java.util.Date instance by
Jackson.
A Binary Transparency Log is a append only log of all binaries published by
a repo. This is useful for people to find whether the binary they have
matches what F-Droid has published, and also makes it more difficult for
the published history to be changed without notice, or for a server to give
specific users custom malware binaries.
https://www.eff.org/deeplinks/2014/02/open-letter-to-tech-companies
Since it is now possible to build and include arbitrary files, like OTA
update ZIP files, the update procedure needs to look for non-APK files that
match the packageName_versionCode pattern of fdroid-generated files.
!193
admin#14
privileged-extension#9
`fdroid build` will generate source tarballs, and now with support for
adding any file to a repo, we need to explicitly ignore the fdroid-
generated source tarballs. If people want to include source tarballs in a
repo, they still can, as long as that source tarball doesn't use the
`fdroid build` tarball naming scheme.
This forces <uses-permission maxSdkVersion=""> to be an integer in the
internal dict, and forces it to have no decimal point in XML. Having it as
an integer in the internal dict means data will pass directly through to
the fdroidclient Apk instance, where it is ultimately an integer.
fdroidclient should handle no minSdkVersion fine, since it just parses the
text in <sdkver></sdkver> as an int, and uses a default value if there is
an Exception.
