Commit graph

898 commits

Author SHA1 Message Date
Hans-Christoph Steiner
ed80391bb5 common.get_examples_dir() to get path to fdroidserver examples 2017-11-22 11:40:16 +01:00
relan
cd8a86a2a2 build: raise VCS client version logging level
Debug logging is not saved in buildserver logs. Also move this line into
build.py's main() to avoid duplication: common.setup_vcs(app) may be
called from parse_metadata() too.
2017-11-02 08:52:02 +03:00
Hans-Christoph Steiner
ffc91e301a make _ always be the gettext function, nothing else
This avoids hard bugs where the _() function gets overidden by a str or
something else.
2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner
dd6d4b2012 preserve metadata when copying APK signatures
Since this code was not setting the "create system" and "compress type",
Python uses it's defaults.  Those will be different than what the Android
tools produces if this is run on UNIX.  The Android tools uses the bare
bones "Windows" ZIP format, e.g. no permissions, etc.

For example:
https://verification.f-droid.org/eu.siacs.conversations_234.apk.diffoscope.html
2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner
e0df6d2479 choose the most recent available version of Java
This came about testing on OSX, where there are often multiple versions of
the JDK installed.  This was choosing the oldest version.  It should
choose the most recent version.
2017-10-25 23:01:25 +02:00
Marcus Hoffmann
e307cdec59 common: check file existence before opening manifest
This fixes a problem when there is a broken symlink with the name
AndroidManifest.xml which will then lead to a crash when trying to open
it for trying to remove the debuggable flag.
2017-10-24 15:19:38 +02:00
Hans-Christoph Steiner
f227af817f standardize on "build-tools" as the name of that Android SDK component 2017-10-19 23:28:13 +02:00
Michael Pöhn
788c8f97fd better error message in publish when repo_key is not set 2017-10-17 22:10:39 +02:00
relan
0ab0b03648 common: print VCS client version
This may help to debug issues with too old VCS clients.
2017-10-06 09:41:24 +03:00
Marcus Hoffmann
50ba108093 fix bare except to satisfy newer pycodestyle 2017-10-04 19:23:53 +02:00
Michael Pöhn
75c5fa6355 added some docstrings and error messages 2017-09-26 14:11:09 +02:00
Michael Pöhn
6fc968f7cd choose best apk version vor index v0 2017-09-26 14:11:09 +02:00
Michael Pöhn
bca07f794f added functions for storing/loading signer fingerprints to stats 2017-09-26 14:11:09 +02:00
Michael Pöhn
ba3844e97e function for finding developer signature in metadata 2017-09-26 14:11:09 +02:00
Michael Pöhn
7c4b8bcac4 added parse release filename function 2017-09-26 14:11:09 +02:00
Michael Pöhn
08627d70a7 check file existance before trying to verify apks; use centext manager for zips when verifying apk 2017-09-26 14:11:09 +02:00
Michael Pöhn
efb0bf6ee5 add common functions for dealing with apk signatures 2017-09-26 14:11:09 +02:00
Michael Pöhn
c196f7dd7a common function for fetching sha256 signing-key fingerprint 2017-09-26 14:11:09 +02:00
Hans-Christoph Steiner
beb79d613d move lots of comments to function doc strings 2017-09-25 16:35:18 +02:00
Hans-Christoph Steiner
176f539647 allow spaces in filenames
This fixes all the bugs I could find that prevented fdroid from
handling files with spaces in them.  This is more important now that
fdroid supports random media files, and Repomaker
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
5dcb48831f index: always use jarsigner for verifying JAR signatures
apksigner v0.7 (build-tools 26.0.1), Google made it require that the
AndroidManifest.xml was present in the archive before it verifies the
signature.  So this needs to stick with the jarsigner hack for JARs.
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
1d4e3a254d create blank config.py using cross-platform technique
os.mknod() fails on OSX with:

Traceback (most recent call last):

  File "/Users/travis/build/fdroidtravis/fdroidserver/tests/../fdroid", line 154, in <module>
    main()
  File "/Users/travis/build/fdroidtravis/fdroidserver/tests/../fdroid", line 130, in main
    mod.main()
  File "/Users/travis/build/fdroidtravis/fdroidserver/fdroidserver/update.py", line 1768, in main
    common.write_to_config(config, 'repo_keyalias', config['repo_keyalias'])
  File "/Users/travis/build/fdroidtravis/fdroidserver/fdroidserver/common.py", line 2328, in write_to_config
    os.mknod(cfg)
PermissionError: [Errno 1] Operation not permitted
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
6c7b987239 fix string formats that are ambiguous for translators 2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner
255932453c ensure _ is used only for gettext strings
_ is often used for variables that are ignored, this makes that a bit more
explicit
!338
2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner
9712fac011 "No config.py found" should warning level, until people get use to it
Since !335 changed the long standing behavior, now allowing `fdroid` to
work without a config.py being present, this message should be quite
visible in case people are expecting `fdroid` to fail when there is no
config.py present.
2017-09-16 11:11:49 +02:00
Hans-Christoph Steiner
acac15f74d Merge branch 'newcomers' into 'master'
allow starting without config.py

See merge request fdroid/fdroidserver!335
2017-09-16 08:07:56 +00:00
Hans-Christoph Steiner
278d67d960 implement gettext localization
This allows all the text to be localized via Weblate.  This is a quick
overview of all the strings, but there are certainly some that were left
out.

closes #342
2017-09-15 11:39:00 +02:00
Hans-Christoph Steiner
cb10f0df09 standardize os.walk() var names based on Python 3.5 docs
There were multiple conventions used in the code, but mostly it was already
using the convention from the docs, so this converts things to using that
convention:

https://docs.python.org/3/library/os.html#os.walk
2017-09-15 11:37:21 +02:00
Marcus Hoffmann
190a2fde81 common: allow starting without a config file
There is no reason this should be disallowed as the default config often
works well when building simple apps.
2017-09-14 02:15:57 +02:00
Marcus Hoffmann
feadc6a565 make sure config exists before writing to it 2017-09-14 02:15:57 +02:00
Hans-Christoph Steiner
3aee9b3f11 Merge branch 'close-subprocess-file-handles' into 'master'
make sure file-streams of subprocesses get closed

See merge request !328
2017-09-07 09:49:51 +00:00
Michael Pöhn
a718c75b05 make sure file-streams of subprocesses get closed 2017-09-07 02:36:58 +02:00
Michael Pöhn
3e6dfacf6c added signatures subcommand 2017-09-07 00:03:35 +02:00
relan
6f05cf85f9 makebuildserver: upgrade NDK r15b to r15c 2017-08-11 10:19:40 +03:00
Hans-Christoph Steiner
dfb07808d3 support configing buildserver VM per-build with sudo=
This adds the 'sudo' build field, which is just a script that is run as
root.  For more info, see the issue that this closes:

refs #318
closes #317
2017-07-06 17:29:23 +02:00
Hans-Christoph Steiner
6d7fd5f07c Merge branch 'ndk' into 'master'
NDK r14b and r15b

See merge request !293
2017-07-04 14:34:42 +02:00
Michael Pöhn
cfd3e84250 use stored metadatapath instead of guessing it 2017-07-04 11:51:08 +02:00
Michael Pöhn
75ebb6e8cc fix building with yml metadata 2017-07-04 11:51:08 +02:00
relan
86b403b599 makebuildserver: add NDK r15b 2017-07-04 11:44:25 +03:00
relan
ecdb2c685a makebuildserver: fix NDK r14b provisioning
Replace r14 with r14b after a57bff7.
2017-07-04 10:46:52 +03:00
Hans-Christoph Steiner
746d4bd4cf update: allow_disabled_algorithms option to keep MD5 sigs in repo
The new policy is to move APKs with invalid signatures to the archive,
and only add those APKs to the archive's index if they have valid MD5
signatures.

closes #323
closes #292
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
b7260ea854 update: allow deprecated signatures only in the archive
In April 2017, Oracle's jarsigner and Google's apksigner both switched to
considering any APK signature that uses MD5 as unsigned.  Any old build
is likely to have a MD5 signature.  This sets up the archive as the only
place where these "disabled algorithms" are allowed in the repo, and
marks any APK signed by a "disabled algorithm" as having a "known
vulnerability"

This also now automatically moves APKs with invalid signatures to the
archive section.

#323
2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
6105f8a184 use var naming scheme in KnownApks (apk --> apkName)
Everywhere else, the file name of the APK is called apkName.
2017-06-01 16:01:05 +02:00
Hans-Christoph Steiner
9471bf2731 regexs for getting packageName and versionCode from filenames
This is useful for parsing APK files, which can include packageName,
versionCode, and optionally 7 char signing key ID (i.e. <sig>).
This also can set the packageName and versionCoe for non APK files, so
that it is easy to assign them to metadata files, and to allow for
upgrades by setting the versionCode in the filename.
2017-06-01 16:01:05 +02:00
Hans-Christoph Steiner
17cd07f1a7 fix pylint unused-argument 2017-05-23 22:34:16 +02:00
Torsten Grote
1fcd8e63a3
Replace sys.exit() in non-main functions by exceptions
Also move all exceptions into one module
2017-05-22 16:51:11 -03:00
Hans-Christoph Steiner
0345d2c79f update: find aapt when it is not in the PATH
This is some very messy logic built up since 2010.  This will all go away
once we have a python3 version of androguard available.

The removed imports and `dir(APK)` is to silence pyflakes

closes #303
2017-05-18 17:21:05 +02:00
thez3ro
06598ae406
use androguard if aapt isn't found 2017-05-04 23:35:17 +02:00
Hans-Christoph Steiner
cdef5bcd92 downcase all 'localized' key names to match the rest of index-v1
This is a little omission.  keys that are used in metadata/*.yml all start
with an UpperCase letter, but in fdroidserver, index-v1.json, and
fdroidclient, it is all camelCase with lowercase first letter. The keys
from the 'localized' section are currently never in metadata/*.yml, so
these keys never get downcase.  This change will break fdroidclient
versions that do not also have this change, but since we're in alpha, that
should be fine.

If support for a 'localized' section is added to metadata/*.yml, then the
keys there should probably be UpperCase CamelCase to match the other keys.
2017-04-27 21:12:49 +02:00
Michael Pöhn
62af9b8e3d check meld config for manual apk diffs 2017-04-22 10:48:50 +02:00