Commit graph

1053 commits

Author SHA1 Message Date
Daniel Martí
e3f60e2b78 common: make jdk detection more strict
We weren't using ^ and $, so stuff like java-8-openjdk-i386 would match
both the Arch and the Debian regexes. A list with one regex per line in
the same format is also easier to read and maintain.

This might be why java8 isn't being detected properly on our Debian
stable buildserver.
2016-02-23 12:43:58 +00:00
Dmitriy Bogdanov
7fc55a3847 Fix pubkey extraction on update
Fix pubkey extraction in case of non-empty _JAVA_OPTIONS
2016-02-17 22:22:57 +04:00
Daniel Martí
9800ed1a1a output= is now a glob path and can do gradle 2016-02-15 12:02:45 +00:00
Hans-Christoph Steiner
5780c14df2 fix PEP9 E713 test for membership should be 'not in' 2016-02-12 08:33:22 +01:00
Hans-Christoph Steiner
751fd3fb0a common: do not crash if the java_paths are not what is expected
Many commands work without the JDK installed, and it is also possible that
someone is using only JDK 8 or 9.
2016-02-12 08:33:21 +01:00
Daniel Martí
7feeaad095 common: add regexp for arch oracle paths 2016-02-11 21:48:51 +00:00
Daniel Martí
28db5b82c2 common: fix JDK detection on latest arch 2016-02-11 21:34:44 +00:00
Hans-Christoph Steiner
69c81c3817 use jarsigner and keytool from same JDK as is being set in JAVA7_HOME
Using the same JDK throughout should prevent weird bugs where a setup might
use Java8's jarsigner and Java7's keytool.  This also allows the user to
set java_paths and have jarsigner and keytool used from that specified JDK.

This incorporates almost all of the patch that is in the Debian package
that forces fdroidserver to use the default JDK on that Debian release.

closes #93 https://gitlab.com/fdroid/fdroidserver/issues/93
2016-02-11 21:17:23 +01:00
Hans-Christoph Steiner
fdf17e809c automatically detect various installed JDKs and set JAVA[6-9]_HOME
This checks for which JDKs are installed in common locations, then sets the
JAVA[6-9]_HOME env vars needed by some build environments.
2016-02-11 21:17:23 +01:00
Ciaran Gultnieks
f6b9e46246 Handle with invalid manifests better
In other words, don't blow up and kill a whole checkupdates run just
because one version of one package has some invalid xml.
2016-01-26 22:05:31 +00:00
Daniel Martí
4482ba971d gradle: stop trying to patch outputFile
This often breaks since we're just removing a single line. Instead, use
a much more reliable way of finding output apks.
2015-12-31 09:47:14 +01:00
NeroBurner
1683df0cf8 handle Queue 2015-12-29 20:11:47 +01:00
NeroBurner
0e0767492e Octal numbers 0600 to 0o600 2015-12-29 20:10:04 +01:00
NeroBurner
5ca182a20d except named exception handling 2015-12-29 20:09:27 +01:00
NeroBurner
af38f151a2 Change print-function to have brackets 2015-12-29 20:06:46 +01:00
Daniel Martí
5b8d593275 gradle: Completely get rid of gradlepluginver
We can support multiple output dirs without having to detect/infer the
plugin version. Much simpler and more robust too.
2015-12-24 16:41:39 +01:00
Daniel Martí
71a5eac494 Pre-compile more regexes 2015-12-02 20:40:30 +01:00
Daniel Martí
a566ee4475 Fix default update= on ant builds 2015-12-02 20:16:25 +01:00
Daniel Martí
bf8518ee8f Rework build into a class
This simplifies usage, goes from

    build['flag']
to
    build.flag

Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.

As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.

While at it, unify "build", "thisbuild", "info", "thisinfo", etc into
just "build".
2015-12-02 20:03:11 +01:00
Daniel Martí
ab614ab442 Rework app into a class
This simplifies usage, goes from

	app['Foo']
to
	app.Foo

Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.

As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.
2015-11-28 17:11:05 +01:00
Daniel Martí
5e1b07dce3 import: don't crash due to missing id
Fixes #129
2015-11-18 15:33:30 -08:00
Daniel Martí
3a616e2d9f Don't crash if AM.xml has no application element 2015-11-16 07:21:19 -08:00
Daniel Martí
c97b0e7ed9 common: error if any glob paths are unused
Like rm=foo and foo doesn't exist.
2015-11-14 13:05:16 +01:00
Daniel Martí
66e82cb077 Bump build-tools to 23.0.2 2015-11-05 11:56:13 +01:00
Daniel Martí
a4d1fa22f7 Install Java 1.8 alongside 1.7 for retrolambda
Fixes #103.
2015-11-03 11:08:34 +01:00
Daniel Martí
c8627764e3 checkupdates: check packages in AM.xml files too 2015-11-01 12:35:36 +01:00
Daniel Martí
3ace102bd5 checkupdates: work around multiple package ids
If there are multiple package ids in a build.gradle file and the first
one we don't want, checkupdates would get stuck. Make it ignore any
package id that we don't want so that it can get past that.
2015-10-30 19:03:53 +01:00
Daniel Martí
37b13c37d5 Also fix local.prop files before subdir
If subdir=foo/bar and there is a foo/local.properties file, it would not
get treated.
2015-10-26 16:49:11 +01:00
Daniel Martí
9017328698 Add an extra strip() to work around tostring issue
In cases like this xml code:

	<string name="app_name">"OpenKeychain"</string>
	<!-- title -->
	<string name="title_encrypt_text">"Encrypt"</string>

tostring() returns trailing whitespaces (including newlines). Which
aren't removed until the very end, after we try to remove enclosing
quotes. So strip right after tostring() too, since we never really care
about whitespaces anyway.
2015-10-26 00:28:29 +01:00
Daniel Martí
6ea51abfaa common: Compile more regexes only once 2015-10-25 19:10:49 +01:00
Daniel Martí
2872027835 Also replace git@gitlab.com submodule urls 2015-10-24 19:42:40 +02:00
Daniel Martí
78a661fc7d Typo fix for 91daacb889 2015-10-24 18:37:57 +02:00
Daniel Martí
91daacb889 common: don't crash if an sdk binary is not found 2015-10-24 17:02:53 +02:00
Daniel Martí
48645072ee extension simplifications 2015-10-04 18:01:32 +02:00
Daniel Martí
1aa891c4fd Add basic yaml metadata writing 2015-10-04 07:55:42 -07:00
Daniel Martí
59f5d19dfe Rework path glob expansion
Slightly simplifies the whole thing and lets us map what each resulting path
comes from. This will be useful to fix #110 later on.
2015-10-03 16:52:23 -07:00
Daniel Martí
4ea5ce8889 checkupdates: Ignore xml tags in string content
This allows us to fetch strings like the following:

<string name="app_name">foo <xliff:g>bar</xliff:g></string>

Up until now, using .text would only return "foo ", but if we use .tostring()
with the text method, it converts everything into plain text for us resulting
in "foo bar".
2015-09-30 16:35:41 -07:00
Daniel Martí
79475d055f Fetch string contents in a safer way 2015-09-30 16:35:41 -07:00
Daniel Martí
cc017d70d2 Fix for bb2cbd527b 2015-09-29 20:39:36 -07:00
Daniel Martí
bb2cbd527b Increase wiki error detail length limit
8k is too small, e.g. when make -jN errors or when there are a lot of scanner
errors and warnings. 16k should be better. Also, use "[...]" to make it
obvious that the output is truncated.
2015-09-29 12:51:11 -07:00
Daniel Martí
27a5dc8429 checkupdates: ignore gradle comments 2015-09-23 18:52:59 -07:00
Daniel Martí
83daf71338 gradle: don't strip out comments
This was never intended. Comments can help sed usage, and can still be useful
in source tarballs.
2015-09-22 11:52:16 -07:00
NeroBurner
694a4ce7da Add asynchronous filereader, fix python3 lockups 2015-09-20 20:57:53 +02:00
Daniel Martí
055759cf76 Rewrite scanner logic
Initially, the scanner used libmagic which used magic numbers in the file's
content to detect what kind of file it appears to be. Since that library isn't
available on all systems, we added support for two other libraries, mimetypes
amongst them.

The issue with mimetypes is that it only uses the file's extension, not its
actual content. So this ends in variable behaviour depending on what system
you're using fdroidserver on. For example, an executable binary without
extension would be ignored if mimetypes was being used.

We now drop all libraries - mimetypes too as it depends on the system's
mime.types file - and instead check extensions ourselves. On top of that, do
a simple binary content check to find binary executables that don't have an
extension.

The new in-house code without any dependencies doesn't add any new checks, so
no builds should break. The current checks still work:

 % fdroid scanner app.openconnect:1029
[...]
Found executable binary at assets/raw/armeabi/curl
Found executable binary at assets/raw/mips/curl
Found executable binary at assets/raw/x86/curl
Found JAR file at lib/XposedBridgeApi-54.jar
Found JAR file at libs/acra-4.5.0.jar
Found JAR file at libs/openconnect-wrapper.jar
Found JAR file at libs/stoken-wrapper.jar
Found shared library at libs/armeabi/libopenconnect.so
Found shared library at libs/armeabi/libstoken.so
Found shared library at libs/mips/libopenconnect.so
Found shared library at libs/mips/libstoken.so
Found shared library at libs/x86/libopenconnect.so
Found shared library at libs/x86/libstoken.so
2015-09-13 22:17:37 -07:00
Daniel Martí
576da1d048 all: deduplicate -v/-q setup 2015-09-11 23:42:50 -07:00
Daniel Martí
7a79b000bf Allow slashes in tags
The wikipedia app uses them, for example.
2015-09-11 23:17:36 -07:00
Daniel Martí
03deda914f checkupdates improvements for gradle projects
* Let the package name override the one found in AndroidManifest.xml
* Support the newer applicationId
2015-09-08 13:12:28 -07:00
Daniel Martí
41443edd55 Bump build-tools 2015-09-05 23:02:03 -04:00
Daniel Martí
91f36acaa6 Even examples/config.py with default_config 2015-09-01 15:30:41 -07:00
Hans-Christoph Steiner
325db90918 set "accepted formats" for metadata in config.py
For a bit repo like f-droid.org, it makes sense to standardize on a single
format for metadata files.  This adds support for enforcing a single data
format, or a reduced set of data formats.  So f-droid.org would run like
this if it changed to YAML:

  accepted_formats = ['txt', 'yaml']

Then once everything was converted to YAML, it could look like this:

  accepted_formats = ['yaml']
2015-09-01 11:53:37 +02:00