Commit graph

577 commits

Author SHA1 Message Date
Marcus Hoffmann
597fc04c9e build: rsync instead of sftp dirs to the buildserver
Sending many small files with paramiko's sftp implementation is quite
slow. There seems to be no way around this, other projects (ansible)
recommend to use rsync in this case.

Our sourcecode directories sometimes have a LOT of files, it can take
up to an hour to copy all required things inside the buildserver.

Instead we just use rsync with the ssh options we get from vagrant.
For rsync specific options we use:

* sync permissions as before
* sync symlinks (was done partially before)
* don't sync hardlinks and ownership as these don't make for things
  coming from a VCS

Closes fdroid/fdroidserver#227.
2017-11-25 14:07:06 +01:00
relan
cd8a86a2a2 build: raise VCS client version logging level
Debug logging is not saved in buildserver logs. Also move this line into
build.py's main() to avoid duplication: common.setup_vcs(app) may be
called from parse_metadata() too.
2017-11-02 08:52:02 +03:00
relan
aad4bd89d8 build: fix build/ and .gradle/ removal after cleaning
An app may not have a top-level build.gradle file, while Gradle still
creates top-level .gradle/ directory. When build.gradle is absent,
fdroidserver will not remove .gradle/, scanner will find binaries in it
and fail the whole build.

Fix this by also checking for settings.gradle file (should always exist
in the top-level directory) in addition to build.gradle.
2017-10-01 10:11:37 +03:00
Michael Pöhn
d8485f7123 build+drozer: error message instead of crash when repo-folder missing 2017-09-25 16:09:57 +02:00
Hans-Christoph Steiner
53e4ec47a7 English source string corrections
From @monolifed <monolifed@gmail.com>
2017-09-16 13:19:38 +02:00
Hans-Christoph Steiner
c7d45b9515 Merge branch 'plural' into 'master'
Correct plurals

See merge request fdroid/fdroidserver!334
2017-09-16 08:11:06 +00:00
Jan Berkel
4723cff0c8 Correct plurals 2017-09-15 14:16:30 +02:00
Hans-Christoph Steiner
fa0d618d2f rename 'app-id' to standard Android 'applicationId'
* https://developer.android.com/studio/build/application-id.html
* https://sites.google.com/a/android.com/tools/tech-docs/new-build-system/applicationid-vs-packagename

This only changes the term in the human texts, not var names or CLI flags.
2017-09-15 11:41:51 +02:00
Hans-Christoph Steiner
278d67d960 implement gettext localization
This allows all the text to be localized via Weblate.  This is a quick
overview of all the strings, but there are certainly some that were left
out.

closes #342
2017-09-15 11:39:00 +02:00
Hans-Christoph Steiner
cb10f0df09 standardize os.walk() var names based on Python 3.5 docs
There were multiple conventions used in the code, but mostly it was already
using the convention from the docs, so this converts things to using that
convention:

https://docs.python.org/3/library/os.html#os.walk
2017-09-15 11:37:21 +02:00
Michael Pöhn
5770e1d3ef log build-server output when verbose flag is set 2017-09-14 01:38:34 +02:00
Marcus Hoffmann
b148b82f83 add comments to gradle output directories 2017-08-26 17:03:13 +02:00
Marcus Hoffmann
b114b44020 handle gradle-plugin 3.0 output apk location
This commit adds support for new gradle plugin 3.0 output directories.
The new structure looks like this:

build/outputs/apk/<flavour1><Flavour2>/release/*.apk

Note the capitalization on the different flavour components.
So if we build a specific flavour combination we add this combination to
the path where we look for an output .apk.

Closes #363

Based on !320 by: Michel Le Bihan <michel@lebihan.pl>
Signed-off-by: Marcus Hoffmann <bubuiic@aol.com>
2017-08-26 16:03:31 +02:00
Hans-Christoph Steiner
67d98c5a36 keep .apk file ext when diffoscope'ing Binaries: 2017-07-19 15:07:50 +02:00
mimi89999
a427bb7b56
Also search for apk in build/outputs/apk/release 2017-07-07 12:21:09 +02:00
Hans-Christoph Steiner
dfb07808d3 support configing buildserver VM per-build with sudo=
This adds the 'sudo' build field, which is just a script that is run as
root.  For more info, see the issue that this closes:

refs #318
closes #317
2017-07-06 17:29:23 +02:00
Michael Pöhn
cfd3e84250 use stored metadatapath instead of guessing it 2017-07-04 11:51:08 +02:00
Michael Pöhn
75ebb6e8cc fix building with yml metadata 2017-07-04 11:51:08 +02:00
lb@lb520
0672a6f1ec build.py corrected 2017-06-20 15:56:15 +02:00
lb@lb520
230286786f Added a method to build python/kivy projects using buildozer. 2017-06-20 14:16:31 +02:00
mimi89999
2626858450
Don't pass root_dir to scan_source 2017-05-25 20:11:14 +02:00
Hans-Christoph Steiner
17cd07f1a7 fix pylint unused-argument 2017-05-23 22:34:16 +02:00
Michael Pöhn
c8234919df refactored vm related code from build.py to vmtools.py 2017-05-23 20:08:29 +02:00
Michael Pöhn
fc660048eb removed unused buildserver code 2017-05-23 20:08:08 +02:00
Michael Pöhn
8f1fabfed6 restart builder vm when ssh connection fails 2017-05-23 20:06:47 +02:00
Hans-Christoph Steiner
6106b962a2 build: delete bad builder/ symlinks
If builder/ is a symlink but is not detected as a directory by
os.path.isdir(), then it is a broken symlink.
2017-05-23 20:06:06 +02:00
Michael Pöhn
4546929d7f wait a sec after suspending 2017-05-23 20:06:06 +02:00
Michael Pöhn
48159f005a reset buildserver vm if vagrant uuid not present 2017-05-23 20:06:06 +02:00
Michael Pöhn
358b00d7aa removed useless vm validity check; attempted to fix vbox support 2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
acf25a3999 build server use up instead or resume; logging 2017-05-23 20:06:06 +02:00
Michael Pöhn
3187d2cbcf revised build server creation 2017-05-23 20:06:06 +02:00
Michael Pöhn
a414aa00ff use overhauled mv destroy code in build.py 2017-05-23 20:06:06 +02:00
Michael Pöhn
92fada803e overhauled and moved destroying builder vm to vmtools.py 2017-05-23 20:04:08 +02:00
Michael Pöhn
fb03e17849 fdroid build: added sleep after destroy/undefine 2017-05-23 20:04:08 +02:00
Michael Pöhn
413c3836d5 auto-reset broken builder vm 2017-05-23 20:04:08 +02:00
Michael Pöhn
ce3c959ce5 scan vm provider again after re-creating builder vm 2017-05-23 20:04:08 +02:00
Michael Pöhn
34cddd3be8 delete associated libvirt domain/image when destroying builder vm 2017-05-23 20:04:08 +02:00
Michael Pöhn
fad98eeb7f build: fixed kvm snapshot support; makebuildserver: setup kvm ssh credentials 2017-05-23 20:04:08 +02:00
Michael Pöhn
fa2d44ee94 added libvirt vm-provider support to build.py 2017-05-23 20:04:08 +02:00
Torsten Grote
1fcd8e63a3
Replace sys.exit() in non-main functions by exceptions
Also move all exceptions into one module
2017-05-22 16:51:11 -03:00
Hans-Christoph Steiner
0345d2c79f update: find aapt when it is not in the PATH
This is some very messy logic built up since 2010.  This will all go away
once we have a python3 version of androguard available.

The removed imports and `dir(APK)` is to silence pyflakes

closes #303
2017-05-18 17:21:05 +02:00
thez3ro
06598ae406
use androguard if aapt isn't found 2017-05-04 23:35:17 +02:00
Michael Pöhn
fc4e40255d remove build output when not reproducible 2017-04-23 11:33:51 +02:00
Michael Pöhn
5b7209d882 unified method for naming build output 2017-04-22 12:04:32 +02:00
Michael Pöhn
62af9b8e3d check meld config for manual apk diffs 2017-04-22 10:48:50 +02:00
Michael Pöhn
4044e06e7b use verify_apks for checking build output 2017-04-22 10:48:50 +02:00
Michael Pöhn
a3e7eacc9b compare apk with Binaries from metadata right after building 2017-04-22 10:48:50 +02:00
Michael Pöhn
abe04272b1 throw proper error when download of apks fail 2017-04-20 17:48:38 +02:00
Michael Pöhn
843595eb33 moved version android tools version log header to corresponding function 2017-03-22 14:56:26 +01:00
Michael Pöhn
0fda23ef80 fix local var name overloading when iterating build flavours 2017-03-22 14:20:54 +01:00