Commit graph

1215 commits

Author SHA1 Message Date
Jochen Sprickerhof
b6f1f4231a Drop code for the old wiki 2021-11-24 18:16:07 +01:00
Gaurav Ujjwal
aead3310bd NDK Install: Handle symlinks present in NDK zip 2021-11-03 08:13:21 +00:00
Jochen Sprickerhof
fa3cceb8e8 git prune branches in case of failure
git branches can be namespaces like directories on a filesystem and are
represented like that. Due to that there can't be a branch with the same
name as a namespace, i.e. foo and foo/bar.
If upstream moves from a branch to namespace, we need to prune the old
branch before fetching the new one.
This broke organic maps:

From https://github.com/organicmaps/organicmaps
 * [new branch]            android/huawei       -> origin/android/huawei
error: cannot lock ref 'refs/remotes/origin/fixes/all': 'refs/remotes/origin/fixes' exists; cannot create 'refs/remotes/origin/fixes/all'
 ! [new branch]            fixes/all            -> origin/fixes/all  (unable to update local ref)
   e2ac324b95..320a1db39b  master               -> origin/master
 * [new tag]               2021.10.09-2-android -> 2021.10.09-2-android
error: some local refs could not be updated; try running
 'git remote prune origin' to remove any old, conflicting branches
2021-10-14 12:42:29 +00:00
Gaurav Ujjwal
07856dc0e0 Fix invalid key error due to NDK versions in "revision" form (e.g. 21.4.7075529)
First element in the 'NDKS' array does not have 'revision'  key, which results in 'KeyError' when accessed through subscript operator.
2021-09-19 09:44:37 +00:00
Felix C. Stegerman
6bd8470d55 find_sdk_tools_cmd(): s/found/not found/ in error message 2021-09-12 22:40:50 +00:00
Hans-Christoph Steiner
7987c746de index: do not include GitLab Pages mirror if it can't be deployed
GitLab Pages sites are limited to 1GB on gitlab.com, so the CI/CD job will
fail if the repo is bigger than that.  It should not be included as a
mirror in that case.

https://docs.gitlab.com/ee/user/gitlab_com/#gitlab-pages
2021-09-11 15:30:55 +00:00
Jochen Sprickerhof
b62124853a Remove waring about no config file
fdroid works fine without a config so there is no reason to warn about
it.
2021-08-23 11:41:44 +00:00
fdroid-bot
54bc6f05de Android NDK None (None) 2021-08-23 10:48:53 +00:00
Jochen Sprickerhof
cacbe88b8c Don't fail fdroid update on empty repository
Steps to reproduce:

$ git init
$ fdroid update --create-key
2021-08-05 16:05:28 +00:00
Jochen Sprickerhof
331b4830dd Support multiple flavours in parse_androidmanifests
Previously only the last flavour was checked.
2021-07-24 21:20:19 +02:00
Hans-Christoph Steiner
40c77892a2
do not crash when config.yml is 0 bytes or empty of data 2021-07-01 14:45:35 +02:00
Benedikt Brückmann
675d034ac7 Fix errors from merge and review 2021-06-25 06:42:17 +02:00
Benedikt Brückmann
3d85b91dc4 rewrite docstrings to match numpy style guide 2021-06-25 06:28:47 +02:00
Benedikt Brückmann
9038161b70 apply numpy docstyle 2021-06-25 06:28:47 +02:00
Benedikt Brückmann
01b9fb1549 Fix pep8 whitespace errors 2021-06-25 06:28:47 +02:00
Benedikt Brückmann
1e943a22df rewrite docstrings to match numpy style guide 2021-06-25 06:28:47 +02:00
Benedikt Brückmann
d168b9c05b rewrite docstrings to match numpy style guide 2021-06-25 06:28:43 +02:00
linsui
763a2ee80d fix invalid extension of output apk 2021-06-24 13:42:10 +00:00
Hans-Christoph Steiner
1f5534d060
require config.yml use UTF-8 as encoding
config.yml requires ASCII or UTF-8 encoding because this code does not
auto-detect the file's encoding.  That is left up to the YAML library.
YAML allows ASCII, UTF-8, UTF-16, and UTF-32 encodings.  Since it is a
good idea to manage config.yml (WITHOUT PASSWORDS!) in git, it makes
sense to use a globally standard encoding.
2021-06-18 11:12:20 +02:00
Hans-Christoph Steiner
48c4354629
always open Android source files as UTF-8
Android Studio recommends "you use UTF-8 encoding whenever possible",
so this code assumes the files use UTF-8.  UTF-8 is also the default
encoding on GNU/Linux and macOS.
https://sites.google.com/a/android.com/tools/knownissues/encoding

Windows will probably default to UTF16, since that's the native
encoding for files.  So forcing things to use UTF-8 should help
compatibility.
2021-06-18 11:12:18 +02:00
Gerhard Olsson
0c31c4a5ab
win fixes 2021-06-18 10:30:59 +02:00
Hans-Christoph Steiner
1e1f0db347
stats/known_apks.txt must be in UTF-8 format on all platforms 2021-06-18 10:30:57 +02:00
Hans-Christoph Steiner
a6d35a7ee1
metadata: always open metadata files as UTF-8
Windows seems to require this, otherwise this happens:

Traceback (most recent call last):

  File "tests/update.TestCase", line 737, in test_translate_per_build_anti_features

    apps = fdroidserver.metadata.read_metadata(xref=True)

  File "C:\Users\travis\build\fdroidtravis\fdroidserver\fdroidserver\metadata.py", line 813, in read_metadata

    app = parse_metadata(metadatapath, appid in check_vcs, refresh)

  File "C:\Users\travis\build\fdroidtravis\fdroidserver\fdroidserver\metadata.py", line 1023, in parse_metadata

    parse_yaml_metadata(mf, app)

  File "C:\Users\travis\build\fdroidtravis\fdroidserver\fdroidserver\metadata.py", line 1073, in parse_yaml_metadata

    yamldata = yaml.safe_load(mf)

  File "C:\python37\lib\site-packages\yaml\__init__.py", line 162, in safe_load

    return load(stream, SafeLoader)

  File "C:\python37\lib\site-packages\yaml\__init__.py", line 112, in load

    loader = Loader(stream)

  File "C:\python37\lib\site-packages\yaml\loader.py", line 34, in __init__

    Reader.__init__(self, stream)

  File "C:\python37\lib\site-packages\yaml\reader.py", line 85, in __init__

    self.determine_encoding()

  File "C:\python37\lib\site-packages\yaml\reader.py", line 124, in determine_encoding

    self.update_raw()

  File "C:\python37\lib\site-packages\yaml\reader.py", line 178, in update_raw

    data = self.stream.read(size)

  File "C:\python37\lib\encodings\cp1252.py", line 23, in decode

    return codecs.charmap_decode(input,self.errors,decoding_table)[0]

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 37: character maps to <undefined>
2021-06-18 10:30:55 +02:00
Jochen Sprickerhof
cd405cc93e Use hash of tag when adding new version
Also extend getref with a refname parameter and implement it for
vcs_git.
2021-06-17 11:51:18 +00:00
Jochen Sprickerhof
3809b4d424 Ignore git submodule failure in gotorevisionx
gotorevisionx tries to clean up the git repo before checking out a new
revision. In b848b99ba this was changed to reset and clean any submodule
as well. In case upstream has a broken submodule configuration this
could fail and we can't checkout the new revision. As we are doing a
reset and clean after checking out the new revision anyhow, this change
ignores submodule errors before the checkout and only makes sure that
the main repo is reset and clean.

This broke checkupdates for apps where old versions had broken
submodules. It checkout out the old version and got stuck, not able to
checkout any other version.
2021-06-15 20:39:18 +02:00
Jochen Sprickerhof
4e97b58d8c latesttags revert to git log and fix comma handling
2de34312 tried to fix the comma handling by relying on git tag --sort.
This did not work out so this reverts to the method used before.
2021-06-15 08:39:59 +02:00
Jochen Sprickerhof
2de3431296 Use git tag in latesttags 2021-06-14 18:56:59 +00:00
Jochen Sprickerhof
1e6de7eb34 Support '{' in extra line in parse_androidmanifests
If the flavour group starts in a separate line don't count it as a
second group.

Closes: #899
2021-06-13 07:23:42 +02:00
Jochen Sprickerhof
0fefecde1e Fix matching substring flavour detection
com.github.jameshnsears.quoteunquote defines flavours 'fdroid' and
'fdroidS'. The old code used flavour in line, which matches both and the
wrong one was selected.

Closes: #912
2021-06-13 00:09:02 +02:00
Jochen Sprickerhof
a9a336a12b Don't allow '_' in first place of version code
Fixes a regression of 2cb0ff45. Example:

versionCode project.versionCode_plus

931b0a3087/app/build.gradle (L38)
2021-06-09 11:04:25 +00:00
linsui
6bafb036ee lint.py: use pathlib and support Windows 2021-06-09 15:46:52 +08:00
linsui
8f21f1e510 metadata.py/rewritemeta.py: use pathlib and support Windows 2021-06-08 21:31:55 +08:00
linsui
d6eece6395 import.py: use pathlib and support Windows 2021-06-08 18:33:22 +08:00
linsui
92ec6265a3 checkupdates.py: use pathlib and support Windows 2021-06-07 21:02:03 +08:00
Hans-Christoph Steiner
d6c8ecc3bc
fix broken NDK extraction routine to correctly name resulting dir
If there were other NDKs installed, this just tried to rename the first
installed NDK.  Now it tries to rename all NDKs that are not named after
the NDK version.

!926
!927
2021-06-01 14:33:39 +02:00
Hans-Christoph Steiner
153b5d4392
stop modifying default_config when running fill_config_defaults()
dicts and lists are passed by reference in assignments, so this needs to
copy them instead.
2021-05-28 10:24:58 +02:00
Hans-Christoph Steiner
09fa49a7a3
make get_android_tools_versions() search ndk_paths from config 2021-05-28 09:13:38 +02:00
Hans-Christoph Steiner
7a1d236c8d
only support zipballs in NDK provisioning
Since I discovered there is an r10e zipball, this can now get all NDKs
in zipball form.
fdroid/android-sdk-transparency-log@447fea86e7

closes #902
2021-05-28 09:13:36 +02:00
Hans-Christoph Steiner
9f77044d0d
auto-detect NDKs installed in standard paths
'ndk_paths' will be automatically filled out from well known sources
like $ANDROID_HOME/ndk-bundle and $ANDROID_HOME/ndk/*.  If a required
version is missing in the buildserver VM, it will be automatically
downloaded and installed into the standard $ANDROID_HOME/ndk/
directory.  Manually setting it here will override the auto-detected
values.  The keys can either be the "release" (e.g. r21e) or the
"revision" (e.g. 21.4.7075529).

https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
* sdkmanager installs "ndk;12.3.4567890" into $ANDROID_SDK_ROOT/ndk/
* sdkmanager installs "ndk-bundle" into $ANDROID_SDK_ROOT/ndk-bundle/
2021-05-28 09:13:34 +02:00
Hans-Christoph Steiner
4686c06f62 metadata: allow ndk: to be str or list of release or revision
There are two version numbers used for NDKs: the "release" and the
"revision".  The "release" is used in the download URL and zipball and the
"revision" is used in the source.properties and the gradle ndkVersion field.

Also, there are some builds which need multiple NDKs installed, so this
makes it possible to have a list of release/revision entries in build.ndk.
This does not yet add full support since _fdroidserver/build.py_ will also
need changes.
2021-05-28 09:13:27 +02:00
Hans-Christoph Steiner
93145a43fb buildserver: use standardized paths for the NDKs
This should help with reproducibility since some tools like Python still
include the build paths in the binaries.
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/919#note_578180986

The default ANDROID_SDK_ROOT base dir of /opt/android-sdk is hard-coded in
buildserver/Vagrantfile.  The $ANDROID_HOME/ndk subdir is where Android
Studio will install the NDK into versioned subdirs.
https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
2021-05-26 09:21:52 +02:00
Hans-Christoph Steiner
9fc2a23713
build: remove default NDK, closes #717 2021-05-25 17:06:26 +02:00
Hans-Christoph Steiner
69fcd6a024
build: auto-download missing NDKS if they're known and can be verified
refs #517 #717
2021-05-25 17:06:24 +02:00
Hans-Christoph Steiner
7a7ecbf9dc
move sha256sum() and sha256base64() to common 2021-05-25 17:06:23 +02:00
Jochen Sprickerhof
84b6617faf Don't return Gradle variables as string
This is used in checkupdates and currently sets the AutoName to
something like ${displayName}:

4ae415f73c
2021-05-13 12:53:19 +00:00
Felix C. Stegerman
b08d54cb90
fix metadata_find_signing_files() 2021-04-14 23:52:13 +02:00
Hans-Christoph Steiner
2946c90dd4 publish: rename vars to match naming in JAR Signature docs
https://docs.oracle.com/javase/tutorial/deployment/jar/intro.html

closes #892
2021-04-14 23:01:47 +02:00
Hans-Christoph Steiner
416790306f common.py: update copyrights 2021-04-14 23:01:47 +02:00
Hans-Christoph Steiner
4308a48717 apksigcopier: remove exclude_all_meta logic
This option is unneeded and unused in fdroidserver, and confusing.

https://github.com/obfusk/apksigcopier/pull/36

# Conflicts:
#	fdroidserver/apksigcopier.py
#	fdroidserver/common.py
2021-04-14 23:01:47 +02:00
Hans-Christoph Steiner
d28c1c0441 force apksigcopier "AUTO" to select sigs by available files 2021-04-14 23:01:47 +02:00