Commit graph

6507 commits

Author SHA1 Message Date
Hans-Christoph Steiner
05bc59c56a gitlab-ci: paramiko from Debian to avoid crypto libs from source
Before it was cryptography, now bcrypt requires a Rust compiler:
https://gitlab.com/fdroid/fdroidserver/-/jobs/2929483724
2022-09-06 11:16:36 +02:00
Hans-Christoph Steiner
06e4d9623a gitlab-ci: fix plugin_fetchsrclibs by using Debian packages
* https://gitlab.com/eighthave/fdroidserver/-/jobs/1961701458

Collecting pynacl>=1.0.1 (from paramiko->fdroidserver==2.1a0)
  Downloading https://files.pythonhosted.org/packages/a7/22/27582568be639dfe22ddb3902225f91f2f17ceff88ce80e4db396c8986da/PyNaCl-1.5.0.tar.gz (3.4MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'
  Complete output from command /builds/eighthave/fdroidserver/env/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-xokvr6uk --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=40.8.0 wheel "cffi>=1.4.1; python_implementation != 'PyPy'":
  Collecting setuptools>=40.8.0
    Using cached https://files.pythonhosted.org/packages/eb/53/0dd4c7960579da8be13fa9b2c2591643d37f323e3d79f8bc8b1b6c8e6217/setuptools-60.5.0-py3-none-any.whl
  Collecting wheel
    Using cached https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl
  Collecting cffi>=1.4.1
    Downloading https://files.pythonhosted.org/packages/00/9e/92de7e1217ccc3d5f352ba21e52398372525765b2e0c4530e6eb2ba9282a/cffi-1.15.0.tar.gz (484kB)
  Collecting pycparser (from cffi>=1.4.1)
    Downloading https://files.pythonhosted.org/packages/62/d5/5f610ebe421e85889f2e55e33b7f9a6795bd982198517d912eb1c76e1a53/pycparser-2.21-py2.py3-none-any.whl (118kB)
  Building wheels for collected packages: cffi
    Running setup.py bdist_wheel for cffi: started
    Running setup.py bdist_wheel for cffi: finished with status 'error'
    Complete output from command /builds/eighthave/fdroidserver/env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-djg9jc8p/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-d1knhl7l --python-tag cp37:
    unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
    unable to execute 'x86_64-linux-gnu-gcc': No such file or directory

        No working compiler found, or bogus compiler options passed to
        the compiler from Python's standard "distutils" module.  See
        the error messages above.  Likely, the problem is not related
        to CFFI but generic to the setup.py of any Python package that
        tries to compile C code.  (Hints: on OS/X 10.8, for errors about
        -mno-fused-madd see http://stackoverflow.com/questions/22313407/
        Otherwise, see https://wiki.python.org/moin/CompLangPython or
        the IRC channel #python on irc.libera.chat.)

        Trying to continue anyway.  If you are trying to install CFFI from
        a build done in a different context, you can ignore this warning.

    running bdist_wheel
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/cffi
    copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/lock.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/recompiler.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/__init__.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/cparser.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/verifier.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/api.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/commontypes.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/error.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/model.py -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/_embedding.h -> build/lib.linux-x86_64-3.7/cffi
    copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.7/cffi
    running build_ext
    building '_cffi_backend' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/c
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/builds/eighthave/fdroidserver/env/include -I/usr/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
    unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
    Failed building wheel for cffi
    Running setup.py clean for cffi
  Failed to build cffi
  Installing collected packages: setuptools, wheel, pycparser, cffi
    Running setup.py install for cffi: started
      Running setup.py install for cffi: finished with status 'error'
      Complete output from command /builds/eighthave/fdroidserver/env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-djg9jc8p/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ek80c81s/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-xokvr6uk --compile --install-headers /builds/eighthave/fdroidserver/env/include/site/python3.7/cffi:
      unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
      unable to execute 'x86_64-linux-gnu-gcc': No such file or directory

          No working compiler found, or bogus compiler options passed to
          the compiler from Python's standard "distutils" module.  See
          the error messages above.  Likely, the problem is not related
          to CFFI but generic to the setup.py of any Python package that
          tries to compile C code.  (Hints: on OS/X 10.8, for errors about
          -mno-fused-madd see http://stackoverflow.com/questions/22313407/
          Otherwise, see https://wiki.python.org/moin/CompLangPython or
          the IRC channel #python on irc.libera.chat.)

          Trying to continue anyway.  If you are trying to install CFFI from
          a build done in a different context, you can ignore this warning.

      running install
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.7
      creating build/lib.linux-x86_64-3.7/cffi
      copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/lock.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/recompiler.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/__init__.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/cparser.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/verifier.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/api.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/commontypes.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/error.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/model.py -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/_embedding.h -> build/lib.linux-x86_64-3.7/cffi
      copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.7/cffi
      running build_ext
      building '_cffi_backend' extension
      creating build/temp.linux-x86_64-3.7
      creating build/temp.linux-x86_64-3.7/c
      x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/builds/eighthave/fdroidserver/env/include -I/usr/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
      unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
      error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

      ----------------------------------------
  Command "/builds/eighthave/fdroidserver/env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-djg9jc8p/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ek80c81s/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-xokvr6uk --compile --install-headers /builds/eighthave/fdroidserver/env/include/site/python3.7/cffi" failed with error code 1 in /tmp/pip-install-djg9jc8p/cffi/

  ----------------------------------------
Command "/builds/eighthave/fdroidserver/env/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-xokvr6uk --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=40.8.0 wheel "cffi>=1.4.1; python_implementation != 'PyPy'"" failed with error code 1 in None
2022-09-06 10:58:18 +02:00
Jochen Sprickerhof
9b135077d7 [CI] install python3-matplotlib from apt 2022-09-06 10:56:14 +02:00
Jochen Sprickerhof
53d559a913 [CI] Install python3-pil from apt
New version would need to be compiled, otherwise.
2022-09-06 10:55:11 +02:00
Jochen Sprickerhof
5e58141f30 [CI] fetchsrclibs use apt cffi 2022-09-06 10:53:25 +02:00
Hans-Christoph Steiner
9be68357ae nightly: add support for GitHub Actions 2022-09-06 10:46:05 +02:00
Hans-Christoph Steiner
0b0e514939 nightly: update Raw URLs to fix breakage and avoid redirects 2022-09-06 10:45:04 +02:00
Jochen Sprickerhof
e367ab0c1c Fix added type
The added time should be a datetime, not a time.struct_time.

Regression of 290b9050
Closes: #1012
2022-09-06 10:42:31 +02:00
Hans-Christoph Steiner
516624c9ce nightly: fix "deploy key URL" that is printed to log output
GitLab now uses the "/-/" form as the canonical.
2022-08-03 14:55:17 -07:00
Hans-Christoph Steiner
82f6ebae5d
version 2.0.4 2022-06-29 10:34:29 +02:00
Hans-Christoph Steiner
777f55ebe7 update CHANGELOG 2022-06-29 10:34:06 +02:00
Hans-Christoph Steiner
ec3d69b9d3 signindex: do not remove index-v1.json after signing
With ~index-v2, the model is changing to offer the plain JSON file for easy
consumption.  Then gpgsign will also provide a detached PGP signature for
systems that would rather verify based on PGP signatures than JAR signatures.

!1080
!1124
#969

# Conflicts:
#	tests/signindex.TestCase
2022-06-29 10:28:43 +02:00
Hans-Christoph Steiner
13e165305d deploy: ensure progress is instantiated before trying to use it 2022-06-29 10:28:35 +02:00
Hans-Christoph Steiner
d432c6396c
version 2.0.3 2021-07-01 11:08:58 +02:00
Hans-Christoph Steiner
a72bee0e4f
update CHANGELOG 2021-07-01 11:08:55 +02:00
Jochen Sprickerhof
943c2b64bf
Support AutoUpdateMode: Version without pattern
Since 24dd6740 UpdateCheckMode: Tags uses the found tag instead of
regenerating it from the AutoUpdateMode pattern making the pattern
superfluous. This adds support for dropping the pattern and a test case.
2021-07-01 11:08:48 +02:00
Hans-Christoph Steiner
ceac1f7264
version 2.0.2 2021-06-01 14:21:30 +02:00
Hans-Christoph Steiner
54ce657b4a
update CHANGELOG 2021-06-01 14:21:29 +02:00
Jochen Sprickerhof
9d589071f0
Fix ruamel round_trip_dump will be removed
https://sourceforge.net/p/ruamel-yaml/code/ci/4b91cd7e820baf97d8d28b1dcf77979530d43c3c/#diff-1

!932
2021-06-01 14:21:27 +02:00
Hans-Christoph Steiner
8e8b87e0b4
gitlab-ci: fix tests using updates from master 2021-06-01 14:21:25 +02:00
Hans-Christoph Steiner
5dddb29bc7 version 2.0.1 2021-03-09 17:53:04 +01:00
Hans-Christoph Steiner
eac75a93bc update CHANGELOG 2021-03-09 17:53:04 +01:00
Hans-Christoph Steiner
8939a336ae run-tests: fix local_copy_dir test, it broke when /tmp/fdroid existed 2021-03-09 17:53:04 +01:00
Hans-Christoph Steiner
92438bbf78 metadata: stop setting up source repo when running lint/rewritemeta
broken in 54e8867a7e
2021-03-09 17:53:03 +01:00
Hans-Christoph Steiner
ff1638c7dc scanner: show error if scan_binary fails to run apkanalyzer 2021-03-09 15:58:04 +01:00
Hans-Christoph Steiner
8b89793d3c common: properly parse version from NDK's source.properties 2021-03-09 15:58:00 +01:00
Hans-Christoph Steiner
2b5093969a update: stop extracting and storing XML icons, they're useless
APKs that only have XML icons in them should include an icon.png in the
metadata or fastlane/triple-t

closes #344
2021-03-09 15:38:19 +01:00
Hans-Christoph Steiner
240139baf9 index: raise error rather than crash on bad repo file
If a non-APK is added with the appid/packageName that matches some APKs, it
should through an error.

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 211, in main
    mod.main()
  File "/home/hans/code/fdroid/server/fdroidserver/update.py", line 2343, in main
    index.make(apps, sortedids, apks, repodirs[0], False)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 142, in make
    fdroid_signing_key_fingerprints)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 166, in make_v1
    v1_sort_packages(packages, fdroid_signing_key_fingerprints)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 292, in v1_sort_packages
    packages.sort(key=v1_sort_keys)
  File "/home/hans/code/fdroid/server/fdroidserver/index.py", line 288, in v1_sort_keys
    .format(apkfilename=package['apkName']))
fdroidserver.exception.FDroidException: at.roteskreuz.stopcorona_8.jobf does not have a valid signature!
2021-03-09 15:38:19 +01:00
Hans-Christoph Steiner
cfbee12ad2 update: handle large, corrupt, or inaccessible fastlane/triple-t files
```
Traceback (most recent call last):
  File "../fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/__main__.py", line 227, in main
    raise e
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/__main__.py", line 208, in main
    mod.main()
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2340, in main
    repoapps = prepare_apps(apps, apks, repodirs[0])
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2176, in prepare_apps
    copy_triple_t_store_metadata(apps_with_packages)
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 1076, in copy_triple_t_store_metadata
    _set_author_entry(app, 'authorWebSite', os.path.join(root, f))
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 784, in _set_author_entry
    with open(f, errors='replace') as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'build/player.efis.cfd/pfd/src/main/play/contact-website.txt'
```
2021-03-09 15:38:19 +01:00
fossdd
152250646e Revert removing Licenses 2021-03-09 15:38:19 +01:00
fossdd
eca7b23fc9 Update SPDX License List 2021-03-09 15:38:19 +01:00
Hans-Christoph Steiner
d94cc94ee0 gitlab-ci: arch job needs to build python-cryptography 2021-03-09 15:38:19 +01:00
Hans-Christoph Steiner
7aa2e3c359 gitlab-ci: use python3-cryptography to avoid building rust sources 2021-03-09 15:38:19 +01:00
Hans-Christoph Steiner
f95f88f64d gitlab-ci: use python3-cryptography to avoid building rust sources 2021-03-09 15:38:19 +01:00
Hans-Christoph Steiner
5ed531ce52 checkupdates: set User-Agent to make gitlab.com happy
closes #863
2021-03-09 14:57:04 +01:00
Jochen Sprickerhof
ba87076d64 Run push_binary_transparency only once 2021-03-09 14:52:45 +01:00
Hans-Christoph Steiner
fa186c40c4 version 2.0 2021-02-01 22:50:36 +01:00
Hans-Christoph Steiner
65ef6f33e3 make -C locale/ update 2021-02-01 22:50:03 +01:00
Golubev Alexander
e67cb8a1fa Translated using Weblate: Russian (ru) by Golubev Alexander <fatzer2@gmail.com>
Currently translated at 100.0% (576 of 576 strings)

Co-authored-by: Golubev Alexander <fatzer2@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/f-droid/fdroidserver/ru/
Translation: F-Droid/F-Droid Server
2021-02-01 22:44:33 +01:00
Peter J. Mello
58d23d17e6 Translated using Weblate: Portuguese (pt) by Peter J. Mello <admin@petermello.net>
Currently translated at 100.0% (576 of 576 strings)

Translated using Weblate: Portuguese (Portugal) (pt_PT) by Peter J. Mello <admin@petermello.net>

Currently translated at 100.0% (576 of 576 strings)

Co-authored-by: Peter J. Mello <admin@petermello.net>
Translate-URL: https://hosted.weblate.org/projects/f-droid/fdroidserver/pt/
Translate-URL: https://hosted.weblate.org/projects/f-droid/fdroidserver/pt_PT/
Translation: F-Droid/F-Droid Server
2021-02-01 22:44:33 +01:00
Eric
acba6b4a88 Translated using Weblate: Chinese (Simplified) (zh_Hans) by Eric <spice2wolf@gmail.com>
Currently translated at 34.7% (200 of 576 strings)

Co-authored-by: Eric <spice2wolf@gmail.com>
Translate-URL: https://hosted.weblate.org/projects/f-droid/fdroidserver/zh_Hans/
Translation: F-Droid/F-Droid Server
2021-02-01 22:44:33 +01:00
Oğuz Ersen
8fcbfd1403 Translated using Weblate: Turkish (tr) by Oğuz Ersen <oguzersen@protonmail.com>
Currently translated at 100.0% (576 of 576 strings)

Co-authored-by: Oğuz Ersen <oguzersen@protonmail.com>
Translate-URL: https://hosted.weblate.org/projects/f-droid/fdroidserver/tr/
Translation: F-Droid/F-Droid Server
2021-02-01 22:44:33 +01:00
WaldiS
81ef8055de Translated using Weblate: Polish (pl) by WaldiS <sto@tutanota.de>
Currently translated at 100.0% (576 of 576 strings)

Co-authored-by: WaldiS <sto@tutanota.de>
Translate-URL: https://hosted.weblate.org/projects/f-droid/fdroidserver/pl/
Translation: F-Droid/F-Droid Server
2021-02-01 22:44:33 +01:00
Ihor Hordiichuk
382fba151e Translated using Weblate: Ukrainian (uk) by Ihor Hordiichuk <igor_ck@outlook.com>
Currently translated at 100.0% (576 of 576 strings)

Translated using Weblate: Ukrainian (uk) by Ihor Hordiichuk <igor_ck@outlook.com>

Currently translated at 100.0% (576 of 576 strings)

Co-authored-by: Ihor Hordiichuk <igor_ck@outlook.com>
Translate-URL: https://hosted.weblate.org/projects/f-droid/fdroidserver/uk/
Translation: F-Droid/F-Droid Server
2021-02-01 22:44:33 +01:00
Hans-Christoph Steiner
31c29f2fb3 support official ANDROID_SDK_ROOT env var when running CLI tools
ANDROID_HOME is deprecated:
https://developer.android.com/studio/command-line/variables#android_sdk_root

!816 #547
2021-01-29 16:34:36 +01:00
Hans-Christoph Steiner
4900c426e9 locale: remove "obsolete" translations from .po files 2021-01-29 16:20:42 +01:00
Hans-Christoph Steiner
19e2ea8adf locale: remove duplicate strings as part of the update 2021-01-29 16:13:52 +01:00
Hans-Christoph Steiner
3e7c55d91d purge all references to dscanner 2021-01-29 15:40:59 +01:00
Hans-Christoph Steiner
63be6f6347 standardize naming in strings and comments on all caps APK 2021-01-29 15:40:59 +01:00
Hans-Christoph Steiner
97d5933a05 change config.py to config.yml everywhere it is needed 2021-01-29 13:24:17 +01:00