Commit graph

3677 commits

Author SHA1 Message Date
Hans-Christoph Steiner
f7dc89e9ba index: error if duplicate package files are in repo
Looks like the ~index-v2 work removed this error case, and the old bash
integration test case failed to catch it.  This reestablishes this error.

@mindston's refactoring of the old bash test suite caught this issue, e.g.
!1587
2025-02-12 18:38:05 +01:00
linsui
a7e56598d6 checkupdates: sort the order of dirs when finding autoname 2025-02-07 22:16:18 +08:00
linsui
2b725a5966 update.py: fix triple-t metadata extract when there is no subdir 2025-01-27 13:47:36 +00:00
Hans-Christoph Steiner
2ac925a249 ban apksigner v33, it has bugs verifying APKs with v3/v3.1 sigs 2025-01-21 15:00:23 +00:00
Hans-Christoph Steiner
5b1b1d12a1 verify: handle corrupt verified.json
verified.json can get quite large on verification.f-droid.org, and for some
unknown reason, it sometimes corrupts it when writing it out.  All the data
is already available in all the other JSON files, so this just automatically
reconstructs it.  Its a hack, but it took me much less time than I've
already spent trying to troubleshoot why it writes out corrupt verified.json.
2025-01-20 09:48:20 +00:00
Hans-Christoph Steiner
20caa6fa1c match the full file name when looking for the v1 signature block
ZipFile.namelist() produces a string per file.  The filename could contain
newline chars, including at the beginning and end.  ^$ in regex matches
around newline chars.  \A\Z matches the beginning/end of the full string.

This is exactly the same as obfusk's r'\AMETA-INF/(?s:.)*\.(DSA|EC|RSA)\Z'
but in a readable format that is also easily searchable, and standard for
this code base.

https://github.com/obfusk/fdroid-fakesigner-poc/blob/master/fdroidserver-regex.patch

#1251
2025-01-15 14:45:35 +01:00
linsui
f93e30f1e9 lint: only error out on missing extlib on versions not archived 2024-12-12 22:33:36 +08:00
linsui
9b59d5cf48 update: extract archive policy calculation to common 2024-12-12 22:20:38 +08:00
Hans-Christoph Steiner
5deb936e86 verify: --clean-up-verified to rm all files except the JSON report 2024-12-11 11:39:02 +00:00
Hans-Christoph Steiner
56814824ee new function get_src_tarball_name; deprecates getsrcname
Some places in the code that need this, like verify.py, do not have
app and build instances, but do have appid and versionCode.  And
fdroidserver/build.py is going away.
2024-12-11 11:39:02 +00:00
Hans-Christoph Steiner
bc118484e4 index.xml: XML escaping from Python < 3.13 which stopped converting "
index.xml is for old clients that are stuck in the past forever.  So the
format should not change at all.  Python 3.13 changed minidom so it no
longer converts " to an XML entity.
154477be72
2024-12-06 16:15:48 +01:00
Hans-Christoph Steiner
ad9b0c3294 update: only gen index.png when making index.html
* https://f-droid.org/repo does not use the index.png at all
* Some repos want to set their own index.png rather than the QR code
2024-12-04 13:13:30 +01:00
Hans-Christoph Steiner
8fc340aaca install: fix download dir when fetching from GitHub Releases 2024-11-26 15:31:47 +01:00
Hans-Christoph Steiner
8c81033ea3 FDroidPopenBytes: do not crash if options are not set
This makes writing test cases a lot easier. For example:

======================================================================
ERROR: test_devices (tests.test_install.InstallTest.test_devices)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/tests/test_install.py", line 31, in test_devices
    devices = fdroidserver.install.devices()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hans/code/fdroid/server/fdroidserver/install.py", line 225, in devices
    p = common.SdkToolsPopen(['adb', "devices"])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hans/code/fdroid/server/fdroidserver/common.py", line 2921, in SdkToolsPopen
    return FDroidPopen([abscmd] + commands[1:],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hans/code/fdroid/server/fdroidserver/common.py", line 3024, in FDroidPopen
    result = FDroidPopenBytes(commands, cwd, envs, output, stderr_to_stdout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hans/code/fdroid/server/fdroidserver/common.py", line 2987, in FDroidPopenBytes
    if output and options.verbose:
                  ^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'verbose'
2024-11-26 15:20:06 +01:00
Hans-Christoph Steiner
d2cc020336 install: fix broken URL building logic for Maven Central
The append logic was wrong, so it was trying to download URLs like:
https://repo.maven.apache.org/maven2/org/fdroid/fdroid/F-Droid/maven-metadata.xml/org/fdroid/fdroid/F-Droid/1.20.0/F-Droid-1.20.0.apk

Which should be:
https://repo.maven.apache.org/maven2/org/fdroid/fdroid/F-Droid/1.20.0/F-Droid-1.20.0.apk

Can be manually tested using:
`test_download_fdroid_apk=1 python -m unittest -k test_download_fdroid_apk_from_maven`
2024-11-26 15:14:18 +01:00
Hans-Christoph Steiner
a2d27ba15e
install: remove forgotten print() 2024-11-26 11:17:46 +01:00
Jochen Sprickerhof
8a5359ab3f scanner: replace deprecated imghdr with libmagic/puremagic
libmagic's Python bindings detect more types, puremagic is pure Python.

imghdr was dropped in Python 3.13.

This reverts commit 3bc246ccad.
2024-11-25 13:26:38 +01:00
Hans-Christoph Steiner
37dc6c121d run tests/refresh-SUSS_DEFAULT.py 2024-11-21 15:23:29 +01:00
Hans-Christoph Steiner
0ec9cd6921 checkupdates: only update app branches if metadata file changed 2024-11-21 12:15:14 +00:00
Hans-Christoph Steiner
e3f724681a checkupdates: parse default branch from upstream remote 2024-11-21 12:15:14 +00:00
Hans-Christoph Steiner
fd15ac9276 checkupdates: mark as Draft when only changing Current Version
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/1551#note_2190155816
2024-11-21 12:15:14 +00:00
Hans-Christoph Steiner
c97503b5f3 checkupdates: get default branch from git config 2024-11-21 12:15:14 +00:00
Hans-Christoph Steiner
fbe9152ee5 checkupdates: commit summary is merge request title 2024-11-21 12:15:14 +00:00
Hans-Christoph Steiner
cd8d4ef88b checkupdates: reuse per-app branches when making merge requests
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/1551#note_2206085258
2024-11-21 12:15:14 +00:00
Hans-Christoph Steiner
20ff302e89 checkupdates: remove duplicate push in push_commits() 2024-11-21 12:15:14 +00:00
Hans-Christoph Steiner
17c480d299 checkupdates: make iter_commits only include commits in appid branch
iter_commits() follows `git rev-list` (which selects different commits
than `git diff`).  With ... notation, `git rev-list` will return all
the commits that are not shared by the two branches.  This needs only
the commits in the right side of the comparison (like how `git diff`
does it).
2024-11-21 12:15:14 +00:00
linsui
56d5f360ed scanner: disallow wasm file 2024-11-20 19:10:30 +08:00
Hans-Christoph Steiner
4d6682bc70 update: fix _strip_and_copy_image to work with pathlib 2024-11-19 20:27:29 +01:00
Hans-Christoph Steiner
290587f449 add ColorFormatter class for optional colorized log output 2024-11-19 13:04:23 +00:00
Hans-Christoph Steiner
7e8e2403cd black formatting 2024-11-19 13:04:23 +00:00
relan
118c7fc783 Remove "hg purge" after "hg update"
1. "hg purge" works only on interactive shell (see #1015).
2. No need to clean anything because we did this before "hg update".
2024-11-18 21:12:04 +00:00
relan
87909fae74 Remove ignored files on gotorevisionx for hg
Other revisions can change the list of ignored files and if we don't
remove them, switching to another revision can fail.
2024-11-18 21:12:04 +00:00
Hans-Christoph Steiner
05e091804d install: verify sig for all downloaded APKs if apksigner is installed 2024-11-08 17:12:42 +02:00
Hans-Christoph Steiner
addb7b9acc install: echo characters that the user inputs at the prompt 2024-11-08 17:12:42 +02:00
Hans-Christoph Steiner
b9b4ca9778 install: download any app from f-droid.org 2024-11-08 17:12:42 +02:00
Licaon_Kter
96ed60fda3 Reword extra blocks 2024-11-08 15:09:05 +00:00
Hans-Christoph Steiner
27e3b5066a install: add --no-privacy-mode as an argument 2024-11-01 00:34:03 +01:00
Hans-Christoph Steiner
4d22a7f67f install: Y/n prompt, --yes, --no controls downloading F-Droid.apk 2024-11-01 00:34:03 +01:00
Hans-Christoph Steiner
560472e4e5 install: download from GitHub Releases 2024-11-01 00:16:13 +01:00
Hans-Christoph Steiner
1eb6516f16 install: add IPNS download method 2024-11-01 00:09:58 +01:00
Hans-Christoph Steiner
3da48e64bc install: use Maven Central as additional source 2024-11-01 00:09:58 +01:00
Hans-Christoph Steiner
b77eba824b install: convert to common.get_config() 2024-11-01 00:09:54 +01:00
Hans-Christoph Steiner
3a3417f6f8 install: add --privacy-mode flag 2024-10-31 23:49:34 +01:00
Hans-Christoph Steiner
c7bc8d0fea install: function to fetch, verify and install the F-Droid.apk 2024-10-31 23:49:30 +01:00
Hans-Christoph Steiner
681d705da0 install: reliable algorithm for picking devices from adb output
Versions of this algorithm are used elsewhere:
* https://github.com/openatx/adbutils/blob/master/adbutils/_adb.py
2024-10-31 21:55:20 +01:00
Hans-Christoph Steiner
59fcfa5dec index: download_repo_index_v2() uses mirrors
test_download_repo_index_v2_url_parsing is no longer needed, since all the
things it tested are now handled in test_download_repo_index_v2
2024-10-31 21:55:18 +01:00
Hans-Christoph Steiner
2e3f6d273a net: download_using_mirrors() to download like fdroidclient does 2024-10-31 21:54:47 +01:00
Hans-Christoph Steiner
49dcc53076 install: download_fdroid_apk() to fetch the recommended initial APK 2024-10-31 21:54:47 +01:00
Hans-Christoph Steiner
56bed02a29 install: download_apk() fetchs APKs by appid based on the index 2024-10-31 21:54:45 +01:00
Hans-Christoph Steiner
ab28c442e2 KnownApks: remove dead code 2024-10-31 09:20:26 +00:00