Commit graph

3677 commits

Author SHA1 Message Date
Michael Pöhn
0fa88c5c20
💇 implement review nits
Implement review nits as requested bei @eighthave in
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/1471
2024-06-27 12:28:54 +02:00
Michael Pöhn
3cf3685280
🐐 appeace bandint security warnings
In this particular case bandit was complaining about potential path
escape exploits on urlopen. However the urls are safe enough, because
all template parameters inserted into the url are from:

 * config.yml - malicious changes to config.yml are possible that's
   already a lot bigger issue than this than redirecting github api
   calls.
 * git tags witch are present in bot the local index-v2.json file (as
   versionName) and the remote github API. (git tags don't allow the
   string '..')
2024-06-27 12:28:41 +02:00
Michael Pöhn
aca98c1355
🏏 incorporate review feedback 2024-06-27 12:28:29 +02:00
Michael Pöhn
a87284cf80
🚥 deploy: tests for github releases deploy functions
Add test cases for github releases function in deploy.py
2024-06-27 12:28:16 +02:00
Michael Pöhn
242490ddc3
🎯 deploy: no releaseChannels on github releases
Don't deploy versions of to GitHub releases where a `releaseChannels`
value is set in index-v2.json. (This usually would mean it's a alpha or
beta version.)
2024-06-27 12:28:04 +02:00
Michael Pöhn
c6598f2835
☄️ deploy: github releases - whatsNew text as note
Use whatsNew text (if available) as release notes text when deploying to
Github releases. This feature will always use 'en-US' locale texts,
since English is the lingua franka on GitHub. Additionally this change
also adds a config option to preprend a static text to those release
notes.
2024-06-27 12:27:51 +02:00
Michael Pöhn
a079f9d85f
♟️ deploy: add global config github_token 2024-06-27 12:27:39 +02:00
Michael Pöhn
7a6fa7f816
🛄 deploy: separate function for release file lookup 2024-06-27 12:27:27 +02:00
Michael Pöhn
44b0af933d
🎡 add unit tests for github.py
add unittests for our github api calls
2024-06-27 12:27:15 +02:00
Michael Pöhn
1b19293ab0
🛰️ deploy: github releases
Implemented basic support for using `fdroid delpoy` to upload APKs from
the repo to GitHub releases.
2024-06-27 12:27:01 +02:00
pmmayero
8255dec6b8 fix: jinja 2 CVE 2024-06-12 06:30:27 +00:00
pmmayero
3f59d0908f fix: subprocess.check_output to produce string and not bytes 2024-06-12 06:30:27 +00:00
proletarius101
845ed3ef47 Add index only deployment mode 2024-06-11 17:45:59 +00:00
paul mayero
7aabfbcbf0 Adding rclone as an option to fdroid deploy 2024-05-29 14:08:07 +00:00
Licaon_Kter
df97ebe052 scanner - pickup by id, sort list 2024-05-22 09:25:49 +00:00
Hans-Christoph Steiner
18f3acc32e split out options from read_config()
There is no longer any reason for these to be intertwined.

This deliberately avoids touching some files as much as possible because
they are super tangled and due to be replaced.  Those files are:

* fdroidserver/build.py
* fdroidserver/update.py

# Conflicts:
#	tests/testcommon.py

# Conflicts:
#	fdroidserver/btlog.py
#	fdroidserver/import_subcommand.py
2024-05-08 16:26:46 +02:00
Hans-Christoph Steiner
1e5699e90c remove all references to optparse (deprecated since Python 3.2) 2024-05-08 16:19:27 +02:00
Hans-Christoph Steiner
717df09be0 clarify that config/options can be global or module-level variable 2024-05-08 16:19:27 +02:00
Hans-Christoph Steiner
92a3f4b191 rename local variable to stop overwriting global options
This fixes a bug where if smartcardoptions is set as a str in config.yml
will overwrite all command line options.

a4d069862
fdroidserver!1106
2024-05-08 16:19:27 +02:00
Hans-Christoph Steiner
1eaba25021 common: do not use module reference for local functions
This just makes things more confusing.
2024-05-08 16:19:27 +02:00
Gregor Düster
dbdefe200c Format files with ruff 2024-05-08 15:41:51 +02:00
Gregor Düster
df27405d8b [build] Add more docstrings 2024-05-08 15:35:28 +02:00
Gregor Düster
1c70758940 [btlog] Add more docstrings 2024-05-08 15:31:43 +02:00
Gregor Düster
4109e8fb03 [checkupdates] Add module docstring 2024-05-08 15:31:38 +02:00
Gregor Düster
97346a2cba [nightly] Add more docstrings 2024-05-08 15:31:32 +02:00
Gregor Düster
6c27fec94b [import] Add more docstrings 2024-05-08 15:31:29 +02:00
Hans-Christoph Steiner
9a327b5097 reliable implementation of get_first_signer_certificate()
This keeps key pieces of @linsui's algorithm, specifically the check
that all certificates are the same.  apksigner also does this check.

closes #1128
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
a8fd360a88 skip AndroidManifest.xml and resources when fetching v2+ certs 2024-05-07 16:22:59 +02:00
FC (Fay) Stegerman
6f5fd2b132 PoC + writeup + patch
6c6dc25112/fdroidserver.patch (L28)

https://github.com/androguard/androguard/issues/1030
refs #1128

(this is an excerpt of the original patch)
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
312f822764 androguard is required, stop using use_androguard() 2024-05-07 16:22:59 +02:00
linsui
2fea71a6c7 get_first_signer_certificate: check all v1 v2 and v3 certs 2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
93f361c623 replace decade old pyasn1 crypto impl with working asn1crypto
For some APKs, get_certificate() was returning a different result than
apksigner and keytool.  So I just took the algorithm from androguard, which
uses asn1crypto instead of pyasn1.  So that removes a dependency as well.
asn1crypto is already required by androguard.

The original get_certificate() came from 6e2d0a9e1
2024-05-07 16:22:59 +02:00
linsui
7104411296 throw error if gradle build method is used but no build.gradle file is found 2024-05-07 14:13:47 +00:00
linsui
d243cbd030 lint: blocklist known AOSP debug keys in AASK 2024-05-07 11:33:04 +00:00
linsui
f82d648cb1 deploy: retry when git push fails 2024-05-05 20:53:35 +00:00
Hans-Christoph Steiner
6f07538cdc mirror: apparently the 'icons' dir does not always exist
https://apt.izzysoft.de/fdroid/archive/com.futsch1.medtimer_28.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.hardbacknutter.nevertoomanybooks_75.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.jovial.jrpn15_20.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.jovial.jrpn2_20.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.logger.app_8.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.lorenzovainigli.foodexpirationdates.foss_22.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.openathena_36.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.oppzippy.openscq30_21.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.sanmer.mrepo_713.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.starry.myne_325.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/dev.imranr.obtainium_22602.apk:
2024-04-21 18:55:22 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/dev.sanmer.pi_137.apk:
2024-04-21 18:55:22 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/org.breezyweather_50108.apk:
2024-04-21 18:55:22 ERROR 404: Not Found.
2024-04-21 18:55:22,448 ERROR: no "icon" in org.breezyweather
2024-04-21 18:55:22,449 ERROR: no "icon" in com.lorenzovainigli.foodexpirationdates.foss
2024-04-21 18:55:22,449 ERROR: no "icon" in com.jovial.jrpn15
2024-04-21 18:55:22,450 ERROR: no "icon" in com.jovial.jrpn2
2024-04-21 18:55:22,450 ERROR: no "icon" in com.logger.app
2024-04-21 18:55:22,450 ERROR: no "icon" in com.futsch1.medtimer
2024-04-21 18:55:22,451 ERROR: no "icon" in com.sanmer.mrepo
2024-04-21 18:55:22,451 ERROR: no "icon" in com.starry.myne
2024-04-21 18:55:22,451 ERROR: no "icon" in com.hardbacknutter.nevertoomanybooks
2024-04-21 18:55:22,452 ERROR: no "icon" in dev.imranr.obtainium
2024-04-21 18:55:22,452 ERROR: no "icon" in com.openathena
2024-04-21 18:55:22,453 ERROR: no "icon" in com.oppzippy.openscq30
2024-04-21 18:55:22,453 ERROR: no "icon" in dev.sanmer.pi
2024-04-21 18:55:22,454 CRITICAL: Unknown exception found!
Traceback (most recent call last):
  File "/home/fdroid/code/fdroid/fdroidserver/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/home/fdroid/code/fdroid/fdroidserver/fdroidserver/__main__.py", line 230, in main
    raise e
  File "/home/fdroid/code/fdroid/fdroidserver/fdroidserver/__main__.py", line 211, in main
    mod.main()
  File "/home/fdroid/code/fdroid/fdroidserver/fdroidserver/mirror.py", line 269, in main
    _run_wget(os.path.join(basedir, section, icondir), urls[icondir])
KeyError: 'icons'
2024-05-02 18:29:42 +02:00
Hans-Christoph Steiner
be59b38ac1 update: handle ValueError from apkInspector in androguard 4.1
androguard 4.1 uses a new lib called apkInspector instead of zipfile.ZipFile
so that it can handle usable but invalid ZIP files.  It will also throw
ValueError on some things, for example:

Traceback (most recent call last):
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/tests/update.TestCase", line 878, in test_scan_apk_bad_zip
    fdroidserver.update.scan_apk(apkfile)
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/fdroidserver/update.py", line 1586, in scan_apk
    scan_apk_androguard(apk, apk_file)
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/fdroidserver/update.py", line 1725, in scan_apk_androguard
    apkobject = common.get_androguard_APK(apkfile)
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/fdroidserver/common.py", line 2673, in get_androguard_APK
    return APK(apkfile)
  File "/usr/local/lib/python3.10/dist-packages/androguard/core/apk/__init__.py", line 273, in __init__
    self.zip = ZipEntry.parse(filename, False)
  File "/usr/local/lib/python3.10/dist-packages/apkInspector/headers.py", line 410, in parse
    eocd = EndOfCentralDirectoryRecord.parse(apk_file)
  File "/usr/local/lib/python3.10/dist-packages/apkInspector/headers.py", line 59, in parse
    raise ValueError("End of central directory record (EOCD) signature not found")
ValueError: End of central directory record (EOCD) signature not found
2024-04-25 13:00:23 +02:00
Hans-Christoph Steiner
ef4ec74882 some parts of androguard 4.x use loguru instead of logging 2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
1c84f63247 replace deprecated get_element() which was removed in 4.x
/usr/lib/python3/dist-packages/androguard/core/bytecodes/apk.py:884: DeprecationWarning: This method is deprecated since 3.3.5.

It was added in 3.3.5.  Debian/bullseye and Ubuntu/20.04/focal both include
new enough versions.  Debian/buster's is too old (3.3.3).
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
5b7abc0423 single function to tame androguard's verbose default output
# Conflicts:
#	fdroidserver/common.py
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
7a144a4762 port to androguard >= 4 and drop support for older than 3.3.3
This also makes androguard a hard requirement, which has been true for a
while anyway.  So the code that handles androguard as an optional
requirement is removed.  androguard from Debian/buster is new enough, so
this does not seem like it will cause any problems.
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
cdc7c98707 common.get_androguard_APK() is no longer private to the module 2024-04-25 12:39:12 +02:00
Michael Pöhn
005a33732c apply review suggestions 2024-04-24 11:56:07 +00:00
Michael Pöhn
bd6afa4365
🐛 index fix: skip altstore when no IPAs present 2024-04-24 12:45:29 +02:00
Hans-Christoph Steiner
79f148167a deploy: automatically convert to black format 2024-04-24 08:42:15 +00:00
Hans-Christoph Steiner
cb04d801d8 deploy: manually move hard cases to black code format 2024-04-24 08:42:15 +00:00
Hans-Christoph Steiner
9716b5e1ab index: manual black format 2024-04-24 10:29:50 +02:00
Michael Pöhn
d00a87ed6c
🏏 alt-store index: incorporate review feedback 2024-04-23 17:28:30 +02:00
Michael Pöhn
45efb88f85
🕴️ add test for make_altstore 2024-04-23 16:05:58 +02:00
Michael Pöhn
f742799a9d
🏟️ add test for _parse_from_pbxproj
Also fix lint issues
2024-04-23 16:05:34 +02:00