One key security property of the F-Droid ecosystem is that the sensitive
code is all stored forever in git repos and source tarballs. That means
we can easily go back and see if there where exploits and where they came
from. Therefore, checkupdates should require everything in fdroiddata be
committed to git before running.
This provides --allow-dirty to override that behavior.
"SVN follows HTTP 301 redirects to svn+ssh:// URLs. As a result, an
innocent looking HTTP URL can be used to trigger a Command Execution with a
301 redirect."
https://blog.recurity-labs.com/2017-08-10/scm-vulns.html#third-round-svn-and-mercurial
I scanned fdroiddata and found no suspicious redirects. Here's how:
grep -A1 '^Repo *Type: *git-svn' *.txt *.yml| sed -n 's,.*Repo:\(.*\),\1,p' > /tmp/urls.txt
import requests
with open('/tmp/urls.txt') as fp:
for line in fp:
try:
r = requests.head(line.strip())
print(r.status_code, line)
except requests.exceptions.SSLError:
print('SSLError', line)
For androguard, @thezero already developed a way to get all the icons after
only extracting the icon name. So this uses that for the aapt-based scans
also, to make them less brittle.
This should fix the problem where `fdroid update` was choosing the XML icon
for apps that include one, like NewPipe.
closesfdroid/fdroid-website#192
This should have less of a change of matching bad things.
thanks to @stf for the report. I ran tests comparing the original vs these
new patterns, and it was a 100% match. So at least it didn't make things
worse.
Here's the test script:
#!/usr/bin/env python3
import os
import re
old_vcsearch_g = re.compile(r'''.*[Vv]ersionCode[ =]+["']*([0-9]+)["']*''').search
old_vnsearch_g = re.compile(r'.*[Vv]ersionName *=* *(["\'])((?:(?=(\\?))\3.)*?)\1.*').search
old_psearch_g = re.compile(r'.*(packageName|applicationId) *=* *["\']([^"]+)["\'].*').search
new_vcsearch_g = re.compile(r'''.*[Vv]ersionCode\s*=?\s*["']*([0-9]+)["']*''').search
new_vnsearch_g = re.compile(r'''.*[Vv]ersionName\s*=?\s*(["'])((?:(?=(\\?))\3.)*?)\1.*''').search
new_psearch_g = re.compile(r'''.*(packageName|applicationId)\s*=*\s*["']([^"']+)["'].*''').search
old = re.compile(r'.*(packageName|applicationId) *=* *["\']([^"]+)["\'].*').search
new = re.compile(r'''.*(packageName|applicationId)\s*=*\s*["']([^"']+)["'].*''').search
for root, dirs, files in os.walk('build'):
for f in files:
if f.endswith('.gradle'):
with open(os.path.join(root, f)) as fp:
for line in fp:
for old, new in ((old_vcsearch_g, new_vcsearch_g),
(old_vnsearch_g, new_vnsearch_g),
(old_psearch_g, new_psearch_g)):
found_old = old(line)
found_new = new(line)
oldresult = None
newresult = None
if found_old or found_new:
if found_old:
oldresult = found_old.groups()
#print('OLD', oldresult)
if found_new:
newresult = found_new.groups()
#print('NEW', newresult)
if oldresult != newresult:
print('--------------------------------')
print(f, oldresult, newresult)
In order to test that aapt defaults minSdkVersion to 3, I ran this script
then compared the output with meld:
cd $ANDROID_HOME/build-tools
for d in *.*; do echo $d; $ANDROID_HOME/build-tools/$d/aapt dump badging /home/hans/code/fdroid/server/tests/repo/com.politedroid_3.apk > /tmp/${d}.txt; done
meld /tmp/17.0.0.txt /tmp/26.0.2.txt /tmp/27.0.3.txt
Apps can now use an XML icon, but if the app supports older Android
versions, it'll also contain PNG versions of the same icon. This finds
those PNGs and uses them instead.
#344closes#392
fdroiddata#913
git-svn will put up the "Reject/Accept" prompt if it encounters a bad HTTPS
certificate. I could find no way to stop it from doing that. So instead,
this checks the HTTPS connection with an HTTP HEAD request first.
Subversion does not verify each commit as strongly as git does, so HTTPS is
really important. Also, there is the possibility of injecting code into
`fdroid checkupdate` calls if plain HTTP is used.
This uses both the env vars and the command line options to ensure
that it works with as many versions of git as possible. Also, git-svn
uses the env vars, but not necessarily the command line options.
This uses /bin/true to pretend that it succesfully got the password.
If password auth is truly required, then it will fail further on down
the line.
Currently f-droid.org has a lot of pending builds and big delays
between "fdroid build --all" runs. Bump overall build time limit from
12 hours to 36 hours to increase buildserver throughput.
We always want to run all utilities non-interactively. By default
subprocess.Popen() inherits stdin descriptor from parent process, i.e.
when fdroid is run from an interactive shell, subprocesses may expect
input from it.
Reading from /dev/null immediately returns EOF, failing any user prompt
and preventing us from hang.
