There are all sorts of unfiltered user inputs like tag and branch names in
source repos. If those names are fed into popen calls that use shell=True,
that opens up a wide range of exploits. All core operations should never
use shell=True.
This is a quick and very incomplete addition of '--' to command line calls
to source VCSs like git and hg that could manipulated by malicious
tag/branch names or other vectors.
These were all manually tested by calling the command lines on my own
machine.
This lets `fdroid scanner my.package.name` run without requiring that the
versionCode is also specified. It also allows scanner.scan_source() to be
called as a function in the public API of fdroidserver.
This code has never been used and contains some insecure uses of shell=True
Building Kivy apps should be done with the buildozer=yes method. The
buildozer method should probably be moved to a provisioner once that is in
place.
The currently included Qt has known security issues and is outdated. This
can now be replaced by downloading and installing the Qt installer using
the sudo= build field. @relan's provisioner system will also replace this
once that's done. There are only two apps that currently use the Qt stuff:
* csd.qtproject.minesweeper
* org.openorienteering.mapper
Only start new builds for 12 hours. This ensures we publish new builds
often enough even on long backlogs.
This could be made configurable at a later point.
GitLab storage provides two mirrors by default:
* https://gitlab.com/user/repo/raw/master/fdroid/repo
* https://user.gitlab.io/repo/fdroid/repo
While the F-Droid client will happily fetch the index*.jar files and
parse them from either of these two mirrors, only the GitLab Pages
mirror will serve files with the correct mime type. Many repos
tend to put index.html files (and associated .css/.js/image files) in
the root of a repository to provide information about that repo.
One example is RepoMaker. The way in which RepoMaker decides the public
URL of a repo, is to take the first mirror in the list. This means that
the URL which RepoMaker directs people to for GitLab storage returns a
.html document in text/plain, which means that it is not rendered.
We could change RepoMaker so that it takes the last mirror, and then it
woruld work. However there is something nice about the first mirror in a
list being the most authoritative (even though the mirror order doesn't
- and perhaps shouldn't have any specific meaning).
ERROR: Could not build app org.fdroid.fdroid due to unknown error: Traceback (most recent call last):
File "/var/lib/jenkins/userContent/reproducible/reproducible_setup_fdroid_build_environment/fdroidserver/build.py", line 1202, in main
options.onserver, options.refresh):
File "/var/lib/jenkins/userContent/reproducible/reproducible_setup_fdroid_build_environment/fdroidserver/build.py", line 972, in trybuild
build_server(app, build, vcs, build_dir, output_dir, log_dir, force)
File "/var/lib/jenkins/userContent/reproducible/reproducible_setup_fdroid_build_environment/fdroidserver/build.py", line 82, in build_server
logging.debug(_('Fetched buildserverid from VM: ') + buildserverid)
TypeError: Can't convert 'bytes' object to str implicitly
We remove the whole "build" directory while cleaning source code tree
because Gradle can leave there files even after "gradle clean". But some
projects (Mozilla Fennec) actually have useful stuff checked into VCS
under the "build" directory.
Remove only those subdirectories that we known for sure are leftovers
from Gradle.
Fixesfdroid/fdroidserver#438.
When `fdroid build` is run using the buildserver, it should fetch the
buildserverid on the first build.
Seems this was really a silly bug in 837fc99d74
Since `fdroid build --all` can run a long time, knowing when that command
was started will be very useful information for figuring out what the build
server is doing.
GitHub only allows an SSH key to be used as a Deploy Key for a single repo.
That means, each nightly build repo on GitHub/Travis must have its own
debug keystore.
This needed now because the buildserver is hanging so often, that we are
often going a week or more without any builds published. Perhaps this is
only temporary, or maybe we will want to flush this feature out more as a
standard thing. But we really need it for now to at least get some builds
out on a daily basis.
Since the website deploy is also triggered by this cycle, making the build
finish more often means the website will be published more often.
Fixes bb758d3f, spotted by @bubu:
DEBUG: buildserver > DEBUG: > sudo apt-get -y purge sudo
DEBUG: buildserver > Reading package lists...
DEBUG: buildserver > Building dependency tree...
DEBUG: buildserver > Reading state information...
DEBUG: buildserver > The following packages will be REMOVED:
DEBUG: buildserver > sudo*
DEBUG: buildserver > 0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
DEBUG: buildserver > After this operation, 2,391 kB disk space will be freed.
(Reading database ... 68491 files and directories currently installed.)
DEBUG: buildserver > Removing sudo (1.8.10p3-1+deb8u4) ...
DEBUG: buildserver > You have asked that the sudo package be removed,
DEBUG: buildserver > but no root password has been set.
DEBUG: buildserver > Without sudo, you may not be able to gain administrative privileges.
DEBUG: buildserver >
DEBUG: buildserver > If you would prefer to access the root account with su(1)
DEBUG: buildserver > or by logging in directly,
DEBUG: buildserver > you must set a root password with "sudo passwd".
DEBUG: buildserver >
DEBUG: buildserver > If you have arranged other means to access the root account,
DEBUG: buildserver > and you are sure this is what you want,
DEBUG: buildserver > you may bypass this check by setting an environment variable
DEBUG: buildserver > (export SUDO_FORCE_REMOVE=yes).
DEBUG: buildserver >
DEBUG: buildserver > Refusing to remove sudo.
DEBUG: buildserver > dpkg: error processing package sudo (--purge):
DEBUG: buildserver > subprocess installed pre-removal script returned error exit status 1
DEBUG: buildserver > Errors were encountered while processing:
DEBUG: buildserver > sudo
DEBUG: buildserver > E: Sub-process /usr/bin/dpkg returned an error code (1)
