Commit Graph

204 Commits (master)

Author SHA1 Message Date
Simon McVittie 48761e778a bzr: Run brz in preference to bzr
Bazaar-NG (bzr) relies on Python 2, which has reached end-of-life, and
appears to be essentially unmaintained itself. Try using Breezy (brz),
a "friendly fork" of bzr that has been ported to Python 3 and is
maintained.

Signed-off-by: Simon McVittie <smcv@debian.org>
2020-03-20 20:19:26 +00:00
Dan Nicholson aec06b3945 Support passing --token-type to build-export
Provide a --token-type command line option and a token-type manifest
property that allows passing the token type to build-export. The
manifest property takes precendence over the CLI option. In either case,
the value will be passed to build-export if it's >= 0. This requires
flatpak >= 1.6 to use the --token-type option in build-export.
2020-03-19 16:29:27 +01:00
Bastien Nocera 3fda50c199 source archive: Add 7zip support 2020-03-19 16:27:30 +01:00
Bastien Nocera 38b6c80a3b archive source: Allow overriding the archive type
This allows unpacking files that would otherwise not be recognised as of
a particular type, such as self-extracting files supported by zip, 7z or
cabextract.
2020-03-19 16:27:30 +01:00
Patrick 99d1d511b6
Quote checksum in mismatch error for clarity
Closes #321
2019-11-14 04:19:34 -08:00
Alexander Larsson 8cd9401619 Support duplicate module names
If two modules happen to have the same name we uniquify them by
appending "-$n". While you shouldn't normally rely on this it is
sometimes hard to avoid when you're including some json snippet you
don't have control over.

Closes: #308
Approved by: alexlarsson
2019-09-13 13:25:32 +00:00
Alexander Larsson 3a3ca4d5a0 module: Add builder_module_set_name()
Closes: #308
Approved by: alexlarsson
2019-09-13 13:25:32 +00:00
Alexander Larsson aab7bcd5e9 Support --show-manifest
We want to use this in flathub to avoid all the problems we've having
parsing json manifests via the python parser, which isn't *quite* compatible
with json-glib (differs in comment and multiline string support for instance).

Closes: #307
Approved by: alexlarsson
2019-09-13 12:55:18 +00:00
Linus Jahn 28aaaaa55e source-git: Add option to disable submodules
Closes: #295
Approved by: alexlarsson
2019-09-13 10:51:18 +00:00
Olaf Leidinger 75c24ce291 Feature: Use CCACHE_DIR from environment, if set
Closes: #288
Approved by: alexlarsson
2019-09-13 10:40:24 +00:00
Ryan Gonzalez 788ffbb630 Minor fix for YAML number warning
This will properly make sure numbers are identified without giving a false positive for ".".

Closes: #298
Approved by: alexlarsson
2019-09-13 10:30:46 +00:00
Ryan Gonzalez b317081048 Support parsing sources files as yaml
Fixes #297.

Closes: #298
Approved by: alexlarsson
2019-09-13 10:30:46 +00:00
Matthew Leeds 9baa2134e7 Use flatpak_spawnv() more
This means when flatpak-builder runs a flatpak command in a subprocess,
we can see the arguments passed to flatpak in the flatpak-builder
output, if -v was used.

Closes: #291
Approved by: alexlarsson
2019-06-28 08:55:11 +00:00
Seppo Yli-Olli cd1923a776 Make error handling more understandable
Closes: #284
Approved by: alexlarsson
2019-06-10 06:30:10 +00:00
Seppo Yli-Olli 30581bf63a Support multiple instances of install-deps-from
Closes: #284
Approved by: alexlarsson
2019-06-10 06:30:10 +00:00
Michael Gratton 068a598025 Ensure shallow repo mirror is updated when the git ref changes
Shallow clones only contain the ref that was cloned, and so if a
mirror is shallow and the ref is changed in a subsequent invocation of
builder, the new ref is absent and git fails with an error.

This patch avoids bailing out too soon when mirroring a repo if the
requested ref does not exist, to ensure it is fetched.

Fixes #285

Closes: #286
Approved by: alexlarsson
2019-05-14 10:56:23 +00:00
Christian Hergert d3399fd785 utils: force exit host commands when flatpak-builder exits
If we are spawning applications on the host using the Development service,
then we want those commands to exit when the flatpak-builder process
exits, as can happen from Ctrl^C or kill().

By using a static FlatpakHostCommandFlags we only ever check this a single
time and then each subsequent request will do the right thing.
2019-05-06 13:03:17 +02:00
Alexander Larsson 15eb895e00 Update platform creation to ensure good mtimes and better cacheing
This splits the platform creation into 3 parts:
 * base - create the initial directory based on the parent platform
 * prepare - run prepare commands and apply all changes
 * cleanup - apply cleanups and cleanup commands

This has cacheing advantages in that prepare_commands and cleanup changes
only cause the minimal amount of rebuilds.

Additionally, it ensures that the mtimes are zeroed out (from the
previous checkout) both when the prepare and cleanup commands are run.
This is actually important, since these often generate caches (for
example fontconfig ones) which rely on zeroed mtimes so they match
what will be deployed.

Closes: #277
Approved by: alexlarsson
2019-04-04 08:48:18 +00:00
Alexander Larsson acec80a8db install-deps: Pass --noninteractive for flatpak >= 1.2.0
Closes: #274
Approved by: alexlarsson
2019-03-05 07:13:34 +00:00
Alexander Larsson 2d938e2ccd Move flatpak_version_check to builder-utils.c
Closes: #274
Approved by: alexlarsson
2019-03-05 07:13:34 +00:00
Alexander Larsson c7f8a50f7b Fix handling of sdk default compiler options
It turns out that newer flatpaks changed the output of flatpak info
and we relied on parsing that due to the lack of --show-location option
to flatpak info. However, that has been added since flatpak 0.11.8, so
just use that instead of parsing it.

Closes: #270
Approved by: alexlarsson
2019-02-08 10:21:47 +00:00
Alexander Larsson 8839ff08b1 Fixup error messages
We were printing argv[3] which is wrong (and typically null).
Instead set the error message where it happens and we know
what we were trying to do.

Closes: #269
Approved by: alexlarsson
2019-02-06 09:04:44 +00:00
Mathieu Bridon 416b7f599b Remove builder_git_checkout_dir
It is completely unused now that we checkout the whole tree in the
from-git code path.

Closes: #260
Approved by: alexlarsson
2019-02-05 14:58:27 +00:00
Mathieu Bridon 7d318128b6 from-git: Check out the whole tree instead of just the manifest
Some manifests now make use of the shared-modules Git repo from Flathub.
At least Totem does that, and we might see others do the same.

    https://gitlab.gnome.org/GNOME/totem/blob/master/flatpak/org.gnome.Totem.json
    http://sdkbuilder.gnome.org/logs/build-2019-01-18-141001/build-gnome-apps-nightly-master-org.gnome.Totem-x86_64.txt

That means we have to checkout the whole tree, otherwise we can't parse
the manifest.

Closes: #260
Approved by: alexlarsson
2019-02-05 14:58:27 +00:00
Mathieu Bridon dec058da72 from-git: Fetch the submodules as well
Some manifests now make use of the shared-modules Git repo from Flathub.
At least Totem does that, and we might see others do the same.

    https://gitlab.gnome.org/GNOME/totem/blob/master/flatpak/org.gnome.Totem.json
    http://sdkbuilder.gnome.org/logs/build-2019-01-18-141001/build-gnome-apps-nightly-master-org.gnome.Totem-x86_64.txt

That means we have to fetch the Git submodules before we can parse the
manifest.

Closes: #260
Approved by: alexlarsson
2019-02-05 14:58:27 +00:00
Alexander Larsson ab72117fdd Add --add/remove-tag options
We'd like to use these in flathub.

Closes: #268
Approved by: alexlarsson
2019-02-04 10:02:33 +00:00
Mathieu Velten 21356f8616 manifest: use the sdk branch from the sdk ref specified by the user
install_extension_deps now takes the sdk branch as param as a side effect

Closes: #266
Approved by: alexlarsson
2019-02-04 09:52:04 +00:00
Alexander Larsson 00f63cfe7e manifest: Add default-branch key
This is similar to branch and used if branch is not set. However, this
key (as opposed to branch) is overridden by the --default-branch option.

The idea is that most apps in flathub would use default-branch=stable, so
that a standart flatpak-builder command will build a "stable" release.
However, when building a test build we can use --default-branch=test to
override that. Then special cases like theme extensions and other things
that require a specific branch value to work at all can use branch="1.0".

Closes: #264
Approved by: alexlarsson
2019-01-26 03:16:27 +00:00
Alexander Larsson 09b2736693 Compress changes when stored in cache commit metadata
The libreoffice cache failed because > 10mb commit object...

Closes: #263
Approved by: alexlarsson
2019-01-25 14:54:54 +00:00
Alexander Larsson ed4c440063 utils: Add flatpak_variant_[un]compress helpers
Closes: #263
Approved by: alexlarsson
2019-01-25 14:54:54 +00:00
Alexander Larsson 1b546771c5 Allow overriding SOURCE_DATE_EPOCH via env option
We apply this variable before the env options so we can clear this.

Closes: #257
Approved by: alexlarsson
2019-01-14 10:31:42 +00:00
Alexander Larsson 4d2ff1a196 build-options: Allow null values to unset env vars
Closes: #257
Approved by: alexlarsson
2019-01-14 10:31:42 +00:00
Alexander Larsson feb839944f yaml: Allow null nodes, for example as dict values
We want this for overriding env vars. If you need a "null" string you
will have to quote it.

Closes: #257
Approved by: alexlarsson
2019-01-14 10:31:42 +00:00
Alexander Larsson eb41d1dd01 post-process: Handle python 3.7 header formats
Python 3.7 adds a flags field which modified the offset of the mtime,
as well as adds a new non-mtime based check mode.

Fixes #247

Closes: #256
Approved by: alexlarsson
2019-01-14 10:17:47 +00:00
Alexander Larsson 51fc2cb07b Set SOURCE_DATE_EPOCH to the mtime of the manifest
This will help make reproducible builds.

Closes: #255
Approved by: alexlarsson
2019-01-11 15:40:43 +00:00
Matthias Clasen 02c69e65d1 Check the flatpak version
Before using --noninteractive, check that we are dealing
with a new-enough flatpak to understand that option.

Closes: #254
Approved by: alexlarsson
2019-01-11 14:53:08 +00:00
Matthias Clasen d5d6debdce Pass --noninteractive to flatpak install
This produces output more suitable for flatpak-builder.

Closes: #254
Approved by: alexlarsson
2019-01-11 14:53:08 +00:00
eszlari 673a00c0b1 add .txz archive suffix
Closes: #249
Approved by: TingPing
2018-12-31 13:53:09 +00:00
Ryan Gonzalez 44c9b69898 Print a warning when a YAML document has a float-like string value
Closes: #221
Approved by: alexlarsson
2018-10-16 06:10:26 +00:00
Alexander Larsson fb43fa0e00 mirror-screenshots: Run appstream-util "mirror-screenshots in the sandbox
This way we don't rely on the host appstream-glib which could be old.
We already rely on it in the runtime for appstream-compose.

Closes: #217
Approved by: alexlarsson
2018-09-26 12:40:11 +00:00
Alexander Larsson 050e075b81 mirror-screenshots: Don't modify cache object
appstream-util mirror-screenshots rewrites the xml file in-place, so
we need to break hardlinks the before running it, otherwise it will
modify the cache object in place, breaking the repo.

Closes: #217
Approved by: alexlarsson
2018-09-26 12:40:11 +00:00
Alexander Larsson 05dd2c68d9 cache: Make sure we escape stage names when we mark them as done
We need to to this, because they are also escaped as refs in the cache.

Fixes https://github.com/flatpak/flatpak-builder/issues/214

Closes: #215
Approved by: alexlarsson
2018-09-26 08:57:36 +00:00
Alexander Larsson 0dad44b214 patch source: Add paths option to specify multiple patches
In many cases you have a list of patches to apply, with the same
args. This allows you to do that by just listing the patches in
order, thus making the manifest more compact.

Closes: #204
Approved by: alexlarsson
2018-08-17 10:22:17 +00:00
Alexander Larsson fd802105c5 Keep x-* properties in manifest.json
This add custom (de)serialization code for the source, module and manifest
objects so that properties starting with "x-" are kep and then put back
in the manifest.

We also add a checksum of the manifest to the "finish" phase so that
if you change them the manifest is re-generated.

Closes: #203
Approved by: TingPing
2018-08-15 16:39:06 +00:00
Bastien Nocera dda576d899 builder: Ensure committer is set when running git-commit
Otherwise buildbots, and other non-interactive/clean-room builds might
not have git committer information set, and make it impossible to
successfully commit, and git erroring out with:
*** Please tell me who you are.

Closes: #183
Approved by: alexlarsson
2018-08-11 10:34:34 +00:00
Alexander Larsson 147d19fba1 rofiles-fuse: Unmount lazily
This has a better chance of succeeding

Closes: #195
Approved by: alexlarsson
2018-08-08 14:02:32 +00:00
Alexander Larsson d7200a4af4 host_spawn: Force kill f-b after forwarding INT/TERM
Otherwise we will just re-enter after INT and forward a TERM.

Closes: #195
Approved by: alexlarsson
2018-08-08 14:02:32 +00:00
Alexander Larsson a594fed33f builder_host_spawnv: NULL dir means cwd
Closes: #195
Approved by: alexlarsson
2018-08-08 14:02:32 +00:00
Alexander Larsson b664e2fc32 Support --run in a sandbox
Closes: #195
Approved by: alexlarsson
2018-08-08 14:02:32 +00:00
Alexander Larsson 00f38fece3 builder_maybe_host_spawnv: Pass on flags in the host case
Closes: #195
Approved by: alexlarsson
2018-08-08 14:02:32 +00:00