Commit Graph

236 Commits (7f31a1acae8f9825ccd083cf450cf411e467fb68)

Author SHA1 Message Date
Alexander Larsson 8737153ded builder: Fix --verbose logging
We need to pass the right log domain in.
2016-12-19 14:55:22 +01:00
Alexander Larsson 364def2135 builder: Add some spew when applying patches
This makes it easier to figure out what patch is broken
2016-12-12 17:20:36 +01:00
Alexander Larsson e514dc2e14 Add flatpak_create_soup_session 2016-12-08 16:41:20 +01:00
Patrick Griffis 21b65947e8 builder: Add buildsystem option and meson support
Closes #400
2016-11-30 16:59:20 +01:00
Alexander Larsson 632f714509 builder: Add "options" property to patch sources
This lets you specify custom arguments to the patch command.

Fixes https://github.com/flatpak/flatpak/issues/417
2016-11-30 10:17:15 +01:00
Alexander Larsson 8ce04c6788 builder: Add support for --jobs=N to limit parallel builds
As requested in: https://github.com/flatpak/flatpak/issues/419
2016-11-30 09:57:55 +01:00
Alexander Larsson 462178bcd3 builder-options: Fix leaks 2016-11-28 17:23:38 +01:00
Alexander Larsson 100038513d builder: Fix coverity warning
If type is NULL we should never look at it.
2016-11-21 10:28:13 +01:00
Matthew Leeds 7b8becd1b2 builder: Don't download modules that won't be built
Currently flatpak-builder downloads the sources for all modules, even if
the --stop-at option prevents them from being built. This commit changes
the behavior so that we don't bother to download the ignored modules.
2016-11-15 15:01:05 +01:00
Alexander Larsson 7c0ef5dccb builder-source: Check for type == NULL before dereferencing 2016-11-11 10:48:37 +01:00
Matthew Leeds 8b50648ec0 builder: Check if .gitmodules exists before showing it. Fixes #366
This commit checks if <revision>:.gitmodules exists in the repo before
calling git-show on it, so an error message doesn't show up in the
program's output.
2016-11-01 09:30:31 +01:00
Alexander Larsson 6e31932221 OCI: Generate an oci image dir structure instead of a runtime 2016-10-31 20:11:14 +01:00
Alexander Larsson 1b09719298 builder: Handle app ids with dashes in them
An app id like org.domain.App-name is now allowed (dashes in the
last element), but blindly appending e.g. ".Locale" to these
for the locale extension will cause an invalid id.

In order to handle this we replace "-" with "_" in this case, which
should be fine as both are under the org.domain control which can
make sure there are no conflicts, and there is no requirement for
the id prefixes to exactly match.

This fixes https://github.com/flatpak/flatpak/issues/370
2016-10-26 21:40:49 +02:00
Alexander Larsson fad8af2f36 builder: Break out name generation for Debug and Locale extensions
These helpers lets us tweak these names
2016-10-26 20:57:57 +02:00
Alexander Larsson 0f05143227 builder: Handle the case where the source directory is in a symlink
It can happen, like in https://github.com/flatpak/flatpak/issues/353
that there are references in the source to the absolute source path.
We need to ensure that is visible in the build. We mostly do this,
but for technical reasons it didn't get the same pathname when
the source was inside a symlinked directory. We fix this with an
extra bind-mount to the symlinked directory name too.

This fixes https://github.com/flatpak/flatpak/issues/353
2016-10-18 11:06:43 +02:00
Alexander Larsson 4cb3f7fd79 builder: Support application inheritance
When initializing a build you can initialize the application directory
with the files from another directory, and optionally a set of its
extensions. This allows one to create a "base" application that
can be derived from in multiple forms. A typical example is an application
framework like Electron. The base electron app has all the dependencies
needed to write Electron app, and then your manifest can add the specific
files needed for your application.

An example snipped could look something like:

    "base": "org.base.App",
    "base-version": "stable",
    "base-extensions": ["org.base.app.Locale", "org.base.app.Debug" ],
2016-10-17 11:40:57 +02:00
Alexander Larsson a9c924928c builder: Add runtime-commit to resolved manifest
Apparently this must have been forgotten when sdk-commit was added.
2016-10-17 10:19:36 +02:00
Matt Watson b9521e719b builder: add allow missing runtimes options
The --allow-missing-runtimes options will allow flatpak builder to
not abort immediately if the sdk or runtime for the app being built
are missing.

This option will be useless when building anything in the modules
section of the app manifest. The calls to flatpak-build will fail
because of the missing sdk.

However, it may be useful when an application does not require
building anything inside the sandbox, and the application files
will be installed via other means.
2016-10-17 09:49:34 +02:00
Matt Watson e968b87f6a builder: add a build finish option
With the option you can resume an application build where it left
off when calling flatpak-buidler --build-only
2016-10-17 09:49:34 +02:00
Alexander Larsson c0e0fe31ef utils: Remove unused variable 2016-10-06 12:31:54 +02:00
Alexander Larsson ea17c27b37 builder: Add missing debug sprew when running an app 2016-10-06 12:31:54 +02:00
Niv Sardi 7f036570fd flatpak-builder: don't strip zip files
It is allowed to make a self-executable zip, it will then be an ELF file
on Linux. The problem is that the current stripping code somehow breaks
that file (i.e. ./file.zip doesn't work anymore after stripping).

Signed-off-by: Niv Sardi <xaiki@endlessm.com>
2016-10-06 12:05:30 +02:00
Alexander Larsson 91c9d474dd flatpak-builder: Add support for --show-deps 2016-09-20 15:58:43 +02:00
Tobias Mueller 72de257ec0 builder: check for name being defined in expand_modules
If this is not checked, flatpak-builder might crash like this:

Starting program: /var/tmp/flatpak/bin/flatpak-builder -v --force-clean
-v --ccache --repo=/tmp/fb.repo --gpg-sign=tobiasmue@gnome.org
/tmp/fpbuilder org.gnome.Sdk.ASan.json
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-
gnu/libthread_db.so.1".
[New Thread 0x7ffff139c700 (LWP 20407)]

Thread 1 "flatpak-builder" received signal SIGSEGV, Segmentation fault.
0x00007ffff697c770 in g_str_hash () from /lib/x86_64-linux-
gnu/libglib-2.0.so.0
(gdb) bt
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
    expanded=expanded@entry=0x7fffffffdae0, names=names@entry=0x690a40,
    error=error@entry=0x7fffffffdc08) at builder/builder-manifest.c:175
    expanded=expanded@entry=0x696158, names=names@entry=0x690a40,
    error=error@entry=0x7fffffffdc08) at builder/builder-manifest.c:169
(self=self@entry=0x696050,
    context=context@entry=0x697970, error=error@entry=0x7fffffffdc08)
    at builder/builder-manifest.c:933
    at builder/builder-main.c:349
(gdb) up
   from /lib/x86_64-linux-gnu/libglib-2.0.so.0
(gdb)
    expanded=expanded@entry=0x7fffffffdae0, names=names@entry=0x690a40,
    error=error@entry=0x7fffffffdc08) at builder/builder-manifest.c:175
175	      if (g_hash_table_lookup (names, name) != NULL)
(gdb) l
170	        return FALSE;
171
172	      *expanded = g_list_concat (*expanded, submodules);
173
174	      name = builder_module_get_name (m);
175	      if (g_hash_table_lookup (names, name) != NULL)
176	        {
177	          g_set_error (error, G_IO_ERROR, G_IO_ERROR_FAILED,
178	                       "Duplicate modules named '%s'", name);
179	          return FALSE;
(gdb) p names
$1 = (GHashTable *) 0x690a40
(gdb) p name
$2 = 0x0
(gdb) p m
$3 = (BuilderModule *) 0x6978d0
(gdb) p *m
$4 = {parent = {g_type_instance = {g_class = 0x6959d0}, ref_count = 1,
    qdata = 0x0}, name = 0x0, subdir = 0x0, post_install = 0x0,
config_opts = 0x0, make_args = 0x0, make_install_args = 0x0, disabled
= 0,
rm_configure = 0, no_autogen = 0, no_parallel_make = 0,
no_python_timestamp_fix = 0, cmake = 0, builddir = 0, build_options =
0x0,
changes = 0x0, cleanup = 0x0, cleanup_platform = 0x0, sources = 0x0,
modules = 0x0}
(gdb)

This happens when a definition does not include a name, e.g.

            "modules": [
                {
                    "type": "archive",
                    "url": ""foo",
                    "sha256": "123"
                }
2016-09-14 15:52:14 +02:00
Mathieu Bridon d165475e17 builder: Don't fail when submodule path isn't a gitlink
This is what git itself does, so we just should do the same.

cda1bbd474/builtin/submodule--helper.c (L247-L248)
2016-09-13 00:24:42 +02:00
Mathieu Bridon 5430d4d40e builder: Properly check for gitlink trees
Instead of expecting the second word to be "commit", we should do like
git does and check for the file mode, which is 160000 for gitlink trees.

cda1bbd474/cache.h (L96)
2016-09-13 00:24:42 +02:00
Alexander Larsson fba645a999 Remove unused local variables reported by clang 2016-09-12 11:11:35 +02:00
Alexander Larsson 5ef43f6ee3 builder: Bump manifest version to trigger rebuild with new mtimes
To make sure we're not using old caches with the previous mtime
timestamps encoded we bump the manifest version to force a rebuild.
2016-09-09 09:44:54 +02:00
Alexander Larsson 6689c5c7f1 Switch back to mtime==0 for ostree checkouts
OSTree upstream changed back from mtime 1
2016-09-09 09:22:56 +02:00
Alexander Larsson 41443313ca session-helper: Move HostCommand to its own interface 2016-09-06 15:30:38 +02:00
Alexander Larsson e792686892 builder: Propagate build-args to shell sources
This way you can e.g. do network i/o in them.
Thats not a recommended way to do it though, as you work around
a lot of the feature in flatpak-builder like the caching and
verification.
2016-09-05 22:01:56 +02:00
Alexander Larsson c66e394266 builder: Use HostCommand calls if in sandbox 2016-09-05 21:41:14 +02:00
Alexander Larsson ddf05ef063 builder: Use flatpak_spawnv helper in a few more places
This is in preparation for it calling the new host-command.
2016-09-05 21:41:14 +02:00
Alexander Larsson 509bb33a2b Merge pull request #237 from mwleeds/add-trace-msg
utils: Print a debug message when launching subprocesses
2016-08-30 09:22:02 +02:00
Alexander Larsson 07fa8138a2 Merge pull request #194 from smcv/shell-injection
unrpm: prevent shell injection
2016-08-26 11:19:34 +02:00
Alexander Larsson 90d4f8d1b4 Don't fail if appstream-compose fails, instead warn.
This means we stop breaking existing builds e.g. when an older
appstream-compose in the runtime can't handle a new appdata field.
2016-08-25 11:34:10 +02:00
Mathieu Bridon 63cddfa80e builder: Handle submodules using with relative URLs
Some projects declare their Git submodules with URLs relative to their
own.

For example, gedit is hosted at git://git.gnome.org/gedit, and has its
libgd submodule declared with the ../libgd URL. We should treat the
latter as git://git.gnome.org/libgd.

This was in fact properly done in the past, but I broke it in commit
1171e1d202.

This fixes it again.
2016-08-24 21:06:00 +02:00
Alexander Larsson b0a7f8052b builder: Add --sandbox support
This disables build-args support, which is nice when building things
on e.g. a shared build-machine, where we don't want the build to
be able to break out of the sandbox (by specifying e.g. --share=network).
2016-08-22 17:23:32 +02:00
Alexander Larsson b03b21f416 Merge branch 'builder-git-submodules' of git://github.com/bochecha/flatpak into bochecha-builder-git-submodules 2016-08-22 16:08:39 +02:00
Alexander Larsson 6578d37984 Drop libgsystem dependency 2016-08-22 16:00:33 +02:00
Alexander Larsson 65f1cf993d Create and use flatpak_file_get_path_cached
This is simpler that the one in libgs, as well as lockless. Also, it
removes one more use of libgs.
2016-08-22 10:22:42 +02:00
Alexander Larsson 9c907c992d Replace gs_shutil_rm_rf with small helper wrapper over libglnx 2016-08-22 10:06:08 +02:00
Alexander Larsson e0bd22bfe8 Add flatpak_mkdir_p helper and use it
This replaces all current callers of gs_file_ensure_directory with
equivalent code.

Actually, two instances were calling gs_file_ensure_directory with
FALSE, i.e. error out on EEXIST, but those cases seem fine with the
do-nothing-if-exists semantics.
2016-08-22 09:29:24 +02:00
Mathieu Bridon 1171e1d202 builder: Redo git submodules handling
flatpak-builder was mixing the path and the name of a submodule, which
would lead to download failures when they were different.

Fixes #216
2016-08-20 12:44:27 +02:00
Alexander Larsson fb3bfae85b builder: Add --stop-at=module
Stop building at a specified module from the json file.
This is useful during development. For instance, you can do --stop-at=main-app
to build all the dependencies into the appdir, then you can do a build of an
already checked out git repo (possibly with changes).
2016-08-19 11:25:23 +02:00
Alexander Larsson f622b014a7 builder: Handle symbolic icons in rename-icons 2016-08-19 11:05:18 +02:00
Alexander Larsson 0d115fb2e7 builder: Save a copy of the base manifest 2016-08-17 14:24:32 +02:00
Alexander Larsson 7431fff588 Fix up pyo stale file handling
We were not doing what the comments said we should, and we also
want to change what happens in the case of an old py file from
a previous layer (such as a base layer in a runtime) has a non-rewritten
.pyc file. In that case we want to assume its correct and fix it up.
2016-08-11 07:54:48 +02:00
Alexander Larsson 348b7f9ab4 builder: Add some printouts if we remove stale .pyc files
This makes it easier to figure out why it disappeared
2016-08-11 07:54:25 +02:00
Alexander Larsson 006d9a1927 Correctly handle .pyc mtimes for .py files changing multiple times
If a .py file changes multiple times we can end up in a situation
where there is an .py file with corresponding .pyc file that we
rewrote, so both are now mtime==1. Then a new version of the .py file
is added, but the corresponding .pyc file is not updated. This means
that the .pyc file is stale, and python would not normally use it.
However, we will later change the mtime on the .py file to 1, causing
the old .pyo file to look up-to-date even though its stale.

We fix this by detecting the case where the is a new mtime on a .py file
where the .pyc file doesn't match, and remove the stale .pyc file.
2016-08-10 16:24:11 +02:00