This adds variable support for collection IDs: they can either be
enabled on the server, on the server and client, or not at all. If
enabled on the server, apps and runtimes are built with collection IDs
and the repository has one set. If enabled on the client, the remote
config is added to the local repository with a collection ID and GPG
verification enabled. They are controlled with
USE_COLLECTIONS_IN_{SERVER,CLIENT}={yes,no}.
These variables are used in the new wrapper tests,
test-repo-collections.sh and test-repo-collections-server-only.sh.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add a sanity check and error message which prevents pulling the
ostree-metadata ref (the repository metadata) unless GPG verification is
enabled, as it needs to be signed to be trusted.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Emit an error message if the user tries to change the collection ID of
an existing repository between two non-empty values. Allow them to set a
collection ID where one was not set before. Changing the collection ID
once it’s already been set will break updates for all clients who have
previously pulled from the repository.
If a developer really wants to change the collection ID for a
repository, they’re going to have to recreate the repository from
scratch.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Emit an error message if a collection ID is specified but GPG is not
enabled, since pulling using collection IDs requires GPG.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This isn’t really used internally, but will be used by gnome-software
for when it configures new flatpak remotes.
This is new public API, but is only declared if compiling with
--enable-p2p.
Includes some basic smoketests.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to provide a transition path for repositories to add collection
IDs to themselves and propagate those collection IDs to clients’ remote
configurations, add another repo config key which controls whether the
repository’s collection ID is published. If xa.collection-id is set in
the repo’s published metadata, the client will update its configuration
to the given ID — but only if no ID is set already. This is a one-time
transition to prevent malicious repositories from remotely changing the
user’s configuration to associate their remote with a well-known
collection ID they don’t own.
Add a test for this.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
To allow staged deployment of collection-ID-based repositories,
introduce the code to update a local repository configuration to add a
collection ID to it, based on updated metadata from the remote (as is
currently supported for other configuration keys).
As a security measure, this only allows updating the collection ID from
an empty to a non-empty value. We do not allow collection IDs to be
renamed (or a malicious repository owner could bypass the user’s manual
verification of the collection ID by changing it after the user has
configured an unrelated remote).
The idea is that most repositories should remain without collection IDs
for now, and use this mechanism to set their collection IDs in future,
once the functionality is more stable.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This will only be built when configured with P2P support. We can’t use
glib-mkenums here, as it doesn’t know about the #ifdef
FLATPAK_ENABLE_P2P which surrounds the enum definition. By manually
writing the get_type() function, we can surround it by #ifdefs as
appropriate.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Rather than silently ignoring them. Note that invalid configurations
are distinct from missing configuration keys.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It can return FALSE with an error set, or FALSE without one set, which
indicates the key was not found.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
When pulling from a local, untrusted repo (i.e. one which the user
downloaded into, and we want to pull into the trusted system repo),
verify the collection ID and ref bindings in the commit metadata for
each commit.
This is something which is normally done by libostree, but since we’re
rewriting the commit manually, we’re bypassing that part of the pull()
code path.
This is an inlined version of the check from verify_bindings() in
libostree.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add support for collection IDs to the code which finds and pulls
related refs and other extensions.
Currently, related refs must have the same collection ID as the parent
ref — this is the most likely scenario anyway. In future, it should be
possible to extend the code to support pulling related refs from other
collections.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Also add collection and ref binding metadata to the generated appstream
commits, so they can be verified when using unsigned summary files.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
We need to consistently handle the case where the collection ID is set
to the empty string (and treat it the same as if it were unset). Best
done in a helper function.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Newer repositories will store metadata there, rather than in the summary
file (although the summary file will still be updated where possible for
backwards compatibility).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to eliminate some race conditions around updating the
summary{,.sig} file on the server, and to decouple signing the summary
from signing commits, and to support peer to peer mirrors of content
from multiple upstream collections: add support for unsigned summary
files.
This relaxes the requirement for gpg-verify-summary=true iff
collection-id is set in a remote’s local configuration. It depends on
some pending libostree changes to verify the ref for each commit using
the commit’s signed metadata. See
https://github.com/ostreedev/ostree/issues/983.
Metadata storage has moved from the summary file to a new
ostree-metadata well-known branch on each repository, since this can be
signed for each update and for each collection separately. If the
collection-id is set in a remote’s local configuration, flatpak will
retrieve all repository metadata from this branch rather than from the
summary file. If collection-id is unset, it will ignore this branch and
continue to use the summary file, which will continue to be updated (and
externally signed as summary.sig) for backwards compatibility.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This adds a new collection-id property which is only enabled if
FLATPAK_ENABLE_P2P is defined. The internal machinery for handling it is
always enabled, to reduce the amount of #ifdef spam.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Pass a --collection-id argument through to `flatpak build-export`.
Also add a ‘collection-id’ property to manifest files, which can be used
to set the collection ID on an exported repo (when using --repo) without
having to provide a command line option.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It will be used by builtins-repo-update in a following commit to allow
updating the collection ID for an upstream repo.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Also expose a new flatpak_remote_get_remote_type() API so that users can
query what type of remote something is — whether it’s a USB or LAN
remote, or something statically configured.
Make this all conditional on compiling with --enable-p2p.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Search for updates on peer to peer sources as well as the internet in
check_for_updates(), and pass the resulting OstreeRepoFinderResult array
to the pull() calls, so a consistent set of checksums are pulled.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This adds support in flatpak-dir.c for using the new libostree API for
finding remotes dynamically for a given set of refs, if flatpak is
configured with --enable-p2p.
The new code paths are only taken if the repository is configured with
a collection ID set.
These changes by themselves aren’t sufficient for full P2P support, as
all the infrastructure for downloading summary files and finding refs
needs to be modified in the following commits.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
A series of following commits will introduce a peer to peer feature for
pulling apps and runtimes from LAN peers and USB sticks without needing
an internet connection. This requires experimental API in libostree
(which needs to have been configured with --enable-experimental-api), so
needs to be hidden behind a configure option in flatpak too. It’s called
--enable-p2p, and bumps our libostree dependency to 2017.8 with
experimental API required too.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This allows you to automatically install/update dependencies required
by the manifest. The dependencies include:
Runtime, Sdk, Base, Sdk Extensions and Platform Extensions
There is also a --install-deps-only switch to make the build
stop after the dependencies are installed.
Fixes https://github.com/flatpak/flatpak/issues/955
This includes a change in how ostree reports missing gpg signatures
which makes the test-suite pass. Additionally that change requires
us to change how we detect such missing gpg signatures in one place.
We could try to support both versions, but the easiest fix is to just
require the latest ostree.
flatpak_decompose_ref() can fail, but in this situation we know it won’t
since find_matching_refs() checks it for us and hence guarantees that it
only returns decomposable refs. Add some assertions to make this clearer
and to shut Coverity up.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Everywhere else that index->manifests is used, it’s checked for being
NULL beforehand, which probably means that, sometimes, it might be NULL.
Let’s check that here too.
Coverity issue: 1452432
Signed-off-by: Philip Withnall <withnall@endlessm.com>
use_builddir is TRUE if meson is TRUE, so there can never be a meson
case in the !use_builddir case.
Coverity issue: 1452428
Signed-off-by: Philip Withnall <withnall@endlessm.com>
If realpath() fails (unlikely) it returns NULL, which we later merrily
dereference. Fix that by taking the unexpanded path.
Also fix a minor indentation problem later on.
Coverity issue 1452435.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This prevents a resource leak, although not really in practice because
va_args are free on common platforms.
Coverity issue 1452430.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Previously they weren’t getting distributed unless the system helper was
enabled at configure time for distcheck. They should be distributed
unconditionally so the user can choose whether to enable the system
helper when they call configure.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Philip Withnall