Factor out the pull options which are common to the collection-based and
non-collection-based code paths. This should make the code a little
easier to read.
This introduces no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
When the summary file is updated, it’s kept in a local cache, so that
parts of flatpak can refresh it at will without network impact. We need
the same for the ostree-metadata ref, which stores the repository’s
metadata when collection IDs and P2P are used.
Implement that by comparing the checksum of the ostree-metadata ref from
the summary file and from the local repository. If they differ, it’s
almost certainly going to be because the summary file is advertising a
more up-to-date ostree-metadata ref, which we should pull. If they don’t
differ, there’s no need to try and update the ref. Therefore, this
chains off the caching of the summary file.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add a sanity check and error message which prevents pulling the
ostree-metadata ref (the repository metadata) unless GPG verification is
enabled, as it needs to be signed to be trusted.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This isn’t really used internally, but will be used by gnome-software
for when it configures new flatpak remotes.
This is new public API, but is only declared if compiling with
--enable-p2p.
Includes some basic smoketests.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to provide a transition path for repositories to add collection
IDs to themselves and propagate those collection IDs to clients’ remote
configurations, add another repo config key which controls whether the
repository’s collection ID is published. If xa.collection-id is set in
the repo’s published metadata, the client will update its configuration
to the given ID — but only if no ID is set already. This is a one-time
transition to prevent malicious repositories from remotely changing the
user’s configuration to associate their remote with a well-known
collection ID they don’t own.
Add a test for this.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
To allow staged deployment of collection-ID-based repositories,
introduce the code to update a local repository configuration to add a
collection ID to it, based on updated metadata from the remote (as is
currently supported for other configuration keys).
As a security measure, this only allows updating the collection ID from
an empty to a non-empty value. We do not allow collection IDs to be
renamed (or a malicious repository owner could bypass the user’s manual
verification of the collection ID by changing it after the user has
configured an unrelated remote).
The idea is that most repositories should remain without collection IDs
for now, and use this mechanism to set their collection IDs in future,
once the functionality is more stable.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Rather than silently ignoring them. Note that invalid configurations
are distinct from missing configuration keys.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It can return FALSE with an error set, or FALSE without one set, which
indicates the key was not found.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
When pulling from a local, untrusted repo (i.e. one which the user
downloaded into, and we want to pull into the trusted system repo),
verify the collection ID and ref bindings in the commit metadata for
each commit.
This is something which is normally done by libostree, but since we’re
rewriting the commit manually, we’re bypassing that part of the pull()
code path.
This is an inlined version of the check from verify_bindings() in
libostree.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add support for collection IDs to the code which finds and pulls
related refs and other extensions.
Currently, related refs must have the same collection ID as the parent
ref — this is the most likely scenario anyway. In future, it should be
possible to extend the code to support pulling related refs from other
collections.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Also add collection and ref binding metadata to the generated appstream
commits, so they can be verified when using unsigned summary files.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
We need to consistently handle the case where the collection ID is set
to the empty string (and treat it the same as if it were unset). Best
done in a helper function.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Newer repositories will store metadata there, rather than in the summary
file (although the summary file will still be updated where possible for
backwards compatibility).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to eliminate some race conditions around updating the
summary{,.sig} file on the server, and to decouple signing the summary
from signing commits, and to support peer to peer mirrors of content
from multiple upstream collections: add support for unsigned summary
files.
This relaxes the requirement for gpg-verify-summary=true iff
collection-id is set in a remote’s local configuration. It depends on
some pending libostree changes to verify the ref for each commit using
the commit’s signed metadata. See
https://github.com/ostreedev/ostree/issues/983.
Metadata storage has moved from the summary file to a new
ostree-metadata well-known branch on each repository, since this can be
signed for each update and for each collection separately. If the
collection-id is set in a remote’s local configuration, flatpak will
retrieve all repository metadata from this branch rather than from the
summary file. If collection-id is unset, it will ignore this branch and
continue to use the summary file, which will continue to be updated (and
externally signed as summary.sig) for backwards compatibility.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It will be used by builtins-repo-update in a following commit to allow
updating the collection ID for an upstream repo.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Search for updates on peer to peer sources as well as the internet in
check_for_updates(), and pass the resulting OstreeRepoFinderResult array
to the pull() calls, so a consistent set of checksums are pulled.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This adds support in flatpak-dir.c for using the new libostree API for
finding remotes dynamically for a given set of refs, if flatpak is
configured with --enable-p2p.
The new code paths are only taken if the repository is configured with
a collection ID set.
These changes by themselves aren’t sufficient for full P2P support, as
all the infrastructure for downloading summary files and finding refs
needs to be modified in the following commits.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This includes a change in how ostree reports missing gpg signatures
which makes the test-suite pass. Additionally that change requires
us to change how we detect such missing gpg signatures in one place.
We could try to support both versions, but the easiest fix is to just
require the latest ostree.
flatpak_decompose_ref() can fail, but in this situation we know it won’t
since find_matching_refs() checks it for us and hence guarantees that it
only returns decomposable refs. Add some assertions to make this clearer
and to shut Coverity up.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Everywhere else that index->manifests is used, it’s checked for being
NULL beforehand, which probably means that, sometimes, it might be NULL.
Let’s check that here too.
Coverity issue: 1452432
Signed-off-by: Philip Withnall <withnall@endlessm.com>
We were checking the prefix twice here, instead of
checking prefix and suffix, which was obviously the
intention.
This caused extensions with non-matching architectures
to be reported as related refs.
If xa.languages is not explicitly set, only use the current locale
if we are installing something per-user. For system-wide installs,
default to installing all locales.
Read an xa.languages key from the [core] section of
the repo config to determine which subpaths to install
for Locales. This lets us maintain a list of system
languages without inventing a new file in /etc, and
will also work for alternative install locations.
The GLib logging framework automatically appends a \n to messages, so it
doesn’t need to be added by callers.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Use g_hash_table_add() so that GHashTable can internally optimise for
use as a set, which results in slightly simpler code for the caller, and
slightly less memory usage overall.
This introduces no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This makes it easier to avoid typos, and also makes it easier to see
what groups and keys are in use. In the header file, they are
clustered according to the group in which the keys are used.
Signed-off-by: Simon McVittie <smcv@collabora.com>
For example, add $(AM_CFLAGS) to mumble_CFLAGS. Since $(WARN_CFLAGS) is
only added to $(AM_CFLAGS), this fixes the lack of inclusion of the
compiler warning flags in the compilation of half of flatpak.
Note that $(AM_*) variables are only used by automake if a more specific
(per-target) special variable is not defined instead. So if you define
mumble_CFLAGS, AM_CFLAGS will not be used for that target unless
explicitly included in mumble_CFLAGS.
See
https://www.gnu.org/software/automake/manual/html_node/Flag-Variables-Ordering.html.
Do the same for $(AM_LIBADD), $(AM_LDFLAGS), etc. These are not
currently defined, but it’s good practice to include them in
mumble_LIBADD (etc.) just in case they’re defined in future. Hopefully
their inclusions will be cargo-culted to any new targets which are
added, retaining full coverage of the code base.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This introduces no functional changes. The split out function will be
reused in a later commit to additionally load commit data from the
ostree-metadata branch.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This will slightly simplify some later refactoring. It makes no real
difference to the function’s behaviour at the moment.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This was accidentally introduced in a8ad3927 in advance of the LAN/USB
changes from PR #884 which will actually use it.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Changes to the Makefile could include changes to the options passed
to gdbus-codegen, which would invalidate the output.
Signed-off-by: Simon McVittie <smcv@collabora.com>
A rule of the form
foo.c foo.h: foo.in
some-generator --output=foo foo.in
is essentially equivalent to writing the same rule once for each target:
foo.c: foo.in
some-generator --output=foo foo.in
foo.h: foo.in
some-generator --output=foo foo.in
In a parallel build, this can result in some-generator being run more
than once with the same inputs and outputs, leading to unpredictable
results if the outputs are overwritten in-place by two parallel copies
(particularly if the generator does not use the standard atomic-writing
trick of writing out a temporary file and renaming it over the top of
the intended name, which gdbus-codegen does not).
gdbus-codegen happens to write the .h file before the .c file, so
use the real build rules to generate the .c file, and consider the
.h file to be a side-effect.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This will make it easier to add another version in upcoming commits
which gets the updated configuration from another source (an
ostree-metadata branch).
This commit introduces no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
There was already a convenience method for this,
fetch_remote_summary_file(), but it wasn’t used uniformly throughout the
file. This introduces no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
There are several places which query keys in the metadata in the summary
file. Factor out a common function to do that, and write it in such a
way that it could be extended to read the metadata from somewhere else
in future (the plan being that metadata will move to an ostree-metadata
branch rather than the summary file).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Alexander Larsson