In order to provide a transition path for repositories to add collection
IDs to themselves and propagate those collection IDs to clients’ remote
configurations, add another repo config key which controls whether the
repository’s collection ID is published. If xa.collection-id is set in
the repo’s published metadata, the client will update its configuration
to the given ID — but only if no ID is set already. This is a one-time
transition to prevent malicious repositories from remotely changing the
user’s configuration to associate their remote with a well-known
collection ID they don’t own.
Add a test for this.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Add support for collection IDs to the code which finds and pulls
related refs and other extensions.
Currently, related refs must have the same collection ID as the parent
ref — this is the most likely scenario anyway. In future, it should be
possible to extend the code to support pulling related refs from other
collections.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Also add collection and ref binding metadata to the generated appstream
commits, so they can be verified when using unsigned summary files.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Newer repositories will store metadata there, rather than in the summary
file (although the summary file will still be updated where possible for
backwards compatibility).
Signed-off-by: Philip Withnall <withnall@endlessm.com>
In order to eliminate some race conditions around updating the
summary{,.sig} file on the server, and to decouple signing the summary
from signing commits, and to support peer to peer mirrors of content
from multiple upstream collections: add support for unsigned summary
files.
This relaxes the requirement for gpg-verify-summary=true iff
collection-id is set in a remote’s local configuration. It depends on
some pending libostree changes to verify the ref for each commit using
the commit’s signed metadata. See
https://github.com/ostreedev/ostree/issues/983.
Metadata storage has moved from the summary file to a new
ostree-metadata well-known branch on each repository, since this can be
signed for each update and for each collection separately. If the
collection-id is set in a remote’s local configuration, flatpak will
retrieve all repository metadata from this branch rather than from the
summary file. If collection-id is unset, it will ignore this branch and
continue to use the summary file, which will continue to be updated (and
externally signed as summary.sig) for backwards compatibility.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
It will be used by builtins-repo-update in a following commit to allow
updating the collection ID for an upstream repo.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Search for updates on peer to peer sources as well as the internet in
check_for_updates(), and pass the resulting OstreeRepoFinderResult array
to the pull() calls, so a consistent set of checksums are pulled.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This includes a change in how ostree reports missing gpg signatures
which makes the test-suite pass. Additionally that change requires
us to change how we detect such missing gpg signatures in one place.
We could try to support both versions, but the easiest fix is to just
require the latest ostree.
We were checking the prefix twice here, instead of
checking prefix and suffix, which was obviously the
intention.
This caused extensions with non-matching architectures
to be reported as related refs.
The GLib logging framework automatically appends a \n to messages, so it
doesn’t need to be added by callers.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Use g_hash_table_add() so that GHashTable can internally optimise for
use as a set, which results in slightly simpler code for the caller, and
slightly less memory usage overall.
This introduces no functional changes.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
This makes it easier to avoid typos, and also makes it easier to see
what groups and keys are in use. In the header file, they are
clustered according to the group in which the keys are used.
Signed-off-by: Simon McVittie <smcv@collabora.com>
This introduces no functional changes. The split out function will be
reused in a later commit to additionally load commit data from the
ostree-metadata branch.
Signed-off-by: Philip Withnall <withnall@endlessm.com>
Writing things on the console every 100msec doesn't really
make any sense, the progress bar is not finegrained enough
and you can't read the text that fast anyway.
We artificially limit the metadata phase to the first 20 metadata
objects, because otherwise that phase takes too long time, as it
downloads content at the same time.
For the delta case we can get perfect results by just ignoring
the counts and relying on the sizes. However, this needs a
recent ostree that supports the fetched-delta-part-size field.
For the object case we keep using the object average size
to estimate a size which we can compare with the extra data
size.
Now that we have access to the total extra-data download size,
we can have much more precise progress reports by summing up
all the download sizes from OSTree and extra-data.
This patch makes the progress report callback use the extra-data
sizes, as well as calculate the average size (in bytes) of the
content.
flatpak/flatpak#609
It turns out the underlying call ostree_repo_query_object_storage_size
doesn't work for staged objects, so we work around this by actually
loading the object, walking back to the original unix input stream
and fstat:ing the fd from that.
When clients install/update they will see this property in the
(signed) summary and update the url in the config, making this
essentially a permanent redirect.
This is a pretty standalone object, and it is nicer to
have it in its own files. All users have been updated
to include the new flatpak-table-printer.h header.
The new `glnx_regfile_copy_bytes()` is better than the previous
`flatpak_copy_bytes()` in that it will use reflink/sendfile if available.
More information in 3a4d0f4684
Alexander Larsson