Open Stack

Note that skopeo will shortly have the layers diffs [0] based "copy" 
operations [1],[2],[3]. It also deduplicates things and uses local cache 
to not repeat redundant data transfers. Would be great if TripleO 
container registry could start using that super cool feature as early 
adoption.

We could make a build with these patches and try that for the subject 
tooling for TCIB distribution over quay.io.

IIUC, the process is like:

* as input we have full layers from the source registry, built with TCIB;
* skopeo uses its super powers to redo it on fly into layers diff
* either skopeo as well, or maybe even tripleoclient prepare image CLI 
pushes those optimized layers of the images to the target quay.io registry.
* ???
* profit!

[0] https://github.com/containers/image/pull/902
[1] https://github.com/containers/image/pull/1084
[2] https://github.com/containers/storage/pull/775
[3] https://github.com/containers/skopeo/pull/1111

On 11/19/20 3:25 PM, Carlos Camacho Gonzalez wrote:
> I mean in terms of just adding the bits you are missing? Like the 
> parsing and tagging?
> 
> On Thu, Nov 19, 2020 at 3:16 PM Carlos Camacho Gonzalez 
> <ccamacho at redhat.com <mailto:ccamacho at redhat.com>> wrote:
> 
>     Quick question here (maybe I don't have the whole context).
> 
>     Isn't it possible to use tools like skopeo to do this image sync and
>     avoid adding code that will be eventually more complex to maintain?
> 
>     Cheers,
>     Carlos.
> 
>     On Thu, Nov 19, 2020 at 3:05 PM Arx Cruz <arxcruz at redhat.com
>     <mailto:arxcruz at redhat.com>> wrote:
> 
>         Hello,
> 
>         I just wrote a tool in go that is right now copying every 2
>         hours all the containers from rdo registry to quay.io
>         <http://quay.io>.
> 
>         Right now it's copying all branches: master, victoria, ussuri,
>         train, stein, queens and rocky. You can see the containers in
>         https://quay.io/tripleo{release}
>         <https://quay.io/tripleo%7Brelease%7D> for example
>         https://quay.io/tripleomaster
> 
>         The tool basically searches for container build jobs with
>         success status, parse the containers that was built, copy these
>         containers to quay.io <http://quay.io> and tag it with
>         current-tripleo and the built hash.
> 
>         The code it's based on skopeo, and I did not use it, because
>         there are some other stuff required on quay.io <http://quay.io>
>         side that requires the use of quay.io <http://quay.io> api, like
>         create an already public repository, parsing the tripleo
>         container build job, tagging, etc. Also I wanted to play with go :)
> 
>         Right now the code is under review at
>         https://review.rdoproject.org/r/#/c/31133/ and I got it running
>         on our toolbox, and he's an log example (without the --debug flag)
> 
>         time="2020-11-19T12:06:17Z" level=info msg="Copying image
>         centos-binary-horizon:4fad79713786f77292e59fa1c036f588 in
>         tripleoussuri namespace"
>         time="2020-11-19T12:06:18Z" level=info msg="Tagging
>         current-tripleo to
>         sha256:db39e7d43d4c8eec82f61acd4956d7f165b595d515f270f8dabe0c9b009c95f2"
>         "Updated"
> 
>         time="2020-11-19T12:06:19Z" level=info msg="Copying image
>         centos-binary-ceilometer-base:4fad79713786f77292e59fa1c036f588
>         in tripleoussuri namespace"
>         time="2020-11-19T12:06:20Z" level=info msg="Tagging
>         current-tripleo to
>         sha256:5eee849a9f74d0107cbfd2f456f701be37256ea711079cffcc189d140e8f3176"
>         "Updated"
> 
>         time="2020-11-19T12:06:21Z" level=info msg="Copying image
>         centos-binary-gnocchi-base:4fad79713786f77292e59fa1c036f588 in
>         tripleoussuri namespace"
>         time="2020-11-19T12:06:22Z" level=info msg="Tagging
>         current-tripleo to
>         sha256:5b46ea43d41e56dd81005a234631905a8ad594ef8432ae78542392114956f85e"
>         "Updated"
> 
>         If you want to play around with this, feel free to do so, and
>         any feedback is welcome :)
> 
>         Kind regards,
> 
> 
>         -- 
> 
>         Arx Cruz
> 
>         Software Engineer
> 
>         Red Hat EMEA <https://www.redhat.com>
> 
>         arxcruz at redhat.com <mailto:arxcruz at redhat.com>
> 
>         @RedHat <https://twitter.com/redhat> Red Hat
>         <https://www.linkedin.com/company/red-hat> Red Hat
>         <https://www.facebook.com/RedHatInc>
>         <https://www.redhat.com>
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando