Open Stack

On 19 October 2017 at 13:32, Joshua Harlow <harlowja at fastmail.com> wrote:
> This reminded me of something I wanted to ask.
>
> Is it true to state that only way to get 'fully' shared-base layers is to
> have `kolla-build` build all the projects (that a person/company/other may
> use) in one invocation? (in part because of the jinja2 template generation
> which would cause differences in dockerfiles?...)

Well jinja2 should render same dockerfile no matter when you call it,
so it should be fine. Alternatively you can run something like
kolla-build nova --skip-parents  - this call will try to build all
images with "nova" in them while not rebuilding openstack-base and
base image.

> I was pretty sure this was the case (unless things have changed), but just
> wanting to check since that question seems (somewhat) on-topic...
>
> At godaddy we build individual projects using `kolla-build` (in part because
> it makes it easy to rebuild + test + deply a single project with either an
> update or a patch or ...) and I suspect others are doing this also (after
> all the kolla-build command does take a regex of projects to build) - though
> doing it in this way does seem like it would not reuse (all the layers
> outside of the base operating system) layers 'optimally'?
>
> Thoughts?
>
> -Josh
>
> Sam Yaple wrote:
>>
>> docker_image wouldn't be the best place for that. Buf if you are looking
>> for a quicker solution, kolla_docker was written specifically to be
>> license compatible for openstack. its structure should make it easily
>> adapted to delete an image. And you can copy it and cut it up thanks to
>> the license.
>>
>> Are you pushing images with no shared base layers at all (300MB
>> compressed image is no shared base layers)? With shared base layers a
>> full image set of Kolla images should be much smaller than the numbers
>> you posted.
>>
>> Thanks,
>> SamYaple
>>
>> On Thu, Oct 19, 2017 at 11:03 AM, Gabriele Cerami <gcerami at redhat.com
>> <mailto:gcerami at redhat.com>> wrote:
>>
>>     Hi,
>>
>>     our CI scripts are now automatically building, testing and pushing
>>     approved openstack/RDO services images to public repositories in
>>     dockerhub using ansible docker_image module.
>>
>>     Promotions have had some hiccups, but we're starting to regularly
>> upload
>>     new images every 4 hours.
>>
>>     When we'll get at full speed, we'll potentially have
>>     - 3-4 different sets of images, one per release of openstack (counting
>> a
>>        EOL release grace period)
>>     - 90-100 different services images per release
>>     - 4-6 different versions of the same image ( keeping older promoted
>>        images for a while )
>>
>>     At around 300MB per image a possible grand total is around 650GB of
>>     space used.
>>
>>     We don't know if this is acceptable usage of dockerhub space and for
>>     this we already sent a similar email the to docker support to ask
>>     specifically if the user would get penalty in any way (e.g. enforcing
>>     quotas, rete limiting, blocking). We're still waiting for a reply.
>>
>>     In any case it's critical to keep the usage around the estimate, and
>> to
>>     achieve this we need a way to automatically delete the older images.
>>     docker_image module does not provide this functionality, and we think
>>     the only way is issuing direct calls to dockerhub API
>>
>>     https://docs.docker.com/registry/spec/api/#deleting-an-image
>>     <https://docs.docker.com/registry/spec/api/#deleting-an-image>
>>
>>     docker_image module structure doesn't seem to encourage the addition
>> of
>>     such functionality directly in it, so we may be forced to use the uri
>>     module.
>>     With new images uploaded potentially every 4 hours, this will become a
>>     problem to be solved within the next two weeks.
>>
>>     We'd appreciate any input for an existing, in progress and/or better
>>     solution for bulk deletion, and issues that may arise with our space
>>     usage in dockerhub
>>
>>     Thanks
>>
>>
>> __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe:
>>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>     <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev