Open Stack

Another use case I can think of is to cache the required docker images in the Glance image.

This is an important use case because we have containerized most of the COE components (e.g. kube-scheduler, swarm-manager, etc.). As a result, each bay needs to pull docker images over the Internet on provisioning or scaling stage. If a large number of bays pull docker images at the same time, it will generate a lot of traffic. Therefore, it is desirable to have all the required docker images pre-downloaded into the Glance image. I expect we can leverage diskimage-builder to achieve the goal.

Best regards,
Hongbin

From: Ton Ngo [mailto:ton at us.ibm.com]
Sent: March-29-16 4:54 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [magnum] Generate atomic images using diskimage-builder


In multiple occasions in the past, we have had to use version of some software that's not available yet
in the upstream image for bug fixes or new features (Kubernetes, Docker, Flannel,...). Eventually the upstream
image would catch up, but having the tool to customize let us push forward with development, and gate tests
if it makes sense.

Ton Ngo,


[Inactive hide details for Yolanda Robla Mota ---03/29/2016 01:35:48 PM---So the advantages I can see with diskimage-builder are]Yolanda Robla Mota ---03/29/2016 01:35:48 PM---So the advantages I can see with diskimage-builder are: - we reuse the same tooling that is present

From: Yolanda Robla Mota <yolanda.robla-mota at hpe.com<mailto:yolanda.robla-mota at hpe.com>>
To: <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: 03/29/2016 01:35 PM
Subject: Re: [openstack-dev] [magnum] Generate atomic images using diskimage-builder

________________________________



So the advantages I can see with diskimage-builder are:
- we reuse the same tooling that is present in other openstack projects
to generate images, rather than relying on an external image
- it improves the control we have on the contents of the image, instead
of seeing that as a black box. At the moment we can rely on the default
tree for fedora 23, but this can be updated per magnum needs
- reusability: we have atomic 23 now, but why not create magnum images
with dib, for ubuntu, or any other distros ? Relying on
diskimage-builder makes it easy and flexible, because it's a matter of
adding the right elements.

Best
Yolanda

El 29/03/16 a las 21:54, Steven Dake (stdake) escribió:
> Adrian,
>
> Makes sense.  Do the images have to be built to be mirrored though?  Can't
> they just be put on the mirror sites fro upstream?
>
> Thanks
> -steve
>
> On 3/29/16, 11:02 AM, "Adrian Otto" <adrian.otto at rackspace.com<mailto:adrian.otto at rackspace.com>> wrote:
>
>> Steve,
>>
>> I¹m very interested in having an image locally cached in glance in each
>> of the clouds used by OpenStack infra. The local caching of the glance
>> images will produce much faster gate testing times. I don¹t care about
>> how the images are built, but we really do care about the performance
>> outcome.
>>
>> Adrian
>>
>>> On Mar 29, 2016, at 10:38 AM, Steven Dake (stdake) <stdake at cisco.com<mailto:stdake at cisco.com>>
>>> wrote:
>>>
>>> Yolanda,
>>>
>>> That is a fantastic objective.  Matthieu asked why build our own images
>>> if
>>> the upstream images work and need no further customization?
>>>
>>> Regards
>>> -steve
>>>
>>> On 3/29/16, 1:57 AM, "Yolanda Robla Mota" <yolanda.robla-mota at hpe.com<mailto:yolanda.robla-mota at hpe.com>>
>>> wrote:
>>>
>>>> Hi
>>>> The idea is to build own images using diskimage-builder, rather than
>>>> downloading the image from external sources. By that way, the image can
>>>> live in our mirrors, and is built using the same pattern as other
>>>> images
>>>> used in OpenStack.
>>>> It also opens the door to customize the images, using custom trees, if
>>>> there is a need for it. Actually we rely on official tree for Fedora 23
>>>> Atomic (https://dl.fedoraproject.org/pub/fedora/linux/atomic/23/) as
>>>> default.
>>>>
>>>> Best,
>>>> Yolanda
>>>>
>>>> El 29/03/16 a las 10:17, Mathieu Velten escribió:
>>>>> Hi,
>>>>>
>>>>> We are using the official Fedora Atomic 23 images here (on Mitaka M1
>>>>> however) and it seems to work fine with at least Kubernetes and Docker
>>>>> Swarm.
>>>>> Any reason to continue building specific Magnum image ?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Mathieu
>>>>>
>>>>> Le mercredi 23 mars 2016 à 12:09 +0100, Yolanda Robla Mota a écrit :
>>>>>> Hi
>>>>>> I wanted to start a discussion on how Fedora Atomic images are being
>>>>>> built. Currently the process for generating the atomic images used
>>>>>> on
>>>>>> Magnum is described here:
>>>>>> http://docs.openstack.org/developer/magnum/dev/build-atomic-image.htm
>>>>>> l.
>>>>>> The image needs to be built manually, uploaded to fedorapeople, and
>>>>>> then
>>>>>> consumed from there in the magnum tests.
>>>>>> I have been working on a feature to allow diskimage-builder to
>>>>>> generate
>>>>>> these images. The code that makes it possible is here:
>>>>>> https://review.openstack.org/287167
>>>>>> This will allow that magnum images are generated on infra, using
>>>>>> diskimage-builder element. This element also has the ability to
>>>>>> consume
>>>>>> any tree we need, so images can be customized on demand. I generated
>>>>>> one
>>>>>> image using this element, and uploaded to fedora people. The image
>>>>>> has
>>>>>> passed tests, and has been validated by several people.
>>>>>>
>>>>>> So i'm raising that topic to decide what should be the next steps.
>>>>>> This
>>>>>> change to generate fedora-atomic images has not already landed into
>>>>>> diskimage-builder. But we have two options here:
>>>>>> - add this element to generic diskimage-builder elements, as i'm
>>>>>> doing now
>>>>>> - generate this element internally on magnum. So we can have a
>>>>>> directory
>>>>>> in magnum project, called "elements", and have the fedora-atomic
>>>>>> element
>>>>>> here. This will give us more control on the element behaviour, and
>>>>>> will
>>>>>> allow to update the element without waiting for external reviews.
>>>>>>
>>>>>> Once the code for diskimage-builder has landed, another step can be
>>>>>> to
>>>>>> periodically generate images using a magnum job, and upload these
>>>>>> images
>>>>>> to OpenStack Infra mirrors. Currently the image is based on Fedora
>>>>>> F23,
>>>>>> docker-host tree. But different images can be generated if we need a
>>>>>> better option.
>>>>>>
>>>>>> As soon as the images are available on internal infra mirrors, the
>>>>>> tests
>>>>>> can be changed, to consume these internals images. By this way the
>>>>>> tests
>>>>>> can be a bit faster (i know that the bottleneck is on the functional
>>>>>> testing, but if we reduce the download time it can help), and tests
>>>>>> can
>>>>>> be more reilable, because we will be removing an external dependency.
>>>>>>
>>>>>> So i'd like to get more feedback on this topic, options and next
>>>>>> steps
>>>>>> to achieve the goals. Best
>>>>>>
>>>>>
>>>>> _______________________________________________________________________
>>>>> __
>>>>> _
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> --
>>>> Yolanda Robla Mota
>>>> Cloud Automation and Distribution Engineer
>>>> +34 605641639
>>>> yolanda.robla-mota at hpe.com<mailto:yolanda.robla-mota at hpe.com>
>>>>
>>>>
>>>>
>>>> ________________________________________________________________________
>>>> __
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _________________________________________________________________________
>>> _
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
Yolanda Robla Mota
Cloud Automation and Distribution Engineer
+34 605641639
yolanda.robla-mota at hpe.com<mailto:yolanda.robla-mota at hpe.com>


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160330/e8fef991/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 105 bytes
Desc: image001.gif
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160330/e8fef991/attachment.gif>