Open Stack

Hey Flavio,

I attached two architecture diagrams highlighting the four main
lifecycle pieces of OpenStack orchestration: bootstrapping,
deployment, upgrading, and scaling. Config is also in there, but
it was constant throughout each diagram.

The diagrams are not designed to be a detailed workflow of events, but
rather an easy to consume high level architecture.

I'll iterate on this further when I get a chance.

Thanks!
-Ryan

----- Original Message -----
From: "Davanum Srinivas" <davanum at gmail.com>
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
Sent: Wednesday, September 28, 2016 12:27:45 PM
Subject: Re: [openstack-dev] [kolla][fuel][tripleo] Reference architecture to deploy OpenStack on k8s

Here you go Flavio, Sergey and team collected some information from
fuel-ccp efforts.

Design for OpenStack Containerized Control Plane :
https://review.openstack.org/#/c/378266/
Design document for clustering services on k8s :
https://review.openstack.org/#/c/378244/
Add test plan/results for fuel-ccp : https://review.openstack.org/#/c/378271/

Thanks,
Dims

On Tue, Sep 27, 2016 at 4:23 AM, Flavio Percoco <flavio at redhat.com> wrote:
> On 27/09/16 00:41 +0000, Fox, Kevin M wrote:
>>
>> I think some of the disconnect here is a potential misunderstanding about
>> what kolla-kubernetes is....
>>
>> Ultimately, to me, kolla-kubernetes is a database of architecture bits to
>> successfully deploy and manage OpenStack on k8s. Its building blocks. Pretty
>> much what you asked for.
>>
>> There are a bunch of ways of building openstacks. There is no one true
>> way. It really depends on what the operator wants the cloud to do. Is a
>> daemonset or a petset the best way to deploy a cinder volume pod in k8s? The
>> answer is, it depends. (We have an example where one or the other is better
>> now)
>>
>> kolla-kubernetes is taking the building block approach. It takes a bit of
>> information in from the operator or other tool, along with their main
>> openstack configs, and generates k8s templates that are optimized for that
>> case.
>>
>> Who builds the configs, who tells it when to build what templates, and in
>> what order they are started is a separate thing.
>>
>> You should be able to do a 'kollakube template pod nova-api' and just see
>> what it thinks is best.
>>
>> If you want a nice set of documents, it should be easy to loop across them
>> and dump them to html.
>>
>> I think doing them in a machine readable way rather then a document makes
>> much more sense, as it can be reused in multiple projects such as tripleo,
>> fuel, and others and we all can share a common database. We're trying to
>> build a community around this database.
>>
>> Asking to basically make a new project, that does just a human only
>> readable version of the same database seems like a lot of work, with many
>> fewer useful outcomes.
>
>
> I just want to point out that I'm not asking anyone to make a new project
> and
> that my intention is to collect info from other projects too, not just
> kolla-kubernetes. This is a pure documentation effort. I understand you
> don't
> think this is useful and I appreciate your feedback.
>
> Flavio
>
>
>> Please help the community make a great machine and human readable
>> reference architecture system by contributing to the kolla-kubernetes
>> project. There are plenty of opportunity to help out.
>>
>> Maybe making some tools to make the data contained in the database more
>> human friendly would suit your interests? Maybe a nice web frontend that
>> asks a few questions and renders templates out in nice human friendly ways?
>>
>> Thanks,
>> Kevin
>> ________________________________________
>> From: Flavio Percoco [flavio at redhat.com]
>> Sent: Monday, September 26, 2016 9:42 AM
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: Re: [openstack-dev] [kolla][fuel][tripleo] Reference architecture
>> to deploy OpenStack on k8s
>>
>> On 23/09/16 17:47 +0000, Steven Dake (stdake) wrote:
>>>
>>> Flavio,
>>>
>>> Forgive the top post and lack of responding inline – I am dealing with
>>> lookout 2016 which apparently has a bug here [0].
>>>
>>> Your question:
>>>
>>> I can contribute to kolla-kubernetes all you want but that won't give me
>>> what I
>>> asked for in my original email and I'm pretty sure there are opinions
>>> about the
>>> "recommended" way for running OpenStack on kubernetes. Questions like:
>>> Should I
>>> run rabbit in a container? Should I put my database in there too? Now
>>> with
>>> PetSets it might be possible. Can we be smarter on how we place the
>>> services in
>>> the cluster? Or should we go with the traditional
>>> controller/compute/storage
>>> architecture.
>>>
>>> You may argue that I should just read the yaml files from
>>> kolla-kubernetes and
>>> start from there. May be true but that's why I asked if there was
>>> something
>>> written already.
>>> Your question ^
>>>
>>> My answer:
>>> I think what you are really after is why kolla-kubernetes has made the
>>> choices we have made.  I would not argue that reading the code would answer
>>> that question because it does not.  Instead it answers how those choices
>>> were implemented.
>>>
>>> You are mistaken in thinking that contributing to kolla-kubernetes won’t
>>> give you what you really want.  Participation in the Kolla community will
>>> answer for you *why* choices were made as they were.  Many choices are left
>>> unanswered as of yet and Red Hat can make a big impact in the future of the
>>> decision making about *why*.  You have to participate to have your voice
>>> heard.  If you are expecting the Kolla team to write a bunch of
>>> documentation to explain *why* we have made the choices we have, we frankly
>>> don’t have time for that.  Ryan and Michal may promise it with architecture
>>> diagrams and other forms of incomplete documentation, but that won’t provide
>>> you a holistic view of *why* and is wasted efforts on their part (no offense
>>> Michal and Ryan – I think it’s a worthy goal.  The timing for such a request
>>> is terrible and I don’t want to derail the team into endless discussions
>>> about the best way to do things).
>>>
>>> The best way to do things is sorted out via the gerrit review process
>>> using the standard OpenStack workflow through an open development process.
>>
>>
>> Steve,
>>
>> Thanks for getting back on this. Unfortunatelly, I think you keep missing
>> my
>> point and my goal.
>>
>> I'd like to document the architectural choices and see if there's a common
>> ground in which different teams can collaborate on. In addition to this,
>> we'll
>> also see at what point these teams will start diverging in architectural
>> choices. Will the time invested on this be entirely wasted? Maybe.
>>
>> I'm failing to see what is wrong about my request. You mention that I need
>> to
>> contribute to have my voice heard in Kolla as if I'm trying to change
>> anything
>> in it. Spoiler alert: I'm not.
>>
>> I'd like to first work on what I've mentioned in my email and then take
>> the next
>> step. It's also important to note that I've not asked the Kolla team to do
>> this
>> themselves. I've said that I'd like to hear thoughts and friendly
>> discussions on
>> this from different teams (not just kolla), which could easily happen over
>> email. For example, we could stop arguing whether my email makes sense or
>> not
>> and perhaps start dropping some ideas here.
>>
>> Anyway, I appreciate your input and you taking the time to explain the
>> status
>> and efforts of the Kolla team. As far as my contributions go, this is a
>> way for
>> me to start contributing on the deployment on containers efforts around
>> the
>> community and more specifically on the kubernetes side. It might not be
>> what
>> everyone wants but I believe it does help and it'll create a common place
>> for
>> collaboration on this topic amongts different communities (including OPs).
>>
>> Flavio
>>
>>> Flavio,
>>>
>>> Consider this an invitation to come join us – we want Red Hat’s
>>> participation.
>>>
>>> Regards
>>> -steve
>>>
>>>
>>> [0]
>>> http://answers.microsoft.com/en-us/msoffice/forum/msoffice_outlook-mso_mac/outlook-for-mac-2016-replying-inline-with-html-no/298b830e-11ea-416c-b951-918d8f9562cb
>>>
>>> From: Flavio Percoco <flavio at redhat.com>
>>> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>>> <openstack-dev at lists.openstack.org>
>>> Date: Friday, September 23, 2016 at 3:10 AM
>>> To: "OpenStack Development Mailing List (not for usage questions)"
>>> <openstack-dev at lists.openstack.org>
>>> Subject: Re: [openstack-dev] [kolla][fuel][tripleo] Reference
>>> architecture to deploy OpenStack on k8s
>>>
>>> On 22/09/16 20:55 +0000, Steven Dake (stdake) wrote:
>>> Flavio,
>>>
>>> Apologies for delay in response – my backlog is large.
>>>
>>> Forgive me if I parsed your message incorrectly.
>>>
>>> It's probably me failing to communicate my intent or just the intent not
>>> being
>>> good enough or worth it at all.
>>>
>>> It came across to me as “How do I blaze a trail for OpenStack on
>>> Kubernetes?”.  That was asked of me personally 3 years ago which led to the
>>> formation of the Kolla project inside Red Hat.  Our initial effort at that
>>> activity failed.  Instead we decided kubernetes wasn’t ready for
>>> trailblazing in this space and used a far more mature project (Ansible) to
>>> solve the “OpenStack in Containers” problems and build from there.
>>>
>>> We have since expanded our scope to re-solve the “How do I blaze a trail
>>> for Openstack on Kubernetes?” question since Kubernetes is now ready for
>>> this sort of trailblazing.  Fuel and several other folks decided to create
>>> derived works of the Kolla community’s innovations in this area.  I would
>>> contend that Fuel didn’t need to behave in such a way because the Kolla
>>> community is open, friendly, mature, diversely affiliated, has a reasonable
>>> philosophy and good set of principles as well as a strong leadership
>>> pipeline.
>>>
>>> Rather than go blaze a trail when one already exists or create a derived
>>> work, why not increase your footprint in Kolla instead?  Red Hat has
>>> invested in Kolla for some time now, and their footprint hasn’t magically
>>> disappeared over night.   We will give you what you want within reasonable
>>> boundaries (the boundaries all open-source projects set of their
>>> contributors).  We also accept more work than the typical OpenStack project
>>> might, so it’s not like you will have to bring donuts into the office for
>>> every patch you merge into Kolla.
>>>
>>> As to your more direct question of reference architecture, that is a
>>> totally loaded term that I’ll leave untouched.
>>>
>>> To answer your question of “Does Kolla have a set of best practices” the
>>> answer is yes in kolla-ansible and kolla itself and strongly forming set of
>>> best practices in kolla-kubernetes.
>>>
>>> As I mentioned in my email, I don't really care about the implementation
>>> right
>>> now. I'm not trying to change the current teams, goals, or anything. I
>>> would go
>>> as far as saying that the acknowledgement of the existing teams in my
>>> original
>>> email was merely a way to identify a set of teams that might be
>>> interested in
>>> writing this reference architecture.
>>>
>>> Is it a loaded term? Maybe, is this point relevant for my original
>>> question? I'd
>>> say no. It doesn't matter what we call this, not to me, not right now.
>>>
>>> Don't get me wrong, I understand where you're coming from and I
>>> appreciate your
>>> input. Unfortunately, I think you addressed my email from the wrong angle
>>> as I'm
>>> a step (or many steps) early from doing any kind of implementation and I
>>> tried
>>> to be clear about this in my original email.
>>>
>>> I can contribute to kolla-kubernetes all you want but that won't give me
>>> what I
>>> asked for in my original email and I'm pretty sure there are opinions
>>> about the
>>> "recommended" way for running OpenStack on kubernetes. Questions like:
>>> Should I
>>> run rabbit in a container? Should I put my database in there too? Now
>>> with
>>> PetSets it might be possible. Can we be smarter on how we place the
>>> services in
>>> the cluster? Or should we go with the traditional
>>> controller/compute/storage
>>> architecture.
>>>
>>> You may argue that I should just read the yaml files from
>>> kolla-kubernetes and
>>> start from there. May be true but that's why I asked if there was
>>> something
>>> written already.
>>>
>>> Thanks for your email,
>>> Flavio
>>>
>>> Regards
>>> -steve
>>>
>>>
>>>
>>> On 9/22/16, 4:04 AM, "Flavio Percoco"
>>> <flavio at redhat.com<mailto:flavio at redhat.com>> wrote:
>>>
>>>    Greetings,
>>>
>>>    I've recently started looking into the container technologies around
>>> OpenStack.
>>>    More specifically, I've been looking into the tools that allow for
>>> deploying
>>>    OpenStack on containers, which is what I'm the most interested in
>>> right now as
>>>    part of the TripleO efforts.
>>>
>>>    I'm familiar with the Kolla project and the tools managed by this
>>> team. In fact,
>>>    TripleO currently uses kolla images for the containerized nova-compute
>>>    deployment.
>>>
>>>    I am, however, looking beyond a docker based deployment. I'd like to
>>> explore in
>>>    more depth a Kubernetes based deployment of OpenStack. I'm familiar
>>> with both
>>>    kolla-kubernetes and fuel-ccp, their structure and direction*. Both
>>> projects
>>>    have now advanced a bit in their implementations and made some
>>> decisions.
>>>
>>>    As someone that started looking into this topic just recently, I'd
>>> love to see
>>>    our communities collaborate more wherever possible. For example, it'd
>>> be great
>>>    to see us working on a reference architecture for deploying OpenStack
>>> on
>>>    kubernetes, letting the implementation details aside for a bit. I'd
>>> assume some
>>>    folks have done this already and I bet we can all learn more from it
>>> if we work
>>>    on this together.
>>>
>>>    So, let me go ahead and ask some further questions here, I might be
>>> missing some
>>>    history and/or context:
>>>
>>>    - Is there any public documentation that acts as a reference
>>> architecture for
>>>      deploying OpenStack on kubernetes?
>>>    - Is this something the architecture working group could help with? Or
>>> would it
>>>      be better to hijack one of kolla meetings?
>>>
>>>    The restult I'd love to see from this collaboration is a reference
>>> architecture
>>>    explaining how OpenStack should be run on Kubernetes.
>>>
>>>    Thanks in advance. I look forward to see us collaborate more on this
>>> area,
>>>    Flavio
>>>
>>>    * thanks to all fuel and kolla contributors that helped me understand
>>> better the
>>>      work in each of these projects and the direction they are headed
>>>    .
>>>    --
>>>    @flaper87
>>>    Flavio Percoco
>>>
>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org<mailto:OpenStack-dev-request at lists.openstack.org>?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>> --
>>> @flaper87
>>> Flavio Percoco
>>>
>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> --
>> @flaper87
>> Flavio Percoco
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> --
> @flaper87
> Flavio Percoco
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Davanum Srinivas :: https://twitter.com/dims

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kolla-kubernetes-arch.pdf
Type: application/pdf
Size: 125980 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160928/c7882d8d/attachment-0002.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: kolla-kubernetes-arch-2.pdf
Type: application/pdf
Size: 123517 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160928/c7882d8d/attachment-0003.pdf>