<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">On 10/22/2014 03:42 AM, Vineet Menon
wrote:<br>
</div>
<blockquote
cite="mid:CAD26Kg0T7D=35msoT6+nYOOLzuAQU7yCTyBQ_XB_-CSGLxtSvw@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div dir="ltr"><br>
<div class="gmail_extra">
<div class="gmail_quote">On 22 October 2014 06:24, Tom Fifield
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:tom@openstack.org" target="_blank">tom@openstack.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb">
<div class="h5">On 22/10/14 03:07, Andrew Laski wrote:<br>
><br>
> On 10/21/2014 04:31 AM, Nikola Đipanov wrote:<br>
>> On 10/20/2014 08:00 PM, Andrew Laski wrote:<br>
>>> One of the big goals for the Kilo cycle
by users and developers of the<br>
>>> cells functionality within Nova is to get
it to a point where it can be<br>
>>> considered a first class citizen of
Nova. Ultimately I think this comes<br>
>>> down to getting it tested by default in
Nova jobs, and making it easy<br>
>>> for developers to work with. But there's
a lot of work to get there.<br>
>>> In order to raise awareness of this
effort, and get the conversation<br>
>>> started on a few things, I've summarized
a little bit about cells and<br>
>>> this effort below.<br>
>>><br>
>>><br>
>>> Goals:<br>
>>><br>
>>> Testing of a single cell setup in the
gate.<br>
>>> Feature parity.<br>
>>> Make cells the default implementation.
Developers write code once and<br>
>>> it works for cells.<br>
>>><br>
>>> Ultimately the goal is to improve
maintainability of a large feature<br>
>>> within the Nova code base.<br>
>>><br>
>> Thanks for the write-up Andrew! Some
thoughts/questions below. Looking<br>
>> forward to the discussion on some of these
topics, and would be happy to<br>
>> review the code once we get to that point.<br>
>><br>
>>> Feature gaps:<br>
>>><br>
>>> Host aggregates<br>
>>> Security groups<br>
>>> Server groups<br>
>>><br>
>>><br>
>>> Shortcomings:<br>
>>><br>
>>> Flavor syncing<br>
>>> This needs to be addressed now.<br>
>>><br>
>>> Cells scheduling/rescheduling<br>
>>> Instances can not currently move between
cells<br>
>>> These two won't affect the default
one cell setup so they will be<br>
>>> addressed later.<br>
>>><br>
>>><br>
>>> What does cells do:<br>
>>><br>
>>> Schedule an instance to a cell based on
flavor slots available.<br>
>>> Proxy API requests to the proper cell.<br>
>>> Keep a copy of instance data at the
global level for quick retrieval.<br>
>>> Sync data up from a child cell to keep
the global level up to date.<br>
>>><br>
>>><br>
>>> Simplifying assumptions:<br>
>>><br>
>>> Cells will be treated as a two level tree
structure.<br>
>>><br>
>> Are we thinking of making this official by
removing code that actually<br>
>> allows cells to be an actual tree of depth N?
I am not sure if doing so<br>
>> would be a win, although it does complicate
the RPC/Messaging/State code<br>
>> a bit, but if it's not being used, even
though a nice generalization,<br>
>> why keep it around?<br>
><br>
> My preference would be to remove that code since
I don't envision anyone<br>
> writing tests to ensure that functionality works
and/or doesn't<br>
> regress. But there's the challenge of not
knowing if anyone is actually<br>
> relying on that behavior. So initially I'm not
creating a specific work<br>
> item to remove it. But I think it needs to be
made clear that it's not<br>
> officially supported and may get removed unless a
case is made for<br>
> keeping it and work is put into testing it.<br>
<br>
</div>
</div>
While I agree that N is a bit interesting, I have seen N=3
in production<br>
<br>
[central API]-->[state/region1]-->[state/region DC1]<br>
\->[state/region DC2]<br>
-->[state/region2 DC]<br>
-->[state/region3 DC]<br>
-->[state/region4 DC]<br>
<div class="HOEnZb">
<div class="h5"><br>
</div>
</div>
</blockquote>
<div>I'm curious. <br>
What are the use cases for this deployment? Agreeably,
root node runs n-api along with horizon, key management
etc. What components are deployed in tier 2 and tier 3?<br>
</div>
<div>And AFAIK, currently, openstack cell deployment isn't
even a tree but DAG since, one cell can have multiple
parents. Has anyone come up any such requirement?<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb">
<div class="h5">
<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
While there's nothing to prevent a cell from having multiple parents
I would be curious to know if this would actually work in practice,
since I can imagine a number of cases that might cause problems.
And is there a practical use for this?<br>
<br>
Maybe we should start logging a warning when this is setup stating
that this is an unsupported(i.e. untested) configuration to start to
codify the design as that of a tree. At least for the initial scope
of work I think this makes sense, and if a case is made for a DAG
setup that can be done independently.<br>
<br>
<blockquote
cite="mid:CAD26Kg0T7D=35msoT6+nYOOLzuAQU7yCTyBQ_XB_-CSGLxtSvw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb">
<div class="h5">
<br>
>><br>
>>> Plan:<br>
>>><br>
>>> Fix flavor breakage in child cell which
causes boot tests to fail.<br>
>>> Currently the libvirt driver needs
flavor.extra_specs which is not<br>
>>> synced to the child cell. Some options
are to sync flavor and extra<br>
>>> specs to child cell db, or pass full data
with the request.<br>
>>> <a moz-do-not-send="true"
href="https://review.openstack.org/#/c/126620/1"
target="_blank">https://review.openstack.org/#/c/126620/1</a>
offers a means of passing full<br>
>>> data with the request.<br>
>>><br>
>>> Determine proper switches to turn off
Tempest tests for features that<br>
>>> don't work with the goal of getting a
voting job. Once this is in place<br>
>>> we can move towards feature parity and
work on internal refactorings.<br>
>>><br>
>>> Work towards adding parity for host
aggregates, security groups, and<br>
>>> server groups. They should be made to
work in a single cell setup, but<br>
>>> the solution should not preclude them
from being used in multiple<br>
>>> cells. There needs to be some discussion
as to whether a host aggregate<br>
>>> or server group is a global concept or
per cell concept.<br>
>>><br>
>> Have there been any previous discussions on
this topic? If so I'd really<br>
>> like to read up on those to make sure I
understand the pros and cons<br>
>> before the summit session.<br>
><br>
> The only discussion I'm aware of is some comments
on<br>
> <a moz-do-not-send="true"
href="https://review.openstack.org/#/c/59101/"
target="_blank">https://review.openstack.org/#/c/59101/</a>
, though they mention a<br>
> discussion at the Utah mid-cycle.<br>
><br>
> The main con I'm aware of for defining these as
global concepts is that<br>
> there is no rescheduling capability in the cells
scheduler. So if a<br>
> build is sent to a cell with a host aggregate
that can't fit that<br>
> instance the build will fail even though there
may be space in that host<br>
> aggregate from a global perspective. That should
be somewhat<br>
> straightforward to address though.<br>
><br>
> I think it makes sense to define these as global
concepts. But these<br>
> are features that aren't used with cells yet so I
haven't put a lot of<br>
> thought into potential arguments or cases for
doing this one way or<br>
> another.<br>
><br>
</div>
</div>
</blockquote>
<div>Keeping aggregates local also poses problem in case
when cells are temporarily dead (out of system). Since top
level doesn't have any idea about local features including
who all to contact for deletion of a particular aggregate.<br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb">
<div class="h5">
><br>
>>> Work towards merging compute/api.py and
compute/cells_api.py so that<br>
>>> developers only need to make
changes/additions in once place. The goal<br>
>>> is for as much as possible to be hidden
by the RPC layer, which will<br>
>>> determine whether a call goes to a
compute/conductor/cell.<br>
>>><br>
>>> For syncing data between cells, look at
using objects to handle the<br>
>>> logic of writing data to the cell/parent
and then syncing the data to<br>
>>> the other.<br>
>>><br>
>> Some of that work has been done already,
although in a somewhat ad-hoc<br>
>> fashion, were you thinking of extending
objects to support this natively<br>
>> (whatever that means), or do we continue to
inline the code in the<br>
>> existing object methods.<br>
><br>
> I would prefer to have some native support for
this. In general data is<br>
> considered authoritative at the global level or
the cell level. For<br>
> example, instance data is synced down from the
global level to a<br>
> cell(except for a few fields which are synced up)
but a migration would<br>
> be synced up. I could imagine decorators that
would specify how data<br>
> should be synced and handle that as transparently
as possible.<br>
><br>
>><br>
>>> A potential migration scenario is to
consider a non cells setup to be a<br>
>>> child cell and converting to cells will
mean setting up a parent cell<br>
>>> and linking them. There are periodic
tasks in place to sync data up<br>
>>> from a child already, but a manual kick
off mechanism will need to be<br>
>>> added.<br>
>>><br>
>>><br>
>>> Future plans:<br>
>>><br>
>>> Something that has been considered, but
is out of scope for now, is that<br>
>>> the parent/api cell doesn't need the same
data model as the child cell.<br>
>>> Since the majority of what it does is act
as a cache for API requests,<br>
>>> it does not need all the data that a cell
needs and what data it does<br>
>>> need could be stored in a form that's
optimized for reads.<br>
>>><br>
>>><br>
>>> Thoughts?<br>
>>><br>
>>>
_______________________________________________<br>
>>> OpenStack-dev mailing list<br>
>>> <a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
>>> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
>>
_______________________________________________<br>
>> OpenStack-dev mailing list<br>
>> <a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
>> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
> <a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>