Open Stack

Hi Salvatore,

Thanks so much for your continuing answers. What you mentioned is partially
where I was going. Indeed, I want to solve the whole consistency issue, not
just at startup. I just thought that ensuring consistency at startup and
each operation is enough for it. A periodic task need more resource.

OK, anyway, I would think about filing a bug or BP to push this moving
forward.
Thanks for the help. Will get back to you if required.

Germy
.



On Fri, Oct 16, 2015 at 1:46 AM, Salvatore Orlando <salv.orlando at gmail.com>
wrote:

> Hi Germy,
>
> It seems that you're looking at solutions for ensuring consistency between
> the "desired" configuration (Neutron), and the actual one (whatever is in
> your backend) at startup.
> This has been discussed several times in the past - not just for
> synchronization at startup, but also for ensuring neutron and the backend
> are in sync at each operation.
>
> At a very high level I think a "general" solution is only partially
> possible. At some point there must be a plugin interface that verifies
> whether, for a given resource, data on the backend differ from those in
> neutron.
> The component which evaluates the result of such operation and updates the
> status of the resources being synchronised could instead be shared across
> plugins.
> For the ML2 plugin I don't see any architectural difference, beyond the
> fact that the plugin level operation should probably query all the
> mechanism drivers.
>
> Anyway, If this is something you'd like to see implemented (regardless of
> whether my analysis matches your use case) you should considering filing a
> RFE bug so that it will be considered during the drivers meetings.
>
> Salvatore
>
> On 14 October 2015 at 11:43, Germy Lure <germy.lure at gmail.com> wrote:
>
>> Hi Salvatore and Kevin,
>>
>> I'm sorry for replying so late.
>> I wanted to see whether the community had considered data sync for these
>> two style(agent and controller) integration. To solve integrating multiple
>> vendor's controllers, I need some help from community. That's the original
>> purpose of this thread. In another word, I had no idea when I sent this
>> message and I just asked some help.
>>
>> Anyway, the issues I mentioned last mail are exists. We still need face
>> them. I have some rough ideas for your reference.
>>
>> 1.try best to keep the source is correct.
>> Think about CREATE operation, if the backend was be in exception and
>> Neutron is timeout, then the record should be destroyed or marked ERROR to
>> warn the operator. If Neutron was be in exception, the backend will has an
>> extra record. To avoid this, Neutron could store and mark a record
>> CREATE_PENDING before push it to backend, then scan data and check with the
>> backend after restarting when exception occurs. If the record in Neutron is
>> extra, destroy or mark ERROR to warn the operator. UPDATE and DELETE need
>> similar logic.
>> Currently in Neutron, some objects have defined XX_PENDING and some not.
>> 2.check each other when they restart.
>> After restarting, the backend should report the states of all objects and
>> may re-load data from Neutron to rebuild or check local data. When Neutron
>> restarting, it should get data from backend and check it. Maybe, it can
>> notify backend, and backend act as it just restarted.
>> All in all, I think it's enough that keeping the data be correct when you
>> write(CUD) it and check it when restarting.
>>
>> About implementation, I think a common frame is best. Plugins or even
>> drivers just provide methods for backend to load data, update state and
>> etc.
>>
>> As I mentioned earlier, this is just a rough and superficial idea. Any
>> comment please.
>>
>> Thanks,
>> Germy
>> .
>>
>>
>>
>> On Tue, Oct 13, 2015 at 3:28 AM, Kevin Benton <blak111 at gmail.com> wrote:
>>
>>> >*But there is no such a feature in Neutron. Right? Will the community
>>> merge it soon? And can we consider it with agent-style mechanism together?*
>>>
>>> The agents have their own mechanisms for getting information from the
>>> server. The community has no plans to merge a feature that is going to be
>>> different for almost every vendor.
>>>
>>> We tried to come up with some common syncing stuff in the recent ML2
>>> meeting, the various backends had different methods of detecting when they
>>> were out of sync with Neutron (e.g. headers in hashes, recording errors,
>>> etc), all of which depended on the capabilities of the backend. Then the
>>> sync method itself was different between backends (sending deltas, sending
>>> entire state, sending a replay log, etc).
>>>
>>> About the only thing they have in common is that they need a way detect
>>> if they are out of sync and they need a method to sync. So that's two
>>> abstract methods, and we likely can't even agree on when they should be
>>> called.
>>>
>>> Echoing Salvatore's comments, what is it that you want to see?
>>>
>>> On Mon, Oct 12, 2015 at 12:29 AM, Germy Lure <germy.lure at gmail.com>
>>> wrote:
>>>
>>>> Hi Kevin,
>>>>
>>>> *Thank you for your response. Periodic data checking is a popular and
>>>> effective method to sync info. But there is no such a feature in Neutron.
>>>> Right? Will the community merge it soon? And can we consider it with
>>>> agent-style mechanism together?*
>>>>
>>>> Vendor-specific extension or coding a periodic task private by vendor
>>>> is not a good solution, I think. Because it means that Neutron-Sever could
>>>> not integrate with multiple vendors' controller and even the controller of
>>>> those vendors that introduced this extension or task could not integrate
>>>> with a standard community Neutron-Server.
>>>> That is just the tip of the iceberg. Many of the other problems
>>>> resulting, such as fixing bugs,upgrade,patch and etc.
>>>> But wait, is it a vendor-specific feature? Of course not. All software
>>>> systems need data checking.
>>>>
>>>> Many thanks.
>>>> Germy
>>>>
>>>>
>>>> On Sun, Oct 11, 2015 at 4:28 PM, Kevin Benton <blak111 at gmail.com>
>>>> wrote:
>>>>
>>>>> You can have a periodic task that asks your backend if it needs sync
>>>>> info.
>>>>> Another option is to define a vendor-specific extension that makes it
>>>>> easy to retrieve all info in one call via the HTTP API.
>>>>>
>>>>> On Sat, Oct 10, 2015 at 2:24 AM, Germy Lure <germy.lure at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> After restarting, Agents load data from Neutron via RPC. What about
>>>>>> 3-rd controller? They only can re-gather data via NBI. Right?
>>>>>>
>>>>>> Is it possible to provide some mechanism for those controllers and
>>>>>> agents to sync data? or something else I missed?
>>>>>>
>>>>>> Thanks
>>>>>> Germy
>>>>>>
>>>>>>
>>>>>>
>>>>>> __________________________________________________________________________
>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>> Unsubscribe:
>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Kevin Benton
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Kevin Benton
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151016/be466e99/attachment.html>