Open Stack

Hi Germy,

It seems that you're looking at solutions for ensuring consistency between
the "desired" configuration (Neutron), and the actual one (whatever is in
your backend) at startup.
This has been discussed several times in the past - not just for
synchronization at startup, but also for ensuring neutron and the backend
are in sync at each operation.

At a very high level I think a "general" solution is only partially
possible. At some point there must be a plugin interface that verifies
whether, for a given resource, data on the backend differ from those in
neutron.
The component which evaluates the result of such operation and updates the
status of the resources being synchronised could instead be shared across
plugins.
For the ML2 plugin I don't see any architectural difference, beyond the
fact that the plugin level operation should probably query all the
mechanism drivers.

Anyway, If this is something you'd like to see implemented (regardless of
whether my analysis matches your use case) you should considering filing a
RFE bug so that it will be considered during the drivers meetings.

Salvatore

On 14 October 2015 at 11:43, Germy Lure <germy.lure at gmail.com> wrote:

> Hi Salvatore and Kevin,
>
> I'm sorry for replying so late.
> I wanted to see whether the community had considered data sync for these
> two style(agent and controller) integration. To solve integrating multiple
> vendor's controllers, I need some help from community. That's the original
> purpose of this thread. In another word, I had no idea when I sent this
> message and I just asked some help.
>
> Anyway, the issues I mentioned last mail are exists. We still need face
> them. I have some rough ideas for your reference.
>
> 1.try best to keep the source is correct.
> Think about CREATE operation, if the backend was be in exception and
> Neutron is timeout, then the record should be destroyed or marked ERROR to
> warn the operator. If Neutron was be in exception, the backend will has an
> extra record. To avoid this, Neutron could store and mark a record
> CREATE_PENDING before push it to backend, then scan data and check with the
> backend after restarting when exception occurs. If the record in Neutron is
> extra, destroy or mark ERROR to warn the operator. UPDATE and DELETE need
> similar logic.
> Currently in Neutron, some objects have defined XX_PENDING and some not.
> 2.check each other when they restart.
> After restarting, the backend should report the states of all objects and
> may re-load data from Neutron to rebuild or check local data. When Neutron
> restarting, it should get data from backend and check it. Maybe, it can
> notify backend, and backend act as it just restarted.
> All in all, I think it's enough that keeping the data be correct when you
> write(CUD) it and check it when restarting.
>
> About implementation, I think a common frame is best. Plugins or even
> drivers just provide methods for backend to load data, update state and
> etc.
>
> As I mentioned earlier, this is just a rough and superficial idea. Any
> comment please.
>
> Thanks,
> Germy
> .
>
>
>
> On Tue, Oct 13, 2015 at 3:28 AM, Kevin Benton <blak111 at gmail.com> wrote:
>
>> >*But there is no such a feature in Neutron. Right? Will the community
>> merge it soon? And can we consider it with agent-style mechanism together?*
>>
>> The agents have their own mechanisms for getting information from the
>> server. The community has no plans to merge a feature that is going to be
>> different for almost every vendor.
>>
>> We tried to come up with some common syncing stuff in the recent ML2
>> meeting, the various backends had different methods of detecting when they
>> were out of sync with Neutron (e.g. headers in hashes, recording errors,
>> etc), all of which depended on the capabilities of the backend. Then the
>> sync method itself was different between backends (sending deltas, sending
>> entire state, sending a replay log, etc).
>>
>> About the only thing they have in common is that they need a way detect
>> if they are out of sync and they need a method to sync. So that's two
>> abstract methods, and we likely can't even agree on when they should be
>> called.
>>
>> Echoing Salvatore's comments, what is it that you want to see?
>>
>> On Mon, Oct 12, 2015 at 12:29 AM, Germy Lure <germy.lure at gmail.com>
>> wrote:
>>
>>> Hi Kevin,
>>>
>>> *Thank you for your response. Periodic data checking is a popular and
>>> effective method to sync info. But there is no such a feature in Neutron.
>>> Right? Will the community merge it soon? And can we consider it with
>>> agent-style mechanism together?*
>>>
>>> Vendor-specific extension or coding a periodic task private by vendor is
>>> not a good solution, I think. Because it means that Neutron-Sever could not
>>> integrate with multiple vendors' controller and even the controller of
>>> those vendors that introduced this extension or task could not integrate
>>> with a standard community Neutron-Server.
>>> That is just the tip of the iceberg. Many of the other problems
>>> resulting, such as fixing bugs,upgrade,patch and etc.
>>> But wait, is it a vendor-specific feature? Of course not. All software
>>> systems need data checking.
>>>
>>> Many thanks.
>>> Germy
>>>
>>>
>>> On Sun, Oct 11, 2015 at 4:28 PM, Kevin Benton <blak111 at gmail.com> wrote:
>>>
>>>> You can have a periodic task that asks your backend if it needs sync
>>>> info.
>>>> Another option is to define a vendor-specific extension that makes it
>>>> easy to retrieve all info in one call via the HTTP API.
>>>>
>>>> On Sat, Oct 10, 2015 at 2:24 AM, Germy Lure <germy.lure at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> After restarting, Agents load data from Neutron via RPC. What about
>>>>> 3-rd controller? They only can re-gather data via NBI. Right?
>>>>>
>>>>> Is it possible to provide some mechanism for those controllers and
>>>>> agents to sync data? or something else I missed?
>>>>>
>>>>> Thanks
>>>>> Germy
>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Kevin Benton
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Kevin Benton
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151015/47c47777/attachment.html>