Open Stack

The problem with this approach is that we can't manage the interfaces
between components without changing the code of 2+ components (i.e. one
that provides the data and all that consume it).

I also don't like polling model for data processors. In my understanding,
components should push their changes through the pipeline. Although this is
pure implementation detail and is not really important ATM.

The point is that for Solar integration, we still need integration points,
and the less of them we have, the more simple the transition is going to
be..

--
Best regards,
Oleg Gelbukh

On Mon, Dec 21, 2015 at 11:32 AM, Evgeniy L <eli at mirantis.com> wrote:

> Hi Oleg,
>
> I understand the concern, but in case of integration specifically with
> Solar,
> I don't see any reasons to add ConfigDB, because Solar by itself is a
> ConfigDB.
> At the same time I would agree that there might be a case, when user uses
> Zookeeper/Puppet Master/CMDB as a data store, in this case we should store
> the data directly in those services, without keeping them in yet another
> storage.
>
> So the flow will look like this:
> Components ->
> Data get polled by data processors ->
> | Solar data processor puts the data into Solar in its format
> | Zookeeper data processor puts the data into Zookeeper in its format
> | Custom CMDB data processor puts the data into CMDB in its own format
>
> Thanks,
>
> On Fri, Dec 18, 2015 at 7:00 PM, Oleg Gelbukh <ogelbukh at mirantis.com>
> wrote:
>
>> Hi,
>>
>> The idea behind configdb is that it is independent component that defines
>> data flows between other components of the system. It has no knowledge
>> about those components or specifics of their data. Data formats are defined
>> by components themselves via schemas/templates and can be changed at any
>> time (i.e. don't require code changes).
>>
>> Important 'pro' having ConfigDB separate from Solar is that it will
>> simplify transition from current Fuel architecture by breaking it into more
>> definite stages and reducing the number of components Solar have to be
>> integrated with.
>>
>> --
>> Best regards,
>> Oleg Gelbukh
>>
>> On Wed, Dec 16, 2015 at 4:38 PM, Evgeniy L <eli at mirantis.com> wrote:
>>
>>> Hi Dmitry,
>>>
>>> I also don't think that we should duplicate the data in configdb,
>>> because in this case there will be +2 additional interfaces which
>>> will require to covert the data into configdb and after that from
>>> configdb to Solar, which seems redundant overhead.
>>>
>>> But we should be able to put the data directly to user's
>>> CMDB/ZooKeeper/Puppet Master/etc.
>>>
>>> Thanks,
>>>
>>> On Wed, Dec 16, 2015 at 2:03 AM, Dmitriy Shulyak <dshulyak at mirantis.com>
>>> wrote:
>>>
>>>> Hello folks,
>>>>
>>>> This topic is about configuration storage which will connect data
>>>> sources (nailgun/bareon/others) and orchestration. And right now we are
>>>> developing two projects that will overlap a bit.
>>>>
>>>> I understand there is not enough context to dive into this thread right
>>>> away, but i will appreciate if those people, who participated in design,
>>>> will add their opinions/clarifications on this matter.
>>>>
>>>> Main disagreements
>>>> ---------------------------
>>>> 1. configdb should be passive, writing to configdb is someone else
>>>> responsibility
>>>> + simpler implementation, easier to use
>>>> - we will need another component that will do writing, or split this
>>>> responsibility somehow
>>>>
>>>> 2. can be used without other solar components
>>>> + clear inteface between solar components and storage layer
>>>> - additional work required to design/refactor communication layer
>>>> between modules in solar
>>>> - some data will be duplicated between solar orchestrator layer and
>>>> configdb
>>>>
>>>> 3. templates for output
>>>> technical detail, can be added on top of solardb if required
>>>>
>>>> Similar functionality
>>>> --------------------------
>>>> 1. Hierachical storage
>>>> 2. Versioning of changes
>>>> 3. Possibility to overwrite config values
>>>> 4. Schema for inputs
>>>>
>>>> Overall it seems that we share same goals for both services,
>>>> the difference lies in organizational and technical implementation
>>>> details.
>>>>
>>>> Possible solutions
>>>> ------------------------
>>>> 1. develop configdb and solar with duplicated functionality
>>>> - at least 2 additional components will be added to the picture,
>>>> one is configdb, another one will need to sync data between configdb
>>>> and solar
>>>> - in some cases data in solar and configdb will be 100% duplicated
>>>> - different teams will work on same functionality
>>>> - integration of additional component for fuel will require integration
>>>> with
>>>> configdb and with solar
>>>> + configdb will be independent from solar orchestration/other components
>>>>
>>>> 2. make service out of solardb, allign with configdb use cases
>>>> + solardb will be independent from solar orchestration/other solar
>>>> components
>>>> + integration of fuel component will be easier than in 1st version
>>>> + clarity about components responsibility and new architecture
>>>> - redesign/refactoring communication between components in solar
>>>>
>>>> 3. do not use configdb/no extraction of solardb
>>>> - inproc communication, which can lead to coupled components (not the
>>>> case currently)
>>>> + faster implementation (no major changes required for integration with
>>>> fuel)
>>>> + clarity about components responsibility and new architecture
>>>>
>>>> Summary
>>>> -------------
>>>> For solar it makes no difference where data will come from: configdb or
>>>> data sources, but in overall fuel architecture it will lead to
>>>> significant
>>>> complexity increase.
>>>> It would be the best to follow 2nd path, because in long term we don't
>>>> want tightly coupled components, but in nearest future we need to
>>>> concentrate
>>>> on:
>>>> - integration with fuel
>>>> - implementing policy engine
>>>> - polishing solar components
>>>> This is why i am not sure that we can spend time on 2nd path right now,
>>>> or even before 9.0.
>>>>
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151221/241a605c/attachment.html>