Open Stack

On 21.12.2015 11:42, Oleg Gelbukh wrote:
> The problem with this approach is that we can't manage the interfaces
> between components without changing the code of 2+ components (i.e. one
> that provides the data and all that consume it).

What code is supposed to be changed in an example LDAP data source?
Otherwise, the only place I see shall require code changes is a data
processor for it, if one may want to support new (or very old) version
of the ldap with another database schema.

> 
> I also don't like polling model for data processors. In my
> understanding, components should push their changes through the
> pipeline. Although this is pure implementation detail and is not really
> important ATM.

What do you think the example LDAP data source shall push? IIUC, for
most auth cases, it shall be only pulled and processed.

> 
> The point is that for Solar integration, we still need integration
> points, and the less of them we have, the more simple the transition is
> going to be..

Data processors may be the only integration point, as they are supposed
to "know" how-to get and how-to shape data from sources

> 
> --
> Best regards,
> Oleg Gelbukh
> 
> On Mon, Dec 21, 2015 at 11:32 AM, Evgeniy L <eli at mirantis.com
> <mailto:eli at mirantis.com>> wrote:
> 
>     Hi Oleg,
> 
>     I understand the concern, but in case of integration specifically
>     with Solar,
>     I don't see any reasons to add ConfigDB, because Solar by itself is
>     a ConfigDB.
>     At the same time I would agree that there might be a case, when user
>     uses
>     Zookeeper/Puppet Master/CMDB as a data store, in this case we should
>     store
>     the data directly in those services, without keeping them in yet
>     another storage.
> 
>     So the flow will look like this:
>     Components ->
>     Data get polled by data processors ->
>     | Solar data processor puts the data into Solar in its format
>     | Zookeeper data processor puts the data into Zookeeper in its format
>     | Custom CMDB data processor puts the data into CMDB in its own format
> 
>     Thanks,
> 
>     On Fri, Dec 18, 2015 at 7:00 PM, Oleg Gelbukh <ogelbukh at mirantis.com
>     <mailto:ogelbukh at mirantis.com>> wrote:
> 
>         Hi,
> 
>         The idea behind configdb is that it is independent component
>         that defines data flows between other components of the system.
>         It has no knowledge about those components or specifics of their
>         data. Data formats are defined by components themselves via
>         schemas/templates and can be changed at any time (i.e. don't
>         require code changes).
> 
>         Important 'pro' having ConfigDB separate from Solar is that it
>         will simplify transition from current Fuel architecture by
>         breaking it into more definite stages and reducing the number of
>         components Solar have to be integrated with.
> 
>         --
>         Best regards,
>         Oleg Gelbukh
> 
>         On Wed, Dec 16, 2015 at 4:38 PM, Evgeniy L <eli at mirantis.com
>         <mailto:eli at mirantis.com>> wrote:
> 
>             Hi Dmitry,
> 
>             I also don't think that we should duplicate the data in
>             configdb,
>             because in this case there will be +2 additional interfaces
>             which
>             will require to covert the data into configdb and after that
>             from
>             configdb to Solar, which seems redundant overhead.
> 
>             But we should be able to put the data directly to user's
>             CMDB/ZooKeeper/Puppet Master/etc.
> 
>             Thanks,
> 
>             On Wed, Dec 16, 2015 at 2:03 AM, Dmitriy Shulyak
>             <dshulyak at mirantis.com <mailto:dshulyak at mirantis.com>> wrote:
> 
>                 Hello folks,
> 
>                 This topic is about configuration storage which will
>                 connect data sources (nailgun/bareon/others) and
>                 orchestration. And right now we are developing two
>                 projects that will overlap a bit.
> 
>                 I understand there is not enough context to dive into
>                 this thread right away, but i will appreciate if those
>                 people, who participated in design, will add their
>                 opinions/clarifications on this matter.
> 
>                 Main disagreements
>                 ---------------------------
>                 1. configdb should be passive, writing to configdb is
>                 someone else responsibility
>                 + simpler implementation, easier to use
>                 - we will need another component that will do writing,
>                 or split this responsibility somehow
> 
>                 2. can be used without other solar components
>                 + clear inteface between solar components and storage layer
>                 - additional work required to design/refactor
>                 communication layer between modules in solar
>                 - some data will be duplicated between solar
>                 orchestrator layer and configdb
> 
>                 3. templates for output
>                 technical detail, can be added on top of solardb if required
> 
>                 Similar functionality
>                 --------------------------
>                 1. Hierachical storage
>                 2. Versioning of changes
>                 3. Possibility to overwrite config values
>                 4. Schema for inputs
> 
>                 Overall it seems that we share same goals for both services,
>                 the difference lies in organizational and technical
>                 implementation details.
> 
>                 Possible solutions
>                 ------------------------
>                 1. develop configdb and solar with duplicated functionality
>                 - at least 2 additional components will be added to the
>                 picture,
>                 one is configdb, another one will need to sync data
>                 between configdb and solar
>                 - in some cases data in solar and configdb will be 100%
>                 duplicated
>                 - different teams will work on same functionality
>                 - integration of additional component for fuel will
>                 require integration with
>                 configdb and with solar
>                 + configdb will be independent from solar
>                 orchestration/other components
> 
>                 2. make service out of solardb, allign with configdb use
>                 cases
>                 + solardb will be independent from solar
>                 orchestration/other solar components
>                 + integration of fuel component will be easier than in
>                 1st version
>                 + clarity about components responsibility and new
>                 architecture
>                 - redesign/refactoring communication between components
>                 in solar
> 
>                 3. do not use configdb/no extraction of solardb
>                 - inproc communication, which can lead to coupled
>                 components (not the case currently)
>                 + faster implementation (no major changes required for
>                 integration with fuel)
>                 + clarity about components responsibility and new
>                 architecture
> 
>                 Summary
>                 -------------
>                 For solar it makes no difference where data will come
>                 from: configdb or
>                 data sources, but in overall fuel architecture it will
>                 lead to significant
>                 complexity increase.
>                 It would be the best to follow 2nd path, because in long
>                 term we don't want tightly coupled components, but in
>                 nearest future we need to concentrate
>                 on:
>                 - integration with fuel
>                 - implementing policy engine
>                 - polishing solar components
>                 This is why i am not sure that we can spend time on 2nd
>                 path right now,
>                 or even before 9.0.
> 
> 
> 
>                 __________________________________________________________________________
>                 OpenStack Development Mailing List (not for usage questions)
>                 Unsubscribe:
>                 OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>                 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>                 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
>             __________________________________________________________________________
>             OpenStack Development Mailing List (not for usage questions)
>             Unsubscribe:
>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
>         __________________________________________________________________________
>         OpenStack Development Mailing List (not for usage questions)
>         Unsubscribe:
>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>         <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando