[openstack-dev] [Group-based Policy] Database migration chain

Ivar Lazzaro ivarlazzaro at gmail.com
Fri Oct 10 22:21:44 UTC 2014


>
> It seems to me that deployment tools such as puppet scripts would also be
> simpler if the GBP service plugin used the neutron DB, as there would be no
> need to create a separate DB, set its permissions, put its URL into
> neutron's config file, etc.. All that would be needed at deployment time is
> to run the additional gbp-db-manage tool to perform the GBP DB migrations.
> Am I missing anything?
>
>
That's correct, being a new schema it needs to be created, the access
granted, and the connection URL configured in neutron.conf (doable also
with devstack in the extra-conf option).


> With dependencies only in one direction, and the foreign keys GBP depends
> on (neutron resource IDs) unlikely to be changed by neutron migrations
> during Kilo, I don't think we need to worry about interleaving GBP
> migrations with neutron migrations. On initial deployments or version
> upgrades, it should be sufficient to run gbp-db-manage after
> neutron-db-manage. On downgrades, some situations might require running
> gbp-db-manage before neutron-db-manage. This seems not to be effected by
> whether the two migration chains are in the same DB/schema or different
> ones.
>

That's correct too. Also, this is true for the downgrade even with a
different schema.


> In addition to compatibility and performance, I'm also concerned about DB
> connection management when the same server is using multiple connection
> URLs. I'm not convinced the approach in the patch is sufficient. At least
> with some DBs, wouldn't we we need a separate sqlalchemy.create_engine()
> call with each DB's connection URL, which might require using separate
> context and session objects rather than the ones neutron uses?


That should not be the case. We are not using a separate DB connection. The
connection is in fact the same, the only thing that changes is the schema.
For the sql engine this  should be only a matter of namespacing (different
schema -> different namespace), therefore I don't see any relevant
performance/connection/greenthread issue.
However I'm no DBMS expert, any additional feedback on this matter is
welcome.

Ivar.

On Fri, Oct 10, 2014 at 2:59 PM, Robert Kukura <kukura at noironetworks.com>
wrote:

>
> On 10/7/14 6:36 PM, Ivar Lazzaro wrote:
>
> I posted a patch that implements the "Different DB Different Chain"
> approach [0].
> That does not mean that this approach is the chosen one! It's just to have
> a grasp of what the change looks like.
>
>  The "Same DB different chain" solution is much simpler to implement
> (basically you just specify a different version table in the alembic
> environment) so I haven't posted anything for that.
>
>  One thing I'm particularly interested about is to hear packagers
> opinions about which approach would be the preferred one: Same DB or
> Different?
>
> It seems to me that deployment tools such as puppet scripts would also be
> simpler if the GBP service plugin used the neutron DB, as there would be no
> need to create a separate DB, set its permissions, put its URL into
> neutron's config file, etc.. All that would be needed at deployment time is
> to run the additional gbp-db-manage tool to perform the GBP DB migrations.
> Am I missing anything?
>
> With dependencies only in one direction, and the foreign keys GBP depends
> on (neutron resource IDs) unlikely to be changed by neutron migrations
> during Kilo, I don't think we need to worry about interleaving GBP
> migrations with neutron migrations. On initial deployments or version
> upgrades, it should be sufficient to run gbp-db-manage after
> neutron-db-manage. On downgrades, some situations might require running
> gbp-db-manage before neutron-db-manage. This seems not to be effected by
> whether the two migration chains are in the same DB/schema or different
> ones.
>
>  Also, on the line of Bob's comment in my patch, is there any kind of
> compatibility or performance issue anyone is aware of about in using cross
> schema FKs?
>
> In addition to compatibility and performance, I'm also concerned about DB
> connection management when the same server is using multiple connection
> URLs. I'm not convinced the approach in the patch is sufficient. At least
> with some DBs, wouldn't we we need a separate sqlalchemy.create_engine()
> call with each DB's connection URL, which might require using separate
> context and session objects rather than the ones neutron uses?
>
> -Bob
>
>
>  Thanks,
> Ivar.
>
>  [0] https://review.openstack.org/#/c/126383/
>
> On Mon, Oct 6, 2014 at 11:09 AM, Ivar Lazzaro <ivarlazzaro at gmail.com>
> wrote:
>
>>  I believe Group-based Policy (which this thread is about) will use the
>>> Neutron
>>> database configuration for its dependent database.
>>>
>>> If Neutron is configured for:
>>>   connection = mysql://user:pass@locationX:3306/neutron
>>> then GBP would use:
>>>   connection = mysql://user:pass@locationX:3306/neutron_gbp
>>
>>
>>  That's correct, that would be the likely approach if we go with the
>> "different schema" route.
>>
>>  if you can get the “other database” to be accessible from the target
>>> database via “otherdatabase.sometable”, then you’re in.
>>> from SQLAlchemy’s perspective, it’s just a name with a dot.   It’s the
>>> database itself that has to support the foreign key at the scope you are
>>> shooting for.
>>>
>>
>>  I'm experimenting this approach with our code and it seems to be the
>> case. '
>> I feel that having the constraint of pointing the same database
>> connection with a different schema is pretty acceptable given how tight is
>> GBP to Neutron.
>>
>>
>> On Sat, Oct 4, 2014 at 8:54 AM, Henry Gessau <gessau at cisco.com> wrote:
>>
>>> Clint Byrum <clint at fewbar.com> wrote:
>>> >
>>> > Excerpts from Mike Bayer's message of 2014-10-04 08:10:38 -0700:
>>> >>
>>> >> On Oct 4, 2014, at 1:10 AM, Kevin Benton <blak111 at gmail.com> wrote:
>>> >>
>>> >>> Does sqlalchemy have good support for cross-database foreign keys? I
>>> was under the impression that they cannot be implemented with the normal
>>> syntax and semantics of an intra-database foreign-key constraint.
>>> >>
>>> >> cross “database” is not typically portable, but cross “schema” is.
>>> >>
>>> >> different database vendors have different notions of “databases” or
>>> “schemas”.
>>> >>
>>> >> if you can get the “other database” to be accessible from the target
>>> database via “otherdatabase.sometable”, then you’re in.
>>> >>
>>> >> from SQLAlchemy’s perspective, it’s just a name with a dot.   It’s
>>> the database itself that has to support the foreign key at the scope you
>>> are shooting for.
>>> >>
>>> >
>>> > All true, however, there are zero guarantees that databases will be
>>> > hosted on the same server, and typically permissions are setup to
>>> prevent
>>> > cross-schema joins.
>>>
>>> I believe Group-based Policy (which this thread is about) will use the
>>> Neutron
>>> database configuration for its dependent database.
>>>
>>> If Neutron is configured for:
>>>   connection = mysql://user:pass@locationX:3306/neutron
>>> then GBP would use:
>>>   connection = mysql://user:pass@locationX:3306/neutron_gbp
>>>
>>> > Typically we use the public API's when we want to access data in a
>>> > different application. The database is a private implementation detail
>>> > of each application.
>>>
>>> Currently GPB is very tightly coupled to Neutron.
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>
>
> _______________________________________________
> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141010/f65491b3/attachment.html>


More information about the OpenStack-dev mailing list