Open Stack

On 03/23/2016 01:33 AM, Vega Cai wrote:
>
>
> On 22 March 2016 at 12:09, Shinobu Kinjo <shinobu.kj at gmail.com
> <mailto:shinobu.kj at gmail.com>> wrote:
>
>     Thank you for your comment (inline for my message).
>
>     On Tue, Mar 22, 2016 at 11:53 AM, Vega Cai <luckyvega.g at gmail.com
>     <mailto:luckyvega.g at gmail.com>> wrote:
>     > Let me try to explain some.
>     >
>     > On 22 March 2016 at 10:09, Shinobu Kinjo <shinobu.kj at gmail.com <mailto:shinobu.kj at gmail.com>> wrote:
>     >>
>     >> On Tue, Mar 22, 2016 at 10:22 AM, joehuang <joehuang at huawei.com <mailto:joehuang at huawei.com>> wrote:
>     >> > Hello, Shinobu,
>     >> >
>     >> > Yes, as what you described here, the "initialize" in "core.py" is used
>     >> > for unit/function test only. For system integration test( for example,
>     >> > tempest ), it would be better to use mysql like DB, this is done by the
>     >> > configuration in DB part.
>     >>
>     >> Thank you for your thought.
>     >>
>     >> >
>     >> > From my point of view, the tricircle DB part could be enhanced in the DB
>     >> > model and migration scripts. Currently unit test use DB model to initialize
>     >> > the data base, but not using the migration scripts,
>     >>
>     >> I'm assuming the migration scripts are in "tricircle/db". Is it right?
>     >
>     >
>     > migration scripts are in tricircle/db/migrate_repo
>     >>
>     >>
>     >> What is the DB model?
>     >> Why do we need 2-way-methods at the moment?
>     >
>     >
>     > DB models are defined in tricircle/db/models.py. Models.py defines tables in
>     > object level, so other modules can import models.py then operate the tables
>     > by operating the objects. Migration scripts defines tables in table level,
>     > you define table fields, constraints in the scripts then migration tool will
>     > read the scripts and build the tables.
>
>     Dose "models.py" manage database schema(e.g., create / delete columns,
>     tables, etc)?
>
>
> In "models.py" we only define database schema. SQLAlchemy provides
> functionality to create tables based on schema definition, which is
> "ModelBase.metadata.create_all". This is used to initialized the
> in-memory database for tests currently.

FTR this is the best way to do this.   SQLite's migration patterns are 
entirely different than for any other database, so while Alembic has a 
"batch" mode that can provide some level of code-compatibility (with 
many caveats, difficulties, and dead-end cases) between a SQLite 
migration and a migration for all the other databases, it is far 
preferable to not use any migration pattern at all for the SQLite 
database and just do a create_all().  It's also much faster, especially 
in the SQLite case where migrations require that the whole table is 
dropped and re-created for most changes.




>
>
>     > Migration tool has a feature to
>     > generate migration scripts from DB models automatically but it may make
>     > mistakes sometimes, so currently we manually maintain the table structure in
>     > both DB model and migration scripts.
>
>     Is *migration tool* different from bot DB models and migration scripts?
>
>
> Migration tool is Alembic, a lightweight database migration tool for
> usage of SQLAlchemy:
>
> https://alembic.readthedocs.org/en/latest/
>
> It runs migration scripts to update database schema. Each database
> version has one migrate script. After defining "upgrade" and "downgrade"
> method in the script, you can update your database from one version to
> another version. Alembic isn't aware of DB models defined in
> "models.py", users need to guarantee the version of database and the
> version of "models.py" match.
>
> If you create a new database, both "ModelBase.metadata.create_all" and
> Alembic can be used. But Alembic can also be used to update an existing
> database to a specific version of schema.
>
>
>      >>
>      >>
>      >> > so the migration scripts can only be tested when using
>     devstack for
>      >> > integration test. It would better to using migration script to
>     instantiate
>      >> > the DB, and tested in the unit test too.
>      >>
>      >> If I understand you correctly, we are moving forward to using the
>      >> migration scripts for both unit and integration tests.
>      >>
>      >> Cheers,
>      >> Shinobu
>      >>
>      >> >
>      >> > (Also move the discussion to the openstack-dev mail-list)
>      >> >
>      >> > Best Regards
>      >> > Chaoyi Huang ( joehuang )
>      >> >
>      >> > -----Original Message-----
>      >> > From: Shinobu Kinjo [mailto:skinjo at redhat.com
>     <mailto:skinjo at redhat.com>]
>      >> > Sent: Tuesday, March 22, 2016 7:43 AM
>      >> > To: joehuang; khayam.gondal; zhangbinsjtu; shipengfei92; newypei;
>      >> > Liuhaixia; caizhiyuan (A); huangzhipeng
>      >> > Subject: Using in-memory database for unit tests
>      >> >
>      >> > Hello,
>      >> >
>      >> > In "initialize" method defined in "core.py", we're using
>     *in-memory*
>      >> > strategy making use of sqlite. AFAIK we are using this
>     solution for only
>      >> > testing purpose. Unit tests using this solution should be fine
>     for small
>      >> > scale environment. But it's not good enough even it's for testing.
>      >> >
>      >> > What do you think?
>      >> > Any thought, suggestion would be appreciated.
>      >> >
>      >> > [1]
>      >> >
>     https://github.com/openstack/tricircle/blob/master/tricircle/db/core.py#L124-L127
>      >> >
>      >> > Cheers,
>      >> > Shinobu
>      >> >
>      >> >
>     __________________________________________________________________________
>      >> > OpenStack Development Mailing List (not for usage questions)
>      >> > Unsubscribe:
>      >> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>      >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>      >>
>      >>
>      >>
>      >> --
>      >> Email:
>      >> shinobu at linux.com <mailto:shinobu at linux.com>
>      >> GitHub:
>      >> shinobu-x
>      >> Blog:
>      >> Life with Distributed Computational System based on OpenSource
>      >>
>      >>
>     __________________________________________________________________________
>      >> OpenStack Development Mailing List (not for usage questions)
>      >> Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>      >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>      >
>      >
>      >
>      >
>     __________________________________________________________________________
>      > OpenStack Development Mailing List (not for usage questions)
>      > Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>      > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>      >
>
>
>
>     --
>     Email:
>     shinobu at linux.com <mailto:shinobu at linux.com>
>     GitHub:
>     shinobu-x
>     Blog:
>     Life with Distributed Computational System based on OpenSource
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>