Open Stack

On 08/18/2013 11:07 PM, Robert Collins wrote:
> On 19 August 2013 14:22, Jay Pipes <jaypipes at gmail.com> wrote:
>
>>> I'm completely with Joshua here - the ORM layer is more often than not
>>> a source of bugs and performance issues.
>>
>> If used improperly, yep.
>
> http://www.codinghorror.com/blog/2006/06/object-relational-mapping-is-the-vietnam-of-computer-science.html
>
> There is no proper use of an ORM.

I'm not a super-fan of ORMs, Robert. I'm not sure why you're insisting 
on taking me down this road...

>>> We don't use the SQLAlchemy ORM for cross-SQL-DB support - thats a
>>> lower layer. It's the model objects themselves that we use the ORM
>>> for, and we could use SQLAlchemy's lower layers but not the ORM.
>>
>> Hmmm, not quite... see below.
>
>>> An alternative I think would be better would be to scrap the use of
>>> the SQLAlchemy ORM; keep using the DB engine abstraction support.
>>
>> Just keep in mind that the Session and Query objects and their related APIs
>> are in the SQLAlchemy ORM, not the SQLAlchemy Core.
>
> Ok, so either it's not a bright line, or we'd need to have an
> alternative thing - not just a reimplementation either, cause that's
> pointless.

All I'm saying is that we should be careful not to swap one set of 
problems for another. I say this because I've seen the Nova data-access 
code develop from its very earliest days, up to this point. I've seen 
the horrors of trying to mask an object approach on top of a 
non-relational data store, witnessed numerous attempts to rewrite the 
way that connection pooling and session handling is done, and in general 
just noticed the tension between the two engineering factions that want 
to keep things agnostic towards backend storage and at the same time 
make the backend storage perform and scale adequately.

I'm not sure why you are being so aggressive about this topic. I 
certainly am not being aggressive about my responses -- just cautioning 
that the existing codebase has seen its fair share of refactoring, some 
of which has been a failure and had to be reverted. I would hate to jump 
into a frenzy to radically change the way that the data access code 
works in Nova without a good discussion.

>> But sure, ok.
>>
>> But then I guarantee somebody is gonna spend a bunch of time writing an
>> object-oriented API to the model objects because the ORM is very useful for
>> the data modification part of the DB interaction.
>
> !cite - seriously...

? I give an example below... a cautionary tale if you will, about one 
possible consequence of "getting rid of the ORM".

>> Because people will complain about having to do this:
>>
>> conn = engine.connection()
>> # instances is the sqlalchemy Table object for instances
>> inst_ins = instances.insert().values(blah=blah)
>> ip_ins = fixed_ips.insert().values(blah=blah)
>> conn.execute(ip_ins)
>> conn.execute(inst_ins)
>> conn.close()
>
> This strawman is one way that it might be written. Given that a
> growing set of our projects have non-SQL backends, this doesn't look
> like the obvious way to phrase it to me.

I'm using the SQLAlchemy Core API above, with none of the SQLAlchemy ORM 
code... which is (I thought), what you were proposing we do? How is that 
a strawman argument? :(

>> instead of this:
>>
>> i = Instance(blah=blah)
>> ip = FixedIp(blah=blah)
>> i.fixed_ips.append(ip)
>> session.add(u)
>> session.commit()
>>
>> And so you've thrown the baby out with the bathwater and made more work for
>> everyone.
>
> Perhaps; or perhaps we've avoided a raft of death-by-thousand-cuts
> bugs across the project.

Could just as easily introduce the same bugs by radically redesigning 
the data access code without first considering all sides of the problem 
domain.

-jay