Open Stack

On 12/21/2013 04:51 AM, Boris Pavlovic wrote:
> Jay,
>
> The session object will be commit()'d after the session.begin() context
> manager exits, which will cause the aforementioned BEGIN; INSERT;
> COMMIT; transaction to be executed against the server for each event record.
>
> It is a just half of problem. We should use SQL Bulk Inserts as well.
> And we should mention that declarative_base won't make this work for us
> transparent. (Even if we make all in one transactions there will be N
> Inserts).

Well, the performance benefit will show up if there is a single 
transaction with multiple INSERT statements in it. The slowdown in 
performance is due to the multiple COMMITs, which each typically cause 
an fsync() (or fdatasync()), which is the slow part of the operation. 
Having a transaction containing thousands of INSERT statements with one 
COMMIT is much better performing since there is only a single call to 
fsync() the log records.

Not quite sure what you mean about the declarative_base not working for 
this. Would you mind elaborating a bit more?

Thanks!
-jay