<div dir="ltr"><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jan 13, 2014 at 1:16 PM, Robert Myers <span dir="ltr"><<a href="mailto:myer0052@gmail.com" target="_blank">myer0052@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">We could always use relative imports in oslo :) Then you could put it where ever you wanted to without needing to rewrite the import statements.</div>
</blockquote><div><br></div><div><div class="gmail_default" style="font-size:small">That may be a good idea, but doesn't really solve the problem at hand.</div><div class="gmail_default" style="font-size:small"><br></div>
<div class="gmail_default" style="font-size:small">Doug</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_extra"><br><br><div class="gmail_quote">
<div><div class="h5">
On Mon, Jan 13, 2014 at 11:07 AM, Doug Hellmann <span dir="ltr"><<a href="mailto:doug.hellmann@dreamhost.com" target="_blank">doug.hellmann@dreamhost.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="h5">
<div dir="ltr"><div style="font-size:small">[resurrecting an old thread]</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Nov 27, 2013 at 6:26 AM, Flavio Percoco <span dir="ltr"><<a href="mailto:flavio@redhat.com" target="_blank">flavio@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div>On 27/11/13 10:59 +0000, Mark McLoughlin wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
On Wed, 2013-11-27 at 11:50 +0100, Flavio Percoco wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
On 26/11/13 22:54 +0000, Mark McLoughlin wrote:<br>
>On Fri, 2013-11-22 at 12:39 -0500, Doug Hellmann wrote:<br>
>> On Fri, Nov 22, 2013 at 4:11 AM, Flavio Percoco <<a href="mailto:flavio@redhat.com" target="_blank">flavio@redhat.com</a>> wrote:<br>
>> > 1) Store the commit sha from which the module was copied from.<br>
>> > Every project using oslo, currently keeps the list of modules it<br>
>> > is using in `openstack-modules.conf` in a `module` parameter. We<br>
>> > could store, along with the module name, the sha of the commit it<br>
>> > was last synced from:<br>
>> ><br>
>> > module=log,commit<br>
>> ><br>
>> > or<br>
>> ><br>
>> > module=log<br>
>> > log=commit<br>
>> ><br>
>><br>
>> The second form will be easier to manage. Humans edit the module field and<br>
>> the script will edit the others.<br>
><br>
>How about adding it as a comment at the end of the python files<br>
>themselves and leaving openstack-common.conf for human editing?<br>
<br>
I think having the commit sha will give us a starting point from which<br>
we could start updating that module from.<br>
</blockquote>
<br>
Sure, my only point was about where the commit sha comes from - i.e.<br>
whether it's from a comment at the end of the python module itself or in<br>
openstack-common.conf<br>
</blockquote>
<br></div>
And, indeed you said 'at the end of the python files'. Don't ask me<br>
how the heck I misread that.<br>
<br>
The benefit I see from having them in the openstack-common.conf is<br>
that we can register a `StrOpt` for each object dynamically and get<br>
the sha using oslo.config. If we put it as a comment at the end of the<br>
python file, we'll have to read it and 'parse' it, I guess.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
It will mostly help with<br>
getting a diff for that module and the short commit messages where it<br>
was modified.<br>
<br>
Here's a pseudo-buggy-algorithm for the update process:<br>
<br>
(1) Get current sha for $module<br>
(2) Get list of new commits for $module<br>
(3) for each commit of $module:<br>
(3.1) for each modified_module in $commit<br>
(3.1.1) Update those modules up to $commit (1)(modified_module)<br>
(3.2) Copy the new file<br>
(3.3) Update openstack-common with the latest sha<br>
<br>
This trusts the granularity and isolation of the patches proposed in<br>
oslo-incubator. However, in cases like 'remove vim mode lines' it'll<br>
fail assuming that updating every module is necessary - which is true<br>
from a git stand point.<br>
</blockquote>
<br>
This is another variant of the kind of inter-module dependency smarts<br>
that update.py already has ... I'd be inclined to just omit those smarts<br>
and just require the caller to explicitly list the modules they want to<br>
include.<br>
<br>
Maybe update.py could include some reporting to help with that choice<br>
like "module foo depends on modules bar and blaa, maybe you want to<br>
include them too" and "commit XXX modified module foo, but also module<br>
bar and blaa, maybe you want to include them too".<br>
</blockquote>
<br></div>
But, if we get to the point of suggesting the user to update module<br>
foo because it was modified in commit XXX, we'd have everything needed<br>
to make it recursive and update those modules as well.<br>
<br>
I agree with you on making it explicit, though. What about making it<br>
interactive then? update.py could ask users if they want to update<br>
module foo because it was modified in commit XXX and do it right away,<br>
which is not very different from updating module foo, print a report<br>
and let the user choose afterwards.<br>
<br>
(/me feels like Gollum now)<br>
<br>
I prefer the interactive way though, at least it doesn't require the<br>
user to run update several times for each module. We could also add a<br>
`--no-stop` flag that does exactly what you suggested.<br></blockquote><div><br></div><div><div style="font-size:small">I spent some time trying to think through how we could improve the update script for [1], and I'm stumped on how to figure out *accurately* what state the project repositories are in today.</div>
<div style="font-size:small"><br></div><div style="font-size:small">We can't just compute the hash of the modules in the project receiving copies, and then look for them in the oslo-incubator repo, because we modify the files as we copy them out (to update the import statements and replace "oslo" with the receiving project name in some places like config option defaults). </div>
<div style="font-size:small"><br></div><div style="font-size:small">We could undo those changes before computing the hash, but the problem is further complicated because syncs are not being done of all modules together. The common code in a project doesn't move forward in step with the oslo-incubator repository as a whole. For example, sometimes only the openstack/common/log.py module is copied and not all of openstack/common. So log.py might be newer than a lot of the rest of the oslo code. The problem is even worse for something like rpc, where it's possible that modules within the rpc package might not all be updated together.</div>
<div style="font-size:small"><div><br></div></div><div style="font-size:small">We could probably spend a lot of effort building a tool to tell us exactly what the state of all of each common file is in each project, to figure out what needs to be synced. I would much rather spend that effort on turning the common code into libraries, though.</div>
<div style="font-size:small"><br></div><div style="font-size:small">So, here's an alternative:</div><div style="font-size:small"><br></div><div style="font-size:small">
1. Projects accept a full sync of Oslo soon, including adding a value in their openstack-common.conf indicating which commit in oslo-incubator is reflected in the sync. We'll try to make those commit messages as detailed as possible.</div>
<div style="font-size:small"><br></div><div style="font-size:small">2. We modify update.py to remove the option to update individual modules when copying from oslo-incubator. The new version would always apply all changes from the last merged commit, as a series of commits, to the receiving project. So if nova is out of step by 3 commits, then 3 new commits would be created in the branch by the person doing the update, each with the commit log message from the change in oslo-incubator. (This lock-step approach is necessary to have any hope of figuring out which commits are actually being synced, so the log messages are accurate.)</div>
<div style="font-size:small"><br></div><div style="font-size:small">3. The person proposing the merge into the project can decide whether to squash the commits, or leave them as separate reviews.</div>
<div style="font-size:small"><br></div><div style="font-size:small">I'm not entirely certain I like this approach myself, but it's the best I've been able to come up with. It essentially gives us the current process, while removing the ability to potentially take a version of a module without taking its dependencies (allowing us to step forward, and track the commit messages accurately). It will also produce results similar to what we will have when all of this oslo code moves into separate libraries, where the changes to the library will be seen by the projects without any action at all on their part.</div>
<div style="font-size:small"><br></div><div style="font-size:small">OTOH, it will also require spending time on update.py, instead of releasing a library from the incubator. And it doesn't really buy us that much in terms of making the sync happen more easily, other than a reliable way of having entirely accurate commit messages.</div>
<div style="font-size:small"><br></div><div style="font-size:small">I would love to have someone else offer an alternative that's less effort to change and provides the desired detailed log messages accurately.</div>
<div style="font-size:small"><br></div><div style="font-size:small">Doug</div><div style="font-size:small"><br></div><div style="font-size:small">
[1] <a href="https://blueprints.launchpad.net/oslo/+spec/improve-update-script" target="_blank">https://blueprints.launchpad.net/oslo/+spec/improve-update-script</a></div><br></div></div></div></div>
<br></div></div><div class="im">_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></div></blockquote></div><br></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>