Open Stack

Nope, you're totally right, corolocal.local is a class, whose instances are
the actual coroutine local storage.

Alex


On Wed, Nov 20, 2013 at 9:11 AM, Roman Podoliaka <rpodolyaka at mirantis.com>wrote:

> Hey all,
>
> I think I found a serious bug in our usage of eventlet thread local
> storage. Please check out this snippet [1].
>
> This is how we use eventlet TLS in Nova and common Oslo code [2]. This
> could explain how [3] actually breaks TripleO devtest story and our
> gates.
>
> Am I right? Or I am missing something and should get some sleep? :)
>
> Thanks,
> Roman
>
> [1] http://paste.openstack.org/show/53686/
> [2]
> https://github.com/openstack/nova/blob/master/nova/openstack/common/local.py#L48
> [3]
> https://github.com/openstack/nova/commit/85332012dede96fa6729026c2a90594ea0502ac5
>
> On Wed, Nov 20, 2013 at 5:55 PM, Derek Higgins <derekh at redhat.com> wrote:
> > On 20/11/13 14:21, Anita Kuno wrote:
> >> Thanks for posting this, Joe. It really helps to create focus so we can
> >> address these bugs.
> >>
> >> We are chatting in #openstack-neutron about 1251784, 1249065, and
> 1251448.
> >>
> >> We are looking for someone to work on 1251784 - I had mentioned it at
> >> Monday's Neutron team meeting and am trying to shop it around in
> >> -neutron now. We need someone other than Salvatore, Aaron or Maru to
> >> work on this since they each have at least one very important bug they
> >> are working on. Please join us in #openstack-neutron and lend a hand -
> >> all of OpenStack needs your help.
> >
> > I've been hitting this in tripleo intermittently for the last few days
> > (or it at least looks to be the same bug), this morning while trying to
> > debug the problem I noticed http request/responses happening out of
> > order. I've added details to the bug.
> >
> > https://bugs.launchpad.net/tripleo/+bug/1251784
> >
> >>
> >> Bug 1249065 is assigned to Aaron Rosen, who isn't in the channel at the
> >> moment, so I don't have an update on his progress or any blockers he is
> >> facing. Hopefully (if you are reading this Aaron) he will join us in
> >> channel soon and I had hear from him about his status.
> >>
> >> Bug 1251448 is assigned to Maru Newby, who I am talking with now in
> >> -neutron. He is addressing the bug. I will share what information I have
> >> regarding this one when I have some.
> >>
> >> We are all looking forward to a more stable gate and this information
> >> really helps.
> >>
> >> Thanks again, Joe,
> >> Anita.
> >>
> >> On 11/20/2013 01:09 AM, Joe Gordon wrote:
> >>> Hi All,
> >>>
> >>> As many of you have noticed the gate has been in very bad shape over
> the
> >>> past few days.  Here is a list of some of the top open bugs (without
> >>> pending patches, and many recent hits) that we are hitting.  Gate
> won't be
> >>> stable, and it will be hard to get your code merged, until we fix these
> >>> bugs.
> >>>
> >>> 1) https://bugs.launchpad.net/bugs/1251920
> >>>  nova
> >>> 468 Hits
> >>> 2) https://bugs.launchpad.net/bugs/1251784
> >>>  neutron, Nova
> >>>  328 Hits
> >>> 3) https://bugs.launchpad.net/bugs/1249065
> >>>  neutron
> >>>   122 hits
> >>> 4) https://bugs.launchpad.net/bugs/1251448
> >>>  neutron
> >>> 65 Hits
> >>>
> >>> Raw Data:
> >>>
> >>>
> >>> Note: If a bug has any hits for anything besides failure, it means the
> >>> fingerprint isn't perfect.
> >>>
> >>> Elastic recheck known issues
> >>>  Bug: https://bugs.launchpad.net/bugs/1251920 =>
> message:"assertionerror:
> >>> console output was empty" AND filename:"console.html" Title: Tempest
> >>> failures due to failure to return console logs from an instance
> Project:
> >>> Status nova: Confirmed Hits FAILURE: 468 Bug:
> >>> https://bugs.launchpad.net/bugs/1251784 => message:"Connection to
> neutron
> >>> failed: Maximum attempts reached" AND filename:"logs/screen-n-cpu.txt"
> >>> Title: nova+neutron scheduling error: Connection to neutron failed:
> Maximum
> >>> attempts reached Project: Status neutron: New nova: New Hits FAILURE:
> 328
> >>> UNSTABLE: 13 SUCCESS: 275 Bug: https://bugs.launchpad.net/bugs/1240256=>
> >>> message:" 503" AND filename:"logs/syslog.txt" AND
> >>> syslog_program:"proxy-server" Title: swift proxy-server returning 503
> >>> during tempest run Project: Status openstack-ci: Incomplete swift: New
> >>> tempest: New Hits FAILURE: 136 SUCCESS: 83
> >>> Pending Patch Bug: https://bugs.launchpad.net/bugs/1249065 =>
> message:"No
> >>> nw_info cache associated with instance" AND
> >>> filename:"logs/screen-n-api.txt" Title: Tempest failure:
> >>> tempest/scenario/test_snapshot_pattern.py Project: Status neutron: New
> >>> nova: Confirmed Hits FAILURE: 122 Bug:
> >>> https://bugs.launchpad.net/bugs/1252514 => message:"Got error from
> Swift:
> >>> put_object" AND filename:"logs/screen-g-api.txt" Title: glance doesn't
> >>> recover if Swift returns an error Project: Status devstack: New
> glance: New
> >>> swift: New Hits FAILURE: 95
> >>> Pending Patch Bug: https://bugs.launchpad.net/bugs/1244255 =>
> >>> message:"NovaException: Unexpected vif_type=binding_failed" AND
> >>> filename:"logs/screen-n-cpu.txt" Title: binding_failed because of l2
> agent
> >>> assumed down Project: Status neutron: Fix Committed Hits FAILURE: 92
> >>> SUCCESS: 29 Bug: https://bugs.launchpad.net/bugs/1251448 => message:"
> >>> possible networks found, use a Network ID to be more specific. (HTTP
> 400)"
> >>> AND filename:"console.html" Title: BadRequest: Multiple possible
> networks
> >>> found, use a Network ID to be more specific. Project: Status neutron:
> New
> >>> Hits FAILURE: 65 Bug: https://bugs.launchpad.net/bugs/1239856 =>
> >>> message:"tempest/services" AND message:"/images_client.py" AND
> >>> message:"wait_for_image_status" AND filename:"console.html" Title:
> >>> "TimeoutException: Request timed out" on
> >>>
> tempest.api.compute.images.test_list_image_filters.ListImageFiltersTestXML
> >>> Project: Status glance: New Hits FAILURE: 62 Bug:
> >>> https://bugs.launchpad.net/bugs/1235435 => message:"One or more ports
> have
> >>> an IP allocation from this subnet" AND message:" SubnetInUse: Unable to
> >>> complete operation on subnet" AND filename:"logs/screen-q-svc.txt"
> Title:
> >>> 'SubnetInUse: Unable to complete operation on subnet UUID. One or more
> >>> ports have an IP allocation from this subnet.' Project: Status neutron:
> >>> Incomplete nova: Fix Committed tempest: New Hits FAILURE: 48 Bug:
> >>> https://bugs.launchpad.net/bugs/1224001 =>
> >>> message:"tempest.scenario.test_network_basic_ops AssertionError: Timed
> out
> >>> waiting for" AND filename:"console.html" Title: test_network_basic_ops
> >>> fails waiting for network to become available Project: Status neutron:
> In
> >>> Progress swift: Invalid tempest: Invalid Hits FAILURE: 42 Bug:
> >>> https://bugs.launchpad.net/bugs/1218391 => message:"Cannot
> 'createImage'"
> >>> AND filename:"console.html" Title:
> >>>
> tempest.api.compute.images.test_images_oneserver.ImagesOneServerTestXML.test_delete_image_that_is_not_yet_active
> >>> spurious failure Project: Status nova: Confirmed swift: Confirmed
> tempest:
> >>> Confirmed Hits FAILURE: 25
> >>>
> >>>
> >>>
> >>> best,
> >>> Joe Gordon
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev at lists.openstack.org
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131120/b99eca42/attachment.html>