Open Stack

Right, I think there are use cases for both.  I don't think it's a huge
burden to have to know about it.  I think it's actually quite important
to understand when the initialization happens.

-- 
Russell Bryant

On 06/08/2015 05:02 PM, Kevin Benton wrote:
> This depends on what initialize is supposed to be doing. If it's just a
> one-time sync with a back-end, then I think calling it once in each
> child process might not be what we want.
> 
> I left a comment on Terry's patch. I think we should just use the
> callback manager to have a pre-fork and post-fork even to let
> drivers/plugins do whatever is appropriate for them.
> 
> On Mon, Jun 8, 2015 at 1:00 PM, Robert Kukura <kukura at noironetworks.com
> <mailto:kukura at noironetworks.com>> wrote:
> 
>     From a driver's perspective, it would be simpler, and I think
>     sufficient, to change ML2 to call initialize() on drivers after the
>     forking, rather than requiring drivers to know about forking.
> 
>     -Bob
> 
> 
>     On 6/8/15 2:59 PM, Armando M. wrote:
>>     Interestingly, [1] was filed a few moments ago:
>>
>>     [1] https://bugs.launchpad.net/neutron/+bug/1463129
>>
>>     On 2 June 2015 at 22:48, Salvatore Orlando <sorlando at nicira.com
>>     <mailto:sorlando at nicira.com>> wrote:
>>
>>         I'm not sure if you can test this behaviour on your own
>>         because it requires the VMware plugin and the eventlet
>>         handling of backend response.
>>
>>         But the issue was manifesting and had to be fixed with this
>>         mega-hack [1]. The issue was not about several workers
>>         executing the same code - the loopingcall was always started
>>         on a single thread. The issue I witnessed was that the other
>>         API workers just hang.
>>
>>         There's probably something we need to understand about how
>>         eventlet can work safely with a os.fork (I just think they're
>>         not really made to work together!).
>>         Regardless, I did not spent too much time on it, because I
>>         thought that the multiple workers code might have been
>>         rewritten anyway by the pecan switch activities you're doing.
>>
>>         Salvatore
>>
>>
>>         [1] https://review.openstack.org/#/c/180145/
>>
>>         On 3 June 2015 at 02:20, Kevin Benton <blak111 at gmail.com
>>         <mailto:blak111 at gmail.com>> wrote:
>>
>>             Sorry about the long delay.
>>
>>             >Even the LOG.error("KEVIN PID=%s network response: %s" %
>>             (os.getpid(), r.text)) line?  Surely the server would have
>>             forked before that line was executed - so what could
>>             prevent it from executing once in each forked process, and
>>             hence generating multiple logs?
>>
>>             Yes, just once. I wasn't able to reproduce the behavior
>>             you ran into. Maybe eventlet has some protection for this?
>>             Can you provide small sample code for the logging driver
>>             that does reproduce the issue? 
>>
>>             On Wed, May 13, 2015 at 5:19 AM, Neil Jerram
>>             <Neil.Jerram at metaswitch.com
>>             <mailto:Neil.Jerram at metaswitch.com>> wrote:
>>
>>                 Hi Kevin,
>>
>>                 Thanks for your response...
>>
>>                 On 08/05/15 08:43, Kevin Benton wrote:
>>
>>                     I'm not sure I understand the behavior you are
>>                     seeing. When your
>>                     mechanism driver gets initialized and kicks off
>>                     processing, all of that
>>                     should be happening in the parent PID. I don't
>>                     know why your child
>>                     processes start executing code that wasn't
>>                     invoked. Can you provide a
>>                     pointer to the code or give a sample that
>>                     reproduces the issue?
>>
>>
>>                 https://github.com/Metaswitch/calico/tree/master/calico/openstack
>>
>>                 Basically, our driver's initialize method immediately
>>                 kicks off a green thread to audit what is now in the
>>                 Neutron DB, and to ensure that the other Calico
>>                 components are consistent with that.
>>
>>                     I modified the linuxbridge mech driver to try to
>>                     reproduce it:
>>                     http://paste.openstack.org/show/216859/
>>
>>                     In the output, I never received any of the init
>>                     code output I added more
>>                     than once, including the function spawned using
>>                     eventlet.
>>
>>
>>                 Interesting.  Even the LOG.error("KEVIN PID=%s network
>>                 response: %s" % (os.getpid(), r.text)) line?  Surely
>>                 the server would have forked before that line was
>>                 executed - so what could prevent it from executing
>>                 once in each forked process, and hence generating
>>                 multiple logs?
>>
>>                 Thanks,
>>                         Neil
>>
>>                     The only time I ever saw anything executed by a
>>                     child process was actual
>>                     API requests (e.g. the create_port method).
>>
>>
>>
>>
>>                     On Thu, May 7, 2015 at 6:08 AM, Neil Jerram
>>                     <Neil.Jerram at metaswitch.com
>>                     <mailto:Neil.Jerram at metaswitch.com>
>>                     <mailto:Neil.Jerram at metaswitch.com
>>                     <mailto:Neil.Jerram at metaswitch.com>>> wrote:
>>
>>                         Is there a design for how ML2 mechanism
>>                     drivers are supposed to cope
>>                         with the Neutron server forking?
>>
>>                         What I'm currently seeing, with api_workers =
>>                     2, is:
>>
>>                         - my mechanism driver gets instantiated and
>>                     initialized, and
>>                         immediately kicks off some processing that
>>                     involves communicating
>>                         over the network
>>
>>                         - the Neutron server process then forks into
>>                     multiple copies
>>
>>                         - multiple copies of my driver's network
>>                     processing then continue,
>>                         and interfere badly with each other :-)
>>
>>                         I think what I should do is:
>>
>>                         - wait until any forking has happened
>>
>>                         - then decide (somehow) which mechanism driver
>>                     is going to kick off
>>                         that processing, and do that.
>>
>>                         But how can a mechanism driver know when the
>>                     Neutron server forking
>>                         has happened?
>>
>>                         Thanks,
>>                                  Neil
>>
>>                        
>>                     __________________________________________________________________________
>>                         OpenStack Development Mailing List (not for
>>                     usage questions)
>>                         Unsubscribe:
>>                        
>>                     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>                     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>                        
>>                     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>                        
>>                     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>>                     --
>>                     Kevin Benton
>>
>>
>>                     __________________________________________________________________________
>>                     OpenStack Development Mailing List (not for usage
>>                     questions)
>>                     Unsubscribe:
>>                     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>                     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>                     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>                 __________________________________________________________________________
>>                 OpenStack Development Mailing List (not for usage
>>                 questions)
>>                 Unsubscribe:
>>                 OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>                 <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>                 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>>             -- 
>>             Kevin Benton
>>
>>             __________________________________________________________________________
>>             OpenStack Development Mailing List (not for usage questions)
>>             Unsubscribe:
>>             OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>             <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>             http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>         __________________________________________________________________________
>>         OpenStack Development Mailing List (not for usage questions)
>>         Unsubscribe:
>>         OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>         <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>>     __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> -- 
> Kevin Benton
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>