[Openstack] heads up regarding keystone dev venv on an Ubuntu VM (VirtualBox)

Sean Reifschneider jafo at tummy.com
Sat May 5 19:54:47 UTC 2012


I'm not sure what "pftp" is, nor do I have a Mac handy to test with, but I
am testing from my Linux firewall box at home, and can't reproduce this
problem.

The box is running CentOS 5, connected to Comcast cable, with no VPN
tunneling to these boxes (the reason I chose it for testing).

I did an "ftp tummy.com", entered "anonymous" and "jafo at tummy.com" and then
did an "ls" and it worked.  I then ran "passive" and it said "passive mode
off".  Then I tried an "ls", and it failed...  I checked the client and I
didn't have the "ip_nat_ftp" module loaded, so I loaded that and did
another "ls" and it worked.  So the issue here was a firewall on the client
side.

While I had the problem, the wire dump showed that the FTP server was
trying to connect to the client, and the client was sending back a "Host
administratively prohibited" message.

Are you *SURE* that tummy.com is sending the "host administratively
prohibited" message and not the Mac?  This would explain why the virtualbox
doesn't see it as well, the mac is generating it back towards tummy.com.

If this is the case, using passive mode or fixing the firewall should
resolve the problem.

If you have the network dump that is being referred to here, I'd love to
see it.

I'd be happy to work with you to resolve this problem.  I'll be away from
the computer for the next couple of hours, 2pm to 4pm Mountain (GMT-6),
but feel free to phone or text me at 970-430-5432 and we can get to the
bottom of this.  Or if I'm online look for me on Google talk at
jafo00 at gmail.com, AIM as "hrayesci", or irc.community.tummy.com on
#hackingsociety.

Sean

On 05/04/2012 10:39 PM, Duncan McGreggor wrote:
> (Adding Sean at tummy, in case this is useful info for him.)
> 
> Nice detective work, Jeremy!
> 
> d
> 
> On Fri, May 4, 2012 at 8:27 PM, Jeremy Hanmer
> <jeremy.hanmer at dreamhost.com> wrote:
>> After quite a bit of experimentation, I've noticed that the crash is triggered by pip when accessing ftp://tummy.com specifically.  After doing some network dumps, it's apparent that after the connection is initiated we receive an ICMP "Host Administratively Prohibited" message.  My assumption is that this message is seen by VirtualBox, who then tears down the NAT session but fails to pass the ICMP message along to the guest (hard to verify because of how quickly the guest gets killed off).  When the guest tries to transfer the first bit of data over the NAT link, I'm guessing it causes something along the lines of a null pointer deref.
>>
>> I'll throw in some of the debug info I've got but the problem is trivial to reproduce:  simply run "pftp tummy.com", log in anonymously, and try an "ls".
>>
>> Process:         VirtualBoxVM [16548]
>> Path:            /Applications/VirtualBox.app/Contents/MacOS/VirtualBoxVM
>> Identifier:      VirtualBoxVM
>> Version:         ??? (???)
>> Code Type:       X86-64 (Native)
>> Parent Process:  VBoxSVC [16535]
>>
>> Date/Time:       2012-05-04 17:14:19.640 -0700
>> OS Version:      Mac OS X 10.7.3 (11D50b)
>> Report Version:  9
>>
>> Crashed Thread:  22
>>
>> Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
>> Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000010
>>
>>
>>
>> On May 4, 2012, at 2:31 PM, Duncan McGreggor wrote:
>>
>>> Update:
>>>
>>> Jeremy Hanmer said that "it looks like it's a port lookup that's
>>> failing and taking cfprefsd with it."
>>>
>>> He's gone over my head, but I'm sure that's meaningful to someone out there ;-)
>>>
>>> d
>>>
>>> On Fri, May 4, 2012 at 12:59 PM, Duncan McGreggor <duncan at dreamhost.com> wrote:
>>>> Updates:
>>>>
>>>>  * Doug Hellmann narrowed this down to the network access that was
>>>> happening with pip
>>>>  * Mark McClain further narrowed it down to VirtualBox's networking:
>>>> with a NATed interface, big probs -- with a bridged interface, things
>>>> go well.
>>>>
>>>> I haven't taken the time to check this on my own system, since I've
>>>> got a working solution right now, but when I need to rebuild, I will
>>>> check.
>>>>
>>>> Mark also mentioned that VBox networking sometimes does some weird
>>>> stuff (rewriting headers or something) and that might be contributing
>>>> to the problem.
>>>>
>>>> Hope this helps,
>>>>
>>>> d
>>>>
>>>> On Fri, May 4, 2012 at 12:40 PM, Duncan McGreggor <duncan at dreamhost.com> wrote:
>>>>> Hey folks,
>>>>>
>>>>> We're really pressed for time right now, so there are certain rabbit
>>>>> holes we can't dive down, but I wanted to bring this up in case it
>>>>> hasn't been seen yet.
>>>>>
>>>>> On Mac OS X 10.6 and 10.7, when running a 12.04 Ubuntu VM and setting
>>>>> up the dev env for Keystone, we get some madness.
>>>>>  10.6: VirtualBox instance aborts, leaving no traces of issue in
>>>>> system logs (that I could see)
>>>>>  10.7: VB dies, OS X kernel panics
>>>>>
>>>>> The second time, I watched carefully, and it happened as
>>>>> python-memcached was getting installed via pip in the .venv.
>>>>>
>>>>> "So I built a third. That burned down, fell over, then sank into the swamp."
>>>>>
>>>>> But the fourth one stayed up after I removed .venv and changed
>>>>> tools/install_venv.py to enable system site-package use.
>>>>>
>>>>> d
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to     : openstack at lists.launchpad.net
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>





More information about the Openstack mailing list