Open Stack

 Daniel,

 Here is the modified _attach_lxc_volume() and _detach_lxc_volume().
I tested _attach_lxc_volume() and it works fine.
I couldn't test _detach_lxc_volume() because detach_volume() reports
error before calling it.
The error is:
[
2012-09-07 15:18:41 TRACE nova.openstack.common.rpc.amqp   File "/usr/local/nova/nova/virt/libvirt/driver.py", line 669, in detach_volume
2012-09-07 15:18:41 TRACE nova.openstack.common.rpc.amqp     raise exception.DiskNotFound(location=mount_device)
2012-09-07 15:18:41 TRACE nova.openstack.common.rpc.amqp DiskNotFound: No disk at i
]
 I didn't have time to make the code more up-to-date using rootwrap. 
(Sorry for the old style programming.)
nova version that I tested them with is:
[
commit fb84ed7b32520b11d4a6273eb002b1f1abd2fbe8
Merge: 4ccc770 5a470f8
Author: Jenkins <jenkins at review.openstack.org>
Date:   Wed Sep 5 18:13:34 2012 +0000

    Merge "fix issues with Nova security groups and Quantum"
]

 Thanks,
 David


    def _attach_lxc_volume(self, xml, virt_dom, instance_name):
        LOG.info(_('ISI: attaching LXC block device'))

        lxc_container_root = self.get_lxc_container_root(virt_dom)
        lxc_host_volume = self.get_lxc_host_device(xml)
        lxc_container_device = self.get_lxc_container_target(xml)
        lxc_container_target = "%s/%s" % (lxc_container_root,
                                          lxc_container_device)

        # get id of the virt_dom
        spid = str(virt_dom.ID())
    
        # get PID of the init process
        ps_command = subprocess.Popen("ps -o pid --ppid %s --noheaders" % \
                           spid, shell=True, stdout=subprocess.PIPE)
        init_pid = ps_command.stdout.read()
        init_pid = str(int(init_pid))
        retcode = ps_command.wait()
        assert retcode == 0, "ps command returned %d" % retcode
    
        # get major, minor number of the device
        s = os.stat(lxc_host_volume)
        major_num = os.major(s.st_rdev)
        minor_num = os.minor(s.st_rdev)
    
        # allow the device
        dev_whitelist = os.path.join("/cgroup/devices/libvirt/lxc/",
                                     instance_name,
                                     'devices.allow')
        # Allow the disk
        perm = "b %d:%d rwm" % (major_num, minor_num)
        cmd = "echo %s | sudo tee -a %s" % (perm, dev_whitelist)
        subprocess.Popen(cmd, shell=True)
    
        cmd_lxc = 'sudo lxc-attach -n %s -- ' % init_pid
        # check if 'mountpoint' already exists
    
        cmd = '/bin/mknod -m 777 /%s b %d %d '\
             % (lxc_container_device, major_num, minor_num)
        cmd = cmd_lxc + cmd
        subprocess.call(cmd, shell=True)


    @exception.wrap_exception()
    def _detach_lxc_volume(self, xml, virt_dom, instance_name):
        LOG.info(_('ISI: detaching LXC block device'))

        lxc_container_root = self.get_lxc_container_root(virt_dom)
        lxc_container_device = self.get_lxc_container_target(xml)
        lxc_container_target = "%s/%s" % (lxc_container_root,
                                          lxc_container_device)

        if lxc_container_target:
            disk.unbind(lxc_container_target)

        # get id of the virt_dom
        spid = str(virt_dom.ID())
        LOG.info(_('detach_volume: pid(%s)') % spid)

        # get PID of the init process
        ps_command = subprocess.Popen("ps -o pid --ppid %s --noheaders" \
                              % spid, shell=True, stdout=subprocess.PIPE)
        init_pid = ps_command.stdout.read()
        init_pid = str(int(init_pid))
        retcode = ps_command.wait()
        assert retcode == 0, "ps command returned %d" % retcode
    
        cmd = cmd_lxc + ' /bin/rm ' + '/' + lxc_container_device
        LOG.info(_('detach_volume: cmd(%s)') % cmd)
        subprocess.call(cmd, shell=True)




----- Original Message -----
> On Fri, Sep 07, 2012 at 08:29:25AM -0700, David Kang wrote:
> >
> >  Hi Daniel,
> >
> >  You are right.
> > That's why we applied a kernel patch to make 'lxc-attach' work.
> 
> Ahh, that explains it ! I was worried I had missed something
> obvious for a minute :-)
> 
> > Now:
> > $ ls /proc/self/ns/
> > ipc mnt net pid uts
> 
> Yep, that's exactly what we'd need to see.
> 
> >
> > Kernel wise, we are using 2.6.38 with the kernel patch.
> > We are using RHEL6.2.
> > Initially we wanted to use RHEL6.2's default kernel with the kernel
> > patch.
> > But, either the patch doesn't exist for the default kernel, or the
> > available patch did not work with the default kernel.
> > So, we tried a few other kernels and found out that 2.6.38 works
> > with the available patch.
> 
> If you don't mind, could you post the patch you're using somewhere,
> or mail it to me. I might use it to hack up libvirt support for
> this while we're waiting for upstream kernel acceptance, so we'll
> be ready ahead of time.
> 
> Daniel
> --
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org -o- http://virt-manager.org :|
> |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|