Open Stack

On Wed, 15 Oct 2014 08:19:03 PM Clint Byrum wrote:
> > > I think it would be a good idea for containers' filesystem contents to
> > > be a whole distro. What's at question in this thread is what should be
> > > running. If we can just chroot into the container's FS and run
> > > apt-get/yum
> > > install our tools, and then nsenter and attach to the running process,
> > > then huzzah: I think we have best of both worlds.
> >
> > 
> >
> > Erm, yes that's exactly what you can do with containers (docker, lxc, and 
> > presumably any other use of containers with a private/ephemeral
> > filesystem).>
> > 
> 
> The point I was trying to make is that this case was not being addressed
> by the "don't run init in a container" crowd. I am in that crowd, and
> thus, wanted to make the point: this is how I think it will work when
> I do finally get around to trying containers for a real workload.

So you don't need init in a container in order to do the above (chroot into a 
container and run apt-get/yum and attach gdb/whatever to a running process).
(Oh wait, perhaps you're already agreeing with that?  I'm confused, so I'm 
going to explain how in case others are curious anyway.)


You just need to find the pid of a process in the container (perhaps using 
docker inspect to go from container name -> pid) and then:
 nsenter -t $pid -m -u -i -n -p -w
will give you a shell (by default) running in the already existing container.  
>From there you can install whatever additional packages you want and poke at 
the existing processes from their own (inside the container) pov.  A handy 
script (posted to os-dev previously) is:

 #!/bin/sh
 # name or id
 container=$1; shift 
 pid=$(docker inspect --format '{{ .State.Pid }}' $container)
 if [ $pid -eq 0 ]; then
    echo "No pid found for $container -> dead?" >&2
    exit 1
 fi
 exec nsenter -t $pid -m -u -i -n -p -w "$@"

If the docker container is destroyed/recreated at any point then your 
modifications to the filesystem (installing additional packages) are lost, which 
is probably what you wanted.


An interesting variant of this approach is CoreOS's "toolbox".  CoreOS is a 
very minimal OS that is designed to be just enough to run docker containers.  
Consequently it doesn't have many debugging tools available (no tcpdump, eg).  
But it has a "toolbox" command that is basically a simple shell script that 
pulls down and runs a full-featured generic distro (redhat by default iirc) 
and runs that in "privileged" mode.
Inside that generic distro container you get access to the real network 
devices and processes so you can install additional tools and tcpdump, etc as 
you wish.  When you exit, it gets cleaned up.  Works quite nicely.

-- 
 - Gus