Open Stack

> Also, how are you proposing to deal with live migration of VMs ? The
> virtio serial channel can get closed due to QEMU migrating while the
> proxy is in the middle of sending data to the guest VM, potentially
> causing a lost or mangled message in the guest and the sender won't
> know this if this channel write-only since there's no ACK.

Basically agent in host will takes care of it. Probably it
periodically pushes necessary information into guest agent.
In Neutron case, the existing neutron agents in host already observe
message loss/reorder,
so the situation doesn't get worse with this proposal


> What sort of things are you expecting the guest agent todo for Neutron ?

It depends on what network service guest agents provides.
For example
In case of firewall, update iptables, update routing tables and so on
in case of loadbalancer, start/stop haproxy with new configuration

neutron server sends message for configuration change.
- guest agents update setting/configuration on new request
  For example, user changes firewall rules, the requested change is
pushed to the agent.

guest agent sends messages to neutron server for maintenance.
- status report: update stats/agent liveness
  In case of loadbalancer example, the loadbalancer needs to report
its status(active/slave) to neutron server.
  then reaction will be taken.
- periodically get the current configuration for message reorder/loss
  agents needs to poll and resync its configuration in case that the
agent local configuration is out of sync with what neutron server
thinks.

thanks,
Isaku Yamahata


On Thu, Apr 10, 2014 at 2:11 AM, Daniel P. Berrange <berrange at redhat.com> wrote:
> On Wed, Apr 09, 2014 at 05:33:49PM +0900, Isaku Yamahata wrote:
>> Hello developers.
>>
>>
>> As discussed many times so far[1], there are many projects that needs
>> to propagate RPC messages into VMs running on OpenStack. Neutron in my case.
>>
>> My idea is to relay RPC messages from management network into tenant
>> network over file-like object. By file-like object, I mean virtio-serial,
>> unix domain socket, unix pipe and so on.
>> I've wrote some code based on oslo.messaging[2][3] and a documentation
>> on use cases.[4][5]
>> Only file-like transport and proxying messages would be in oslo.messaging
>> and agent side code wouldn't be a part of oslo.messaging.
>>
>>
>> use cases:([5] for more figures)
>> file-like object: virtio-serial, unix domain socket, unix pipe
>>
>>   server <-> AMQP <-> agent in host <-virtio serial-> guest agent in VM
>>                       per VM
>>
>>   server <-> AMQP <-> agent in host <-unix socket/pipe->
>>              agent in tenant network <-> guest agent in VM
>>
>>
>> So far there are security concerns to forward oslo.messaging from management
>> network into tenant network. One approach is to allow only cast-RPC from
>> server to guest agent in VM so that guest agent in VM only receives messages
>> and can't send anything to servers. With unix pipe, it's write-only
>> for server, read-only for guest agent.
>>
>> Thoughts? comments?
>
> I'm still somewhat aprehensive about the idea of just proxying arbitrary
> data betwetween host & guest agent at the message bus protocol level.
> I'd tend to be more comfortable with some that going through the virt
> driver API in the compute node.
>
> Also, how are you proposing to deal with live migration of VMs ? The
> virtio serial channel can get closed due to QEMU migrating while the
> proxy is in the middle of sending data to the guest VM, potentially
> causing a lost or mangled message in the guest and the sender won't
> know this if this channel write-only since there's no ACK.
>
>> Details of Neutron NFV use case[6]:
>> Neutron services so far typically runs agents in host, the host agent
>> in host receives RPCs from neutron server, then it executes necessary
>> operations. Sometimes the agent in host issues RPC to neutron server
>> periodically.(e.g. status report etc)
>> It's desirable to make such services virtualized as Network Function
>> Virtualizaton(NFV), i.e. make those features run in VMs. So it's quite
>> natural approach to propagate those RPC message into agents into VMs.
>
> What sort of things are you expecting the guest agent todo for Neutron ?
> You have to bear in mind that the guest OS is 100% untrusted from the
> hosts POV, so anything that Neutron asks the guest agent todo can be
> completely ignored, or manipulated in any way the guest OS decides to.
> Similarly, if there were a feedback channel, any data the Neutron might
> receive back from the guest agent has to be considered untrustworthy,
> so should not be used to make functional decisions in Neutron.
>
> Regards,
> Daniel
> --
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org              -o-             http://virt-manager.org :|
> |: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|