Open Stack

When the OpenStack project was started in 2010, we conceived of two languages that would be considered to have first class status: Python, and C++. The idea is that Python would be used for the API services, and that C++ would be used in special cases where Python was not a good fit, such as for ultra-high performance, or kernel drivers, or for memory constrained situations.

Although the Python language preference has prevailed, we should not be allergic to the idea of an agent being done in C++ if it means that there are end-user benefits that justify it. I think that having a modular agent that can be easily extended that has a very small resource footprint is wise. Key issues for a "base" agent are:

1) A way to sign the distributed bits so users can detect/prevent tampering.
2) Ways to extend the agent using flexible, well documented extension APIs.
3) A way to securely issue remote commands to the agent (to be serviced in accordance with registered commands).
4) A way to update the agent in-place, initiated by a remote signal (with an option to disable).

Whether standard AMQP protocol is used for messaging is besides the point, and should be discussed as an implementation detail. I see no reason why C++ could not be used to implement a low memory footprint agent that could offer the functionality I outlined above. Perhaps one of the extension api's is a shell exec with standard IO connected to the parent process. That way you could easily extend it using Python, or whatever you want (existing configuration management tools, etc.)

Adrian

On Dec 19, 2013, at 7:51 AM, Dmitry Mescheryakov <dmescheryakov at mirantis.com<mailto:dmescheryakov at mirantis.com>> wrote:

2013/12/19 Fox, Kevin M <kevin.fox at pnnl.gov<mailto:kevin.fox at pnnl.gov>>
How about a different approach then... OpenStack has thus far been very successful providing an API and plugins for dealing with things that cloud providers need to be able to switch out to suit their needs.

There seems to be two different parts to the unified agent issue:
 * How to get rpc messages to/from the VM from the thing needing to control it.
 * How to write a plugin to go from a generic rpc mechanism, to doing something useful in the vm.

How about standardising what a plugin looks like, "python api, c++ api, etc". It won't have to deal with transport at all.

Also standardize the api the controller uses to talk to the system, rest or amqp.

I think that is what we discussed when we tried to select between Salt + oslo.messaging and pure oslo.messaging
framework for the agent. As you can see, we didn't came to agreement so far :-) Also Clint started a new thread to discuss what, I believe, you defined as the first part of unified agent issue. For clarity, the thread I am referring to is

http://lists.openstack.org/pipermail/openstack-dev/2013-December/022690.html

Then the mechanism is an implementation detail. If rackspace wants to do a VM serial driver, thats cool. If you want to use the network, that works too. Savanna/Trove/etc don't have to care which mechanism is used, only the cloud provider.
Its not quite as good as one and only one implementation to rule them all, but would allow providers to choose what's best for their situation and get as much code shared as can be.

What do you think?

Thanks,
Kevin




________________________________________
From: Tim Simpson [tim.simpson at rackspace.com<mailto:tim.simpson at rackspace.com>]
Sent: Wednesday, December 18, 2013 11:34 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [trove] My thoughts on the Unified Guest Agent

Thanks for the summary Dmitry. I'm ok with these ideas, and while I still disagree with having a single, forced standard for RPC communication, I should probably let things pan out a bit before being too concerned.

- Tim


________________________________
From: Dmitry Mescheryakov [dmescheryakov at mirantis.com<mailto:dmescheryakov at mirantis.com>]
Sent: Wednesday, December 18, 2013 11:51 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [trove] My thoughts on the Unified Guest Agent

Tim,

The unified agent we proposing is based on the following ideas:
  * the core agent has _no_ functionality at all. It is a pure RPC mechanism with the ability to add whichever API needed on top of it.
  * the API is organized into modules which could be reused across different projects.
  * there will be no single package: each project (Trove/Savanna/Others) assembles its own agent based on API project needs.

I hope that covers your concerns.

Dmitry


2013/12/18 Tim Simpson <tim.simpson at rackspace.com<mailto:tim.simpson at rackspace.com><mailto:tim.simpson at rackspace.com<mailto:tim.simpson at rackspace.com>>>
I've been following the Unified Agent mailing list thread for awhile now and, as someone who has written a fair amount of code for both of the two existing Trove agents, thought I should give my opinion about it. I like the idea of a unified agent, but believe that forcing Trove to adopt this agent for use as its by default will stifle innovation and harm the project.

There are reasons Trove has more than one agent currently. While everyone knows about the "Reference Agent" written in Python, Rackspace uses a different agent written in C++ because it takes up less memory. The concerns which led to the C++ agent would not be addressed by a unified agent, which if anything would be larger than the Reference Agent is currently.

I also believe a unified agent represents the wrong approach philosophically. An agent by design needs to be lightweight, capable of doing exactly what it needs to and no more. This is especially true for a project like Trove whose goal is to not to provide overly general PAAS capabilities but simply installation and maintenance of different datastores. Currently, the Trove daemons handle most logic and leave the agents themselves to do relatively little. This takes some effort as many of the first iterations of Trove features have too much logic put into the guest agents. However through perseverance the subsequent designs are usually cleaner and simpler to follow. A community approved, "do everything" agent would endorse the wrong balance and lead to developers piling up logic on the guest side. Over time, features would become dependent on the Unified Agent, making it impossible to run or even contemplate light-weight agents.

Trove's interface to agents today is fairly loose and could stand to be made stricter. However, it is flexible and works well enough. Essentially, the duck typed interface of the trove.guestagent.api.API class is used to send messages, and Trove conductor is used to receive them at which point it updates the database. Because both of these components can be swapped out if necessary, the code could support the Unified Agent when it appears as well as future agents.

It would be a mistake however to alter Trove's standard method of communication to please the new Unified Agent. In general, we should try to keep Trove speaking to guest agents in Trove's terms alone to prevent bloat.

Thanks,

Tim

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org><mailto:OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131219/c283dd3d/attachment.html>