<HTML>
<HEAD>
<TITLE>Re: RFC: Rewritten libvirt driver XML generation</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>Awesome!<BR>
<BR>
Great job :-)<BR>
<BR>
+1 milllion<BR>
<BR>
On 3/8/12 11:03 AM, "Daniel P. Berrange" <<a href="berrange@redhat.com">berrange@redhat.com</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>Back in January Joshua Harlow raised the question of whether we should<BR>
replace the usage of Cheetah templates for generating XML in the libvirt<BR>
driver:<BR>
<BR>
<a href="https://lists.launchpad.net/openstack/msg06481.html">https://lists.launchpad.net/openstack/msg06481.html</a><BR>
<BR>
Since then I have had some time to work on this idea and now have a<BR>
working branch available for testing. I don't want to push this to<BR>
Gerrit right now, since it isn't really material suitable for the<BR>
Essex release, and AFAICT we don't have a separate review/GIT branch<BR>
for non-Essex feature dev work.<BR>
<BR>
Thus for now I have pushed it to a private a branch here:<BR>
<BR>
<a href="https://github.com/berrange/nova/tree/libvirt-xml-config-v1">https://github.com/berrange/nova/tree/libvirt-xml-config-v1</a><BR>
<BR>
The foundation for the work is early in the series, where I create<BR>
a new nova/virt/libvirt/config.py module with a set of classes for<BR>
representing the aspects of libvirt configuration that Nova is interested<BR>
in. Each of the config classes implement a format_dom() method for<BR>
serializing themselves to an lxml.etree.Element DOM instance.<BR>
<BR>
Currently these objects can be used to generate XML, but in the future<BR>
they will also be able to parse the XML. For this they will implement<BR>
a parse_dom() method which will de-deserialize the xml.etree.Element<BR>
DOM.<BR>
<BR>
Joshua's original posting had talked about having separate layers<BR>
for the config objects vs the serialization. IMHO this would be<BR>
overkill, just adding abstraction for little real world gain. We<BR>
don't need to have pluggable XML serialization impls, one good one<BR>
is sufficient.<BR>
<BR>
The rest of the series is simply a piece-by-piece conversion of the<BR>
template code to the new object based APIs. I did it in a great many<BR>
steps, to make it easier to review & test the changes.<BR>
<BR>
As well as the guest config creation, I also took the opportunity to<BR>
change two others places where we generate XML. The host CPU comparison<BR>
code and the domain snapshot creation. There is still one place left<BR>
to fix, the firewall filter generator.<BR>
<BR>
By the end of the series we have the following benefits<BR>
<BR>
- No code anywhere outside config.py ever needs to know about XML<BR>
documents<BR>
<BR>
- We actually have proper XML escaping, making us safe from potential<BR>
exploits in that area<BR>
<BR>
- There is clean separation of the logic for constructing the<BR>
guest config, from the logic for generating XML.<BR>
<BR>
<BR>
My next step following on from this is to actually start making the<BR>
config generation more flexible, removing alot of hardcoding it<BR>
currently does (eg horrible global virtio on/off switch). This will<BR>
entail tagging images on import with an operating system identifier,<BR>
and then using libosinfo to query exactly what hardware devices the<BR>
OS supports & picking the optimal ones.<BR>
<BR>
I tested this on a KVM host and verified the XML generated for the<BR>
guest before/after was the same. I've not tested all the possible<BR>
block / network driver combinations though, so might have broken<BR>
something not covered by the test suite<BR>
<BR>
Diffstat for the whole patch series<BR>
<BR>
b/nova/tests/fakelibvirt.py | 11<BR>
b/nova/tests/test_libvirt.py | 67 +++--<BR>
b/nova/tests/test_libvirt_config.py | 448 +++++++++++++++++++++++++++++++++<BR>
b/nova/tests/test_libvirt_vif.py | 54 +---<BR>
b/nova/virt/libvirt/config.py | 420 +++++++++++++++++++++++++++++++<BR>
b/nova/virt/libvirt/connection.py | 476 ++++++++++++++++++++++--------------<BR>
b/nova/virt/libvirt/vif.py | 102 ++++---<BR>
b/nova/virt/libvirt/volume.py | 52 ++-<BR>
nova/virt/cpuinfo.xml.template | 9<BR>
nova/virt/libvirt.xml.template | 188 --------------<BR>
10 files changed, 1323 insertions(+), 504 deletions(-)<BR>
<BR>
<BR>
Regards,<BR>
Daniel<BR>
--<BR>
|: <a href="http://berrange.com">http://berrange.com</a> -o- <a href="http://www.flickr.com/photos/dberrange/">http://www.flickr.com/photos/dberrange/</a> :|<BR>
|: <a href="http://libvirt.org">http://libvirt.org</a> -o- <a href="http://virt-manager.org">http://virt-manager.org</a> :|<BR>
|: <a href="http://autobuild.org">http://autobuild.org</a> -o- <a href="http://search.cpan.org/~danberr/">http://search.cpan.org/~danberr/</a> :|<BR>
|: <a href="http://entangle-photo.org">http://entangle-photo.org</a> -o- <a href="http://live.gnome.org/gtk-vnc">http://live.gnome.org/gtk-vnc</a> :|<BR>
<BR>
</SPAN></FONT></BLOCKQUOTE>
</BODY>
</HTML>