<div dir="ltr">Guys,<div><br></div><div>I did a few changes on my environment (OpenStack IceHouse on IPv6), everything seems to be working smoothly now...</div><div><br></div><div>Just deployed Heat on IPv6 too...</div><div>
<br></div><div>I didn't tested Ceilomenter and "Cinder Volume" (iSCSI traffic) with IPv6 yet...</div><div><br></div><div>I'm writing a new "Multinode Quick Guide" to deploy OpenStack IceHouse on an (almost) IPv6-Only environment.</div>
<div><br></div><div>Nevertheless, OpenStack still depends on an IPv4-Only networks for Metadata, for "GRE / VXLAN" tunnels and for "Project Subnets" (no Neutron IPv6 yet), everything else (Management, APIs and Endpoints) seems to be working with IPv6 (including RabbitMQ, MySQL, Keystone, Nova, Glance, Neutron (API/Endpoint), Horizon, SPICE Consoles, Heat, Cinder (APIs / Management (iSCSI not tests with IPv6 yet))...</div>
<div><br></div><div>Soon as I finish the new guide, I'll post it here...</div><div><br></div><div>BTW, because of Glance can't use Proxy to download Images, I configured a NAT64/DNS64 here, so, it can reach the old Internet infrastructure normally...</div>
<div><br></div><div>Best!<br></div><div>Thiago</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 13 May 2014 03:17, Martinx - ジェームズ <span dir="ltr"><<a href="mailto:thiagocmartinsc@gmail.com" target="_blank">thiagocmartinsc@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Guys,<div><br></div><div>I'm running OpenStack IceHouse configured with IPv6 in almost every part of it, I can say that both `Management Network` and `API Endpoints` works with IPv6, but, there are still only three places that I am unable to use it with IPv6, which is:</div>
<div><br></div><div><br></div><div>1- Metadata (no IPv6 here, the equivalent of <a href="http://169.254.0.0/16" target="_blank">169.254.0.0/16</a> for IPv6 is the subnet fe80::/64, am I right?);</div><div><br></div><div>
2- VXLAN / GRE tunnels, precisely at `local_ip` in ml2_conf.ini (it doesn't work when with IPv6);</div>
<div><br></div><div>3- Tenant subnet (IPv6 works with Flat Networks and statically/manually configured, no SLAAC and no Neutron L3 with IPv6 yet).</div><div><br></div><div><br></div><div>NOTE: I still did not tested Heat, Cinder or Swift.</div>
<div><br></div><div><br></div><div>Everything else is working with IPv6!</div><div><br></div><div>Here is a few more details about my environment:</div><div><br></div><div>Controller's /etc/network/interface file:</div>
<div><br></div><div>---</div><div><div><font face="courier new, monospace"><div># This file describes the network interfaces available on your system</div><div># and how to activate them. For more information, see interfaces(5).</div>
<div><br></div><div># The loopback network interface</div><div>auto lo</div><div>iface lo inet loopback</div></font></div><div><div><font face="courier new, monospace"><br># The primary network interface</font></div>
</div><div><font face="courier new, monospace">#</font></div><div><font face="courier new, monospace"># OpenStack API Endpoints</font></div><div><font face="courier new, monospace">auto eth0</font></div><div><font face="courier new, monospace">iface eth0 inet6 static</font></div>
<div><font face="courier new, monospace"> address 2804:29X:Y:dead::10 </font></div><div><font face="courier new, monospace"> netmask 64</font></div><div><font face="courier new, monospace"> gateway 2804:29X:Y:dead::1</font></div>
<div><font face="courier new, monospace"> dns-domain <a href="http://tcmc.com.br" target="_blank">tcmc.com.br</a></font></div><div><font face="courier new, monospace"> dns-search <a href="http://tcmc.com.br" target="_blank">tcmc.com.br</a></font></div>
<div><font face="courier new, monospace"> dns-nameservers 2804:29X:4::1 2001:129X:2bX::1</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># OpenStack - Management</font></div>
<div><font face="courier new, monospace">auto eth1</font></div><div><font face="courier new, monospace">iface eth1 inet6 static</font></div><div><font face="courier new, monospace"> address fddc:3c8c:6e8c:b129::10</font></div>
<div><font face="courier new, monospace"> netmask 64</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># Legacy - Only required because of Metadata, it doesn't have an IPv6</font></div>
<div><font face="courier new, monospace"># equivalent service for subnet IPv4 = <a href="http://169.254.0.0/16" target="_blank">169.254.0.0/16</a> (IPv6 = fc80::/64)</font></div><div><font face="courier new, monospace">iface eth1 inet static</font></div>
<div><font face="courier new, monospace"> address 192.168.5.10</font></div><div><font face="courier new, monospace"> netmask 24</font></div></div><div>---</div><div><br></div><div>Network Node /etc/network/interfaces file:<br>
</div><div><br></div><div>---</div><div><div><font face="courier new, monospace"># This file describes the network interfaces available on your system</font></div><div><font face="courier new, monospace"># and how to activate them. For more information, see interfaces(5).</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># The loopback network interface.</font></div><div><font face="courier new, monospace">auto lo</font></div><div><font face="courier new, monospace">iface lo inet loopback</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">#</font></div><div><font face="courier new, monospace"># Reachable from the Internet.</font></div><div><font face="courier new, monospace">#</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># The primary network interface. Node Internet access.</font></div><div><font face="courier new, monospace">auto eth0</font></div>
<div><font face="courier new, monospace">iface eth0 inet6 static</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>address 2804:29X:Y:dead::20</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>netmask 64</font></div>
<div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>gateway 2804:29X:Y:dead::1</font></div><div><font face="courier new, monospace"> dns-domain <a href="http://tcmc.com.br" target="_blank">tcmc.com.br</a></font></div>
<div><font face="courier new, monospace"> dns-search <a href="http://tcmc.com.br" target="_blank">tcmc.com.br</a></font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>dns-nameservers 2804:290:4::1 2001:1291:2bf::1</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">#</font></div><div><font face="courier new, monospace"># Unreachable from the Internet.</font></div><div><font face="courier new, monospace">#</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># OpenStack - Management</font></div><div><font face="courier new, monospace">auto eth1</font></div><div><font face="courier new, monospace">iface eth1 inet6 static</font></div>
<div><font face="courier new, monospace"> address fddc:3c8c:6e8c:b129::20</font></div><div><font face="courier new, monospace"> netmask 64</font></div><div><font face="courier new, monospace"><br></font></div>
<div><font face="courier new, monospace"># Legacy - Only required because of Metadata, it doesn't have an IPv6</font></div><div><font face="courier new, monospace"># equivalent service for subnet IPv4 = <a href="http://169.254.0.0/16" target="_blank">169.254.0.0/16</a> (IPv6 = fc80::/64).</font></div>
<div><font face="courier new, monospace">iface eth1 inet static</font></div><div><font face="courier new, monospace"> address 192.168.5.20</font></div><div><font face="courier new, monospace"> netmask 24</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># VXLAN Traffic - Not working right now with IPv6.</font></div><div><font face="courier new, monospace">auto eth2</font></div>
<div><font face="courier new, monospace">iface eth2 inet6 static</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>address fda2:c917:cd2e:0552::20</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>netmask 64</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># Legacy - Only required because Neutron doesn't support VXLAN tunnels on top</font></div><div><font face="courier new, monospace"># of a IPv6 network.</font></div>
<div><font face="courier new, monospace">iface eth2 inet static</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>address 192.168.6.20</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>netmask 24</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">#</font></div><div><font face="courier new, monospace"># Reachable from the Internet only from within each Namespace router.</font></div>
<div><font face="courier new, monospace">#</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># Bridge br-ex attached here, this is the "WAN Port" of tenant's routers.</font></div>
<div><font face="courier new, monospace">auto eth3</font></div><div><font face="courier new, monospace">iface eth3 inet manual</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>up ip addr add 0/0 dev eth3</font></div>
<div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>up ip link set dev $IFACE up</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>up ip link set $IFACE promisc on</font></div>
<div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>up ethtool --offload $IFACE gro off</font></div><div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>down ip link set $IFACE promisc off</font></div>
<div><font face="courier new, monospace"><span style="white-space:pre-wrap"> </span>down ip link set $IFACE down</font></div></div><div>---</div><div><br></div><div><br></div><div>Common /etc/hosts file across the Cloud:</div>
<div><br></div><div>---</div><div><div><font face="courier new, monospace">127.0.0.1 localhost.localdomain localhost</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># The following lines are desirable for IPv6 capable hosts</font></div>
<div><font face="courier new, monospace">::1 localhost ip6-localhost ip6-loopback</font></div><div><font face="courier new, monospace">ff02::1 ip6-allnodes</font></div><div><font face="courier new, monospace">ff02::2 ip6-allrouters</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># OpenStack APIs Endpoints</font></div><div><font face="courier new, monospace">2804:29X:Y:dead::10 <a href="http://psuaa-1.tcmc.com.br" target="_blank">psuaa-1.tcmc.com.br</a> psuaa-1</font></div>
<div><font face="courier new, monospace">2804:29X:Y:dead::20 <a href="http://psuab-1.tcmc.com.br" target="_blank">psuab-1.tcmc.com.br</a> psuab-1</font></div><div><font face="courier new, monospace">2804:29X:Y:dead::30 <a href="http://psuac-1.tcmc.com.br" target="_blank">psuac-1.tcmc.com.br</a> psuac-1</font></div>
<div><font face="courier new, monospace">2804:29X:Y:dead::1000 <a href="http://psuah-1.tcmc.com.br" target="_blank">psuah-1.tcmc.com.br</a> psuah-1</font></div><div><font face="courier new, monospace"><br></font></div>
<div>
<font face="courier new, monospace"># OpenStack Management - MySQL, RabbitMQ, SPICE, Glance...</font></div><div><font face="courier new, monospace">fddc:3c8c:6e8c:b129::10 <a href="http://psuaa-1.mng.tcmc.com.br" target="_blank">psuaa-1.mng.tcmc.com.br</a> psuaa-1.mng</font></div>
<div><font face="courier new, monospace">fddc:3c8c:6e8c:b129::20 <a href="http://psuab-1.mng.tcmc.com.br" target="_blank">psuab-1.mng.tcmc.com.br</a> psuab-1.mng</font></div><div><font face="courier new, monospace">fddc:3c8c:6e8c:b129::1000 <a href="http://psuah-1.mng.tcmc.com.br" target="_blank">psuah-1.mng.tcmc.com.br</a> psuah-1.mng</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"># VXLAN Network - Project's subnet - DOESN'T WORK WITH IPv6</font></div><div><font face="courier new, monospace">fda2:c917:cd2e:0552::20 <a href="http://psuab-1.vxlan.tcmc.com.br" target="_blank">psuab-1.vxlan.tcmc.com.br</a> psuab-1.vxlan</font></div>
<div><font face="courier new, monospace">fda2:c917:cd2e:0552::1000 <a href="http://psuah-1.vxlan.tcmc.com.br" target="_blank">psuah-1.vxlan.tcmc.com.br</a> psuah-1.vxlan</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace"># Cinder Network - iSCSI Traffic</font></div><div><font face="courier new, monospace">fd72:3148:4c74:2f60::30 <a href="http://psuac-1.blk.tcmc.com.br" target="_blank">psuac-1.blk.tcmc.com.br</a> psuac-1.blk</font></div>
<div><font face="courier new, monospace">fd72:3148:4c74:2f60::1000 <a href="http://psuah-1.blk.tcmc.com.br" target="_blank">psuah-1.blk.tcmc.com.br</a> psuah-1.blk</font></div></div><div>---</div><div><br></div>
<div>NOTE: Those private IPv6 subnets was generated here: <a href="http://www.simpledns.com/private-ipv6.aspx" target="_blank">http://www.simpledns.com/private-ipv6.aspx</a></div>
<div><br></div><div>Then, for example, I configured `auth_host` under `[keystone_authtoken]` poiting to `<a href="http://psuaa-1.mng.tcmc.com.br" target="_blank">psuaa-1.mng.tcmc.com.br</a>` and `auth_uri` poiting to `http://psuaa-1.tcmc.com.br:5000`.</div>
<div><br></div><div>But, as I figured out, Metadata doesn't work with IPv6, which means that `metadata_host / metadata_listen` is configured to `192.168.5.10` at Controller's nova.conf (it doesn't work when I tried it with `<span style="font-family:'courier new',monospace">fddc:3c8c:6e8c:b129::10</span>`) and, at my Network Node, the `local_ip` at `ml2_conf.ini` points to `192.168.6.20` (I would like to use at local_ip, the address `<span style="font-family:'courier new',monospace">fda2:c917:cd2e:0552::20</span>`).</div>
<div><br></div><div>Plus, here is my IPv6 API Endpoints (resolved via /etc/hosts or AAAA DNS Records):</div><div><br></div><div>---</div><div><div><font face="courier new, monospace">root@controller-1:~# keystone endpoint-list</font></div>
<div><font face="courier new, monospace">WARNING: Bypassing authentication using a token & endpoint (authentication credentials are being ignored).</font></div><div><font face="courier new, monospace">+----------------------------------+-----------+-------------------------------------------------------+-----------------------------------------------------------+------------------------------------------------------+----------------------------------+</font></div>
<div><font face="courier new, monospace">| id | region | publicurl | internalurl | adminurl | service_id |</font></div>
<div><font face="courier new, monospace">+----------------------------------+-----------+-------------------------------------------------------+-----------------------------------------------------------+------------------------------------------------------+----------------------------------+</font></div>
<div><font face="courier new, monospace">| 0a7314067f144f94ad907e4023add10a | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8777" target="_blank">http://psuaa-1.tcmc.com.br:8777</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8777" target="_blank">http://psuaa-1.mng.tcmc.com.br:8777</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8777" target="_blank">http://psuaa-1.mng.tcmc.com.br:8777</a> | ef3f7861e7534049a171547bac6189d8 |</font></div>
<div><font face="courier new, monospace">| 10ed8edcbecd4feebfccfaae8b9d6ba6 | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:5000/v2.0" target="_blank">http://psuaa-1.tcmc.com.br:5000/v2.0</a> | <a href="http://psuaa-1.mng.tcmc.com.br:5000/v2.0" target="_blank">http://psuaa-1.mng.tcmc.com.br:5000/v2.0</a> | <a href="http://psuaa-1.mng.tcmc.com.br:35357/v2.0" target="_blank">http://psuaa-1.mng.tcmc.com.br:35357/v2.0</a> | fe7c9413a430421d9b4c2d3760d468fc |</font></div>
<div><font face="courier new, monospace">| 18fb5ccf76c2426ca88ec130461e4f26 | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8004/v1/$(tenant_id)s" target="_blank">http://psuaa-1.tcmc.com.br:8004/v1/$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8004/v1/$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8004/v1/$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8004/v1/$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8004/v1/$(tenant_id)s</a> | 1e101eaa6eb64b6ea046e1ac345e83dc |</font></div>
<div><font face="courier new, monospace">| 29a2b2e132cc46648ba82b6cafdfeb8d | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8776/v1/$(tenant_id)s" target="_blank">http://psuaa-1.tcmc.com.br:8776/v1/$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8776/v1/$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8776/v1/$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8776/v1/$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8776/v1/$(tenant_id)s</a> | 6aa0854705c1468584b118a8d102cd80 |</font></div>
<div><font face="courier new, monospace">| 2f6e5497cafc478cbe3e54d077a05c15 | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:9696" target="_blank">http://psuaa-1.tcmc.com.br:9696</a> | <a href="http://psuaa-1.mng.tcmc.com.br:9696" target="_blank">http://psuaa-1.mng.tcmc.com.br:9696</a> | <a href="http://psuaa-1.mng.tcmc.com.br:9696" target="_blank">http://psuaa-1.mng.tcmc.com.br:9696</a> | 4da8894722404eac8cae5c74a9667eeb |</font></div>
<div><font face="courier new, monospace">| 41740afdb160499b9d24231bd4aec736 | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s" target="_blank">http://psuaa-1.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8080/v1" target="_blank">http://psuaa-1.mng.tcmc.com.br:8080/v1</a> | 2d91b7a432f146b0a20636ab036d4e72 |</font></div>
<div><font face="courier new, monospace">| 75e6fec21dc04277a2950f3fb82cd649 | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:9292" target="_blank">http://psuaa-1.tcmc.com.br:9292</a> | <a href="http://psuaa-1.mng.tcmc.com.br:9292" target="_blank">http://psuaa-1.mng.tcmc.com.br:9292</a> | <a href="http://psuaa-1.mng.tcmc.com.br:9292" target="_blank">http://psuaa-1.mng.tcmc.com.br:9292</a> | 347a19c4a5fd45809350f298bd3bca6d |</font></div>
<div><font face="courier new, monospace">| a35bbcf38abe4d598913800a4dfd2437 | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s" target="_blank">http://psuaa-1.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8080/v1/AUTH_$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8080/v1" target="_blank">http://psuaa-1.mng.tcmc.com.br:8080/v1</a> | 2d91b7a432f146b0a20636ab036d4e72 |</font></div>
<div><font face="courier new, monospace">| f311681da2d14c41acc086cd3d47b93f | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8774/v2/$(tenant_id)s" target="_blank">http://psuaa-1.tcmc.com.br:8774/v2/$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8774/v2/$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8774/v2/$(tenant_id)s</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8774/v2/$(tenant_id)s" target="_blank">http://psuaa-1.mng.tcmc.com.br:8774/v2/$(tenant_id)s</a> | cdd4cb09d53a44f78e2d33042dbfbf79 |</font></div>
<div><font face="courier new, monospace">| fab3ab9df735451bac4932a4797e4f1e | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8000/v1" target="_blank">http://psuaa-1.tcmc.com.br:8000/v1</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8000/v1" target="_blank">http://psuaa-1.mng.tcmc.com.br:8000/v1</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8000/v1" target="_blank">http://psuaa-1.mng.tcmc.com.br:8000/v1</a> | edee5082a58e435ba5d05b3d4c6ea9f4 |</font></div>
<div><font face="courier new, monospace">| fe19983cbc034bdaa1e7b6909e3e6c5e | sp-east-1 | <a href="http://psuaa-1.tcmc.com.br:8773/services/Cloud" target="_blank">http://psuaa-1.tcmc.com.br:8773/services/Cloud</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8773/services/Cloud" target="_blank">http://psuaa-1.mng.tcmc.com.br:8773/services/Cloud</a> | <a href="http://psuaa-1.mng.tcmc.com.br:8773/services/Admin" target="_blank">http://psuaa-1.mng.tcmc.com.br:8773/services/Admin</a> | 9f9b7f9a466d443893457b582435c786 |</font></div>
<div><font face="courier new, monospace">+----------------------------------+-----------+-------------------------------------------------------+-----------------------------------------------------------+------------------------------------------------------+----------------------------------+</font></div>
</div><div>---</div><div><br></div><div>BTW, rabbitmq-server fails to install/start on a IPv6-Only environment, vide BUG: <a href="https://bugs.launchpad.net/ubuntu/+source/rabbitmq-server/+bug/1312507" target="_blank">https://bugs.launchpad.net/ubuntu/+source/rabbitmq-server/+bug/1312507</a></div>
<div><br></div><div>Just for the record, I'm documenting the procedure to guide me, on this gist: <a href="https://gist.github.com/tmartinx/9177697" target="_blank">https://gist.github.com/tmartinx/9177697</a></div><div>
<br></div><div>
I wish the best for you guys at next OpenStack summit! I would love to join it but, unfortunately, I can't...</div><div><br></div><div><div>Sorry about the huge e-mail... I did it this way, instead of using pastebin services, mostly to let it here for posterity in one place... :-P</div>
</div><div><br></div><div>Best Regards!</div><div>Thiago Martins</div></div>
</blockquote></div><br></div>