[openstack-dev] [OpenStack-Infra] [ThirdPartyCI] Need help setting up CI

Eduard Matei eduard.matei at cloudfounders.com
Thu Jan 15 14:04:42 UTC 2015


Hi Amit,
This is the only logging i see:
2015-01-15 14:48:28,034 DEBUG zuul.Gerrit: Change <Change 0x7fa0bac97a50
146404,10> status: NEW
2015-01-15 14:48:28,035 INFO zuul.Gerrit: Updating information for 146460,5
2015-01-15 14:51:07,851 DEBUG zuul.Gearman: Looking for lost builds
2015-01-15 14:56:07,852 DEBUG zuul.Gearman: Looking for lost builds
2015-01-15 15:01:00,627 DEBUG zuul.Scheduler: Prepare to exit
2015-01-15 15:01:00,627 DEBUG zuul.Scheduler: Waiting for exit
2015-01-15 15:01:00,627 DEBUG zuul.Scheduler: Run handler awake
2015-01-15 15:01:00,627 DEBUG zuul.Scheduler: Checking if all builds are
complete
2015-01-15 15:01:00,628 DEBUG zuul.Scheduler: All builds are complete
2015-01-15 15:01:00,628 DEBUG zuul.Scheduler: Exiting


At 15.01 it was restarted by jenkins and between 14:48 and 15:01 there was
a job that finished but Gerrit didn't post the results.

Eduard

On Thu, Jan 15, 2015 at 3:06 PM, Amit Das <amit.das at cloudbyte.com> wrote:

> Hi Eduard,
>
> Can you put the logs or details for below comment:
>
> it seems that sometimes Zuul gets stuck (sometimes in Looking for lost
>> builds, sometimes doing nothing) and it misses notifications.
>
>
> I guess, we too have been facing the same issue with Zuul.
>
>
> Regards,
> Amit
> *CloudByte Inc.* <http://www.cloudbyte.com/>
>
> On Thu, Jan 15, 2015 at 6:29 PM, Eduard Matei <
> eduard.matei at cloudfounders.com> wrote:
>
>> Hi,
>> Just so i don't start another thread: it seems that sometimes Zuul gets
>> stuck (sometimes in Looking for lost builds, sometimes doing nothing) and
>> it misses notifications.
>> I added a Jenkins timer job to restart it every 30 minutes. Is there a
>> fix for this or something to change in config?
>>
>> Thanks,
>> Eduard
>>
>> On Thu, Jan 15, 2015 at 2:56 PM, Eduard Matei <
>> eduard.matei at cloudfounders.com> wrote:
>>
>>> Hi Ramy,
>>>
>>> The issue with disconnect/abort no longer happens, so i guess it was
>>> some issues with networking.
>>>
>>> Regarding the ssh keys i finally used Jenkins Configuration Provider
>>> Plugin to inject ssh keys as a pre-build step, then i added a manual
>>> execution step to scp the logs to the server, so now everything appears to
>>> be working.
>>>
>>> Now for the REAL tests:
>>> looking at
>>> https://github.com/rasselin/os-ext-testing-data/blob/master/etc/jenkins_jobs/config/dsvm-cinder-driver.yaml.sample#L7
>>> it seems i need to put the code to install my "third party libs" in the
>>> pre_test_hook which appears in devstack-vm-gate-wrap.sh to be executing
>>> BEFORE installing devstack. The problem is we need our "third party libs"
>>> to be installed (actually configured) AFTER installing devstack, since they
>>> install on top of devstack and do some configuration on devstack.
>>>
>>> How could i integrate my code into devstack-vm-gate-wrap.sh so that my
>>> libs are installed after devstack but before running tempest.
>>> Also, as a side question, do i need to run all the test (since they
>>> already run in dsvm-tempest-full) or can i run only our internal
>>> functionality tests? (in this case i don't need devstack-vm-gate-wrap.sh
>>> and i can run custom code directly from jenkins job).
>>>
>>> Thanks,
>>>
>>> Eduard
>>>
>>> On Wed, Jan 14, 2015 at 6:49 PM, Asselin, Ramy <ramy.asselin at hp.com>
>>> wrote:
>>>
>>>>  Double check your ssh public/private key, authorized keys
>>>> configuration in the slave & log server.
>>>>
>>>> They should all match up. No passwords should be requested.
>>>>
>>>> Try doing a manual scp as Jenkins user from the slave to the log server
>>>> (as Jenkins user there too).
>>>>
>>>>
>>>>
>>>> Regarding manually setting up the publisher, I don’t know. Jenkins Job
>>>> Builder configuration I referenced below should work correctly.
>>>>
>>>>
>>>>
>>>> Ramy
>>>>
>>>>
>>>>
>>>> *From:* Eduard Matei [mailto:eduard.matei at cloudfounders.com]
>>>> *Sent:* Wednesday, January 14, 2015 6:56 AM
>>>>
>>>> *To:* OpenStack Development Mailing List (not for usage questions)
>>>> *Subject:* Re: [openstack-dev] [OpenStack-Infra] [ThirdPartyCI] Need
>>>> help setting up CI
>>>>
>>>>
>>>>
>>>> Hi Ramy,
>>>>
>>>>
>>>>
>>>> I think i managed to setup SCP plugin and updated zuul.conf but it only
>>>> uploads the console.html not the logs
>>>>
>>>> I have an SCP site pointing to the apache server and the root location
>>>> is /srv/static, and in the dsvm job manually added a post-build step,
>>>> Publish Artifacts to SCP Repository, with source: logs/** and destination
>>>> logs/${LOG_PATH} and also checked Copy Console Log.
>>>>
>>>>
>>>>
>>>> The destination path is created, but it only contains console.html.
>>>>
>>>>
>>>>
>>>> I tried to manually scp the directory, but the jenkins user in the
>>>> devstack_slave asks for a password when trying to ssh to apache server.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Eduard
>>>>
>>>>
>>>>
>>>> On Wed, Jan 14, 2015 at 10:40 AM, Eduard Matei <
>>>> eduard.matei at cloudfounders.com> wrote:
>>>>
>>>>  Thanks Ramy, i'll try to set it up manually.
>>>>
>>>>
>>>>
>>>> Meanwhile i found another problem: jobs are failing because of error or
>>>> are being aborted.
>>>>
>>>> FATAL: java.io.IOException: Unexpected termination of the channel
>>>>
>>>> hudson.remoting.RequestAbortedException <http://stacktrace.jenkins-ci.org/search?query=hudson.remoting.RequestAbortedException>: java.io.IOException: Unexpected termination of the channel
>>>>
>>>>
>>>>
>>>> Is this related to another warning i got:
>>>>
>>>> WARNING: devstack run took > 15 minutes, this is a very slow node.
>>>>
>>>>
>>>>
>>>> Is there a timeout for how long a job is allowed to run?
>>>>
>>>> Most jobs take between 40 and 60 minutes, some are able to run successfully, some are aborted or get this error.
>>>>
>>>>
>>>>
>>>> Hardware is Xeon E3-1220 Quad core 3.10 Ghz, 32 GB RAM and 3x 1TB sata HDD.
>>>>
>>>> Devstack slaves are configured as m1.large (4 vCPUs, 8 GB vRAM and 80 GB vHDD).
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Eduard
>>>>
>>>>
>>>>
>>>> On Tue, Jan 13, 2015 at 5:34 PM, Asselin, Ramy <ramy.asselin at hp.com>
>>>> wrote:
>>>>
>>>>  Hi Eduard,
>>>>
>>>>
>>>>
>>>> Apache logs server address is set here (should probably add some
>>>> comment/doc for that)
>>>>
>>>>
>>>> https://github.com/rasselin/os-ext-testing-data/blob/master/vars.sh.sample#L10
>>>>
>>>>
>>>>
>>>> Jenkins will get configured here:
>>>> https://github.com/rasselin/os-ext-testing/blob/master/puppet/modules/os_ext_testing/templates/jenkins/be.certipost.hudson.plugin.SCPRepositoryPublisher.xml.erb
>>>>
>>>> Note that you need to restart Jenkins for those changes to take effect.
>>>> After it’s set, you can use the Jenkins UI to ‘test’ the connection.
>>>>
>>>>
>>>>
>>>> Jenkins Job Builder publishers are here:
>>>> https://github.com/rasselin/os-ext-testing/blob/master/puppet/modules/os_ext_testing/templates/jenkins_job_builder/config/macros.yaml.erb#L110
>>>>
>>>>
>>>>
>>>> Use the publishers as shown here:
>>>>
>>>>
>>>> https://github.com/rasselin/os-ext-testing-data/blob/master/etc/jenkins_jobs/config/dsvm-cinder-driver.yaml.sample#L7
>>>>
>>>>
>>>>
>>>> Zuul will then use this url when commenting in gerrit:
>>>> https://github.com/rasselin/os-ext-testing/blob/master/puppet/modules/os_ext_testing/manifests/master.pp#L18
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Ramy
>>>>
>>>>
>>>>
>>>> *From:* Eduard Matei [mailto:eduard.matei at cloudfounders.com]
>>>> *Sent:* Tuesday, January 13, 2015 2:31 AM
>>>>
>>>>
>>>> *To:* OpenStack Development Mailing List (not for usage questions)
>>>> *Subject:* Re: [openstack-dev] [OpenStack-Infra] [ThirdPartyCI] Need
>>>> help setting up CI
>>>>
>>>>
>>>>
>>>> Thanks Ramy, i got it now working (the NODEPOOL_SSH_KEY was not set
>>>> before starting first time nodepool, so template got incorrect key)
>>>>
>>>> Next question: where do i configure the apache logs server address? I
>>>> have a separate vm with jenkins account and running apache2 exposing
>>>> /srv/static/logs, but where do i enter its ip address ? (in jenkins,
>>>> nodepool or... )
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Eduard
>>>>
>>>>
>>>>
>>>> On Mon, Jan 12, 2015 at 6:09 PM, Asselin, Ramy <ramy.asselin at hp.com>
>>>> wrote:
>>>>
>>>>  You are correct to run nodepoold as nodepool user.
>>>>
>>>> I didn’t see any issues…
>>>>
>>>> Could you double check the public keys listed in .ssh/authorized_keys
>>>> in the template for Ubuntu and Jenkins users match $NODEPOOL_SSH_KEY?
>>>>
>>>> Ramy
>>>>
>>>>
>>>>
>>>> *From:* Eduard Matei [mailto:eduard.matei at cloudfounders.com]
>>>> *Sent:* Monday, January 12, 2015 5:30 AM
>>>>
>>>>
>>>> *To:* OpenStack Development Mailing List (not for usage questions)
>>>> *Subject:* Re: [openstack-dev] [OpenStack-Infra] [ThirdPartyCI] Need
>>>> help setting up CI
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> Regarding the last issue, i fixed it by logging in and manually "pip
>>>> install docutils". Image was created successfully.
>>>>
>>>>
>>>>
>>>> Now the problem is that nodepool is not able to login into instances
>>>> created from that image.
>>>>
>>>> I have NODEPOOL_SSH_KEY exported in the screen where nodepool is
>>>> running, and also i am able to login to the instance from user nodepool,
>>>> but nodepoold gives error:
>>>>
>>>> 2015-01-12 14:19:03,095 DEBUG paramiko.transport: Switch to new keys ...
>>>>
>>>> 2015-01-12 14:19:03,109 DEBUG paramiko.transport: Trying key
>>>> c03fbf64440cd0c2ecbc07ce4ed59804 from /home/nodepool/.ssh/id_rsa
>>>>
>>>> 2015-01-12 14:19:03,135 DEBUG paramiko.transport: userauth is OK
>>>>
>>>> 2015-01-12 14:19:03,162 INFO paramiko.transport: Authentication
>>>> (publickey) failed.
>>>>
>>>> 2015-01-12 14:19:03,185 DEBUG paramiko.transport: Trying discovered key
>>>> c03fbf64440cd0c2ecbc07ce4ed59804 in /home/nodepool/.ssh/id_rsa
>>>>
>>>> 2015-01-12 14:19:03,187 DEBUG paramiko.transport: userauth is OK
>>>>
>>>> ^C2015-01-12 14:19:03,210 INFO paramiko.transport: Authentication
>>>> (publickey) failed.
>>>>
>>>> 2015-01-12 14:19:03,253 DEBUG paramiko.transport: EOF in transport
>>>> thread
>>>>
>>>> 2015-01-12 14:19:03,254 INFO nodepool.utils: Password auth exception.
>>>> Try number 4...
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> echo $NODEPOOL_SSH_KEY
>>>>
>>>>
>>>> AAAAB3NzaC1yc2EAAAADAQABAAABAQC9gP6qui1fmHrj02p6OGvnz7kMTJ2rOC3SBYP/Ij/6yz+SU8rL5rqL6jqT30xzy9t1q0zsdJCNB2jExD4xb+NFbaoGlvjF85m12eFqP4CQenxUOdYAepf5sjV2l8WAO3ylspQ78ipLKec98NeKQwLrHB+xon6QfAHXr6ZJ9NRZbmWw/OdpOgAG9Cab+ELTmkfEYgQz01cZE22xEAMvPXz57KlWPvxtE7YwYWy180Yib97EftylsNkrchbSXCwiqgKUf04qWhTgNrVuRJ9mytil6S82VNDxHzTzeCCxY412CV6dDJNLzJItpf/CXQelj/6wJs1GgFl5GWJnqortMR2v
>>>>
>>>>
>>>>
>>>>  cat /home/nodepool/.ssh/id_rsa.pub
>>>>
>>>> ssh-rsa
>>>> AAAAB3NzaC1yc2EAAAADAQABAAABAQC9gP6qui1fmHrj02p6OGvnz7kMTJ2rOC3SBYP/Ij/6yz+SU8rL5rqL6jqT30xzy9t1q0zsdJCNB2jExD4xb+NFbaoGlvjF85m12eFqP4CQenxUOdYAepf5sjV2l8WAO3ylspQ78ipLKec98NeKQwLrHB+xon6QfAHXr6ZJ9NRZbmWw/OdpOgAG9Cab+ELTmkfEYgQz01cZE22xEAMvPXz57KlWPvxtE7YwYWy180Yib97EftylsNkrchbSXCwiqgKUf04qWhTgNrVuRJ9mytil6S82VNDxHzTzeCCxY412CV6dDJNLzJItpf/CXQelj/6wJs1GgFl5GWJnqortMR2v
>>>> jenkins at jenkins-cinderci
>>>>
>>>>
>>>>
>>>> ssh ubuntu at 10.100.128.136 -v
>>>>
>>>> OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
>>>>
>>>> debug1: Reading configuration data /home/nodepool/.ssh/config
>>>>
>>>> debug1: Reading configuration data /etc/ssh/ssh_config
>>>>
>>>> debug1: /etc/ssh/ssh_config line 19: Applying options for *
>>>>
>>>> debug1: Connecting to 10.100.128.136 [10.100.128.136] port 22.
>>>>
>>>> debug1: Connection established.
>>>>
>>>> ....
>>>>
>>>> debug1: Offering RSA public key: /home/nodepool/.ssh/id_rsa
>>>>
>>>> debug1: Server accepts key: pkalg ssh-rsa blen 279
>>>>
>>>> debug1: key_parse_private2: missing begin marker
>>>>
>>>> debug1: read PEM private key done: type RSA
>>>>
>>>> debug1: Authentication succeeded (publickey).
>>>>
>>>> Authenticated to 10.100.128.136 ([10.100.128.136]:22).
>>>>
>>>> ...
>>>>
>>>>
>>>>
>>>> I was able to login into the "template" instance and also am able to
>>>> login into the "slave" instances.
>>>>
>>>> Also nodepoold was able to login into "template" instance but now it
>>>> fails loging in into "slave".
>>>>
>>>>
>>>>
>>>> I tried running it as either nodepol or jenkins users, same result.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Eduard
>>>>
>>>>
>>>>
>>>> On Mon, Jan 12, 2015 at 2:09 PM, Eduard Matei <
>>>> eduard.matei at cloudfounders.com> wrote:
>>>>
>>>>  Hi,
>>>>
>>>> Back with another error during image creation with nodepool:
>>>>
>>>> 2015-01-12 13:05:17,775 INFO nodepool.image.build.local_01.d-p-c:
>>>> Downloading python-daemon-2.0.1.tar.gz (62kB)
>>>>
>>>> 2015-01-12 13:05:18,022 INFO nodepool.image.build.local_01.d-p-c:
>>>> Traceback (most recent call last):
>>>>
>>>> 2015-01-12 13:05:18,023 INFO nodepool.image.build.local_01.d-p-c:
>>>> File "<string>", line 20, in <module>
>>>>
>>>> 2015-01-12 13:05:18,023 INFO nodepool.image.build.local_01.d-p-c:
>>>> File "/tmp/pip-build-r6RJKq/python-daemon/setup.py", line 27, in <module>
>>>>
>>>> 2015-01-12 13:05:18,024 INFO nodepool.image.build.local_01.d-p-c:
>>>>   import version
>>>>
>>>> 2015-01-12 13:05:18,024 INFO nodepool.image.build.local_01.d-p-c:
>>>> File "version.py", line 51, in <module>
>>>>
>>>> 2015-01-12 13:05:18,024 INFO nodepool.image.build.local_01.d-p-c:
>>>>   import docutils.core
>>>>
>>>> 2015-01-12 13:05:18,024 INFO nodepool.image.build.local_01.d-p-c:
>>>> ImportError: No module named docutils.core
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>> Complete output from command python setup.py egg_info:
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>> Traceback (most recent call last):
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>> File "<string>", line 20, in <module>
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>> File "/tmp/pip-build-r6RJKq/python-daemon/setup.py", line 27, in <module>
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>>
>>>> 2015-01-12 13:05:18,025 INFO nodepool.image.build.local_01.d-p-c:
>>>>   import version
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>> File "version.py", line 51, in <module>
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>>   import docutils.core
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>> ImportError: No module named docutils.core
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>>
>>>> 2015-01-12 13:05:18,026 INFO nodepool.image.build.local_01.d-p-c:
>>>> ----------------------------------------
>>>>
>>>> 2015-01-12 13:05:18,054 INFO nodepool.image.build.local_01.d-p-c:
>>>> Command "python setup.py egg_info" failed with error code 1 in
>>>> /tmp/pip-build-r6RJKq/python-daemon
>>>>
>>>>
>>>>
>>>> Python-daemon pip package fails to install due to ImportError.
>>>>
>>>>
>>>>
>>>> Any ideas how to fix this?
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Eduard
>>>>
>>>>
>>>>
>>>> On Fri, Jan 9, 2015 at 10:00 PM, Patrick East <
>>>> patrick.east at purestorage.com> wrote:
>>>>
>>>>  Thanks for the links!
>>>>
>>>>
>>>>
>>>> After digging around in my configs I figured out the issue, I had a
>>>> typo in my JENKINS_SSH_PUBLIC_KEY_NO_WHITESPACE (copy pasta cut off a
>>>> character...). But I managed to put the right one in the key for nova to
>>>> use so it was able to log in to set up the instance, but didn't end up with
>>>> the right thing in the NODEPOOL_SSH_KEY variable.
>>>>
>>>>
>>>>   -Patrick
>>>>
>>>>
>>>>
>>>> On Fri, Jan 9, 2015 at 9:25 AM, Asselin, Ramy <ramy.asselin at hp.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>> Regarding SSH Keys and logging into nodes, you need to set the
>>>> NODEPOOL_SSH_KEY variable
>>>>
>>>> 1.       I documented my notes here
>>>> https://github.com/rasselin/os-ext-testing-data/blob/master/etc/nodepool/nodepool.yaml.erb.sample#L48
>>>>
>>>> 2.       This is also documented ‘officially’ here:
>>>> https://github.com/openstack-infra/nodepool/blob/master/README.rst
>>>>
>>>> 3.       Also, I had an issue getting puppet to do the right thing
>>>> with keys, so it gets forced here:
>>>> https://github.com/rasselin/os-ext-testing/blob/master/puppet/install_master.sh#L197
>>>>
>>>>
>>>>
>>>> Ramy
>>>>
>>>>
>>>>
>>>> *From:* Eduard Matei [mailto:eduard.matei at cloudfounders.com]
>>>> *Sent:* Friday, January 09, 2015 8:58 AM
>>>> *To:* OpenStack Development Mailing List (not for usage questions)
>>>> *Subject:* Re: [openstack-dev] [OpenStack-Infra] [ThirdPartyCI] Need
>>>> help setting up CI
>>>>
>>>>
>>>>
>>>> Thanks Patrick,
>>>>
>>>> Indeed it seems the cloud provider was setting up vms on a bridge whose
>>>> eth was DOWN so the vms could not connect to the outside world so the
>>>> prepare script was failing.
>>>>
>>>> Looking into that.
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>>
>>>>
>>>> Eduard
>>>>
>>>>
>>>>
>>>> On Fri, Jan 9, 2015 at 6:44 PM, Patrick East <
>>>> patrick.east at purestorage.com> wrote:
>>>>
>>>>  Ah yea, sorry, should have specified; I am having it run
>>>> the prepare_node_devstack.sh from the infra repo. I see it adding the same
>>>> public key to the user specified in my nodepool.yaml. The strange part (and
>>>> I need to double check.. feel like it can't be right) is that on my master
>>>> node the nodepool users id_rsa changed at some point in the process.
>>>>
>>>>
>>>>
>>>>
>>>>   -Patrick
>>>>
>>>>
>>>>
>>>> On Fri, Jan 9, 2015 at 8:38 AM, Jeremy Stanley <fungi at yuggoth.org>
>>>> wrote:
>>>>
>>>> On 2015-01-09 08:28:39 -0800 (-0800), Patrick East wrote:
>>>> [...]
>>>> > On a related note, I am having issues with the ssh keys. Nodepool
>>>> > is able to log in to the node to set up the template and create an
>>>> > image from it, but then fails to log in to a build node. Have you
>>>> > run into any issues with that?
>>>>
>>>> Your image build needs to do _something_ to make SSH into the
>>>> resulting nodes possible. We accomplish that by applying a puppet
>>>> manifest which sets up an authorized_keys file for the account we
>>>> want it to use, but there are countless ways you could go about it
>>>> in your environment.
>>>> --
>>>> Jeremy Stanley
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Eduard Biceri Matei, Senior Software Developer*
>>>>
>>>>  www.cloudfounders.com
>>>>
>>>>  | eduard.matei at cloudfounders.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  *CloudFounders, The Private Cloud Software Company*
>>>>
>>>>
>>>>
>>>>  Disclaimer:
>>>>
>>>>  This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>>>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>>>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Eduard Biceri Matei, Senior Software Developer*
>>>>
>>>>  www.cloudfounders.com
>>>>
>>>>  | eduard.matei at cloudfounders.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  *CloudFounders, The Private Cloud Software Company*
>>>>
>>>>
>>>>
>>>>  Disclaimer:
>>>>
>>>>  This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>>>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>>>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Eduard Biceri Matei, Senior Software Developer*
>>>>
>>>>  www.cloudfounders.com
>>>>
>>>>  | eduard.matei at cloudfounders.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  *CloudFounders, The Private Cloud Software Company*
>>>>
>>>>
>>>>
>>>>  Disclaimer:
>>>>
>>>>  This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>>>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>>>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Eduard Biceri Matei, Senior Software Developer*
>>>>
>>>>  www.cloudfounders.com
>>>>
>>>>  | eduard.matei at cloudfounders.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  *CloudFounders, The Private Cloud Software Company*
>>>>
>>>>
>>>>
>>>>  Disclaimer:
>>>>
>>>>  This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>>>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>>>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Eduard Biceri Matei, Senior Software Developer*
>>>>
>>>>  www.cloudfounders.com
>>>>
>>>>  | eduard.matei at cloudfounders.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  *CloudFounders, The Private Cloud Software Company*
>>>>
>>>>
>>>>
>>>>  Disclaimer:
>>>>
>>>>  This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>>>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>>>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *Eduard Biceri Matei, Senior Software Developer*
>>>>
>>>>  www.cloudfounders.com
>>>>
>>>>  | eduard.matei at cloudfounders.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  *CloudFounders, The Private Cloud Software Company*
>>>>
>>>>
>>>>
>>>>  Disclaimer:
>>>>
>>>>  This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>>>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>>>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> *Eduard Biceri Matei, Senior Software Developer*
>>> www.cloudfounders.com
>>>  | eduard.matei at cloudfounders.com
>>>
>>>
>>>
>>> *CloudFounders, The Private Cloud Software Company*
>>>
>>> Disclaimer:
>>> This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>>
>>>
>>
>>
>> --
>>
>> *Eduard Biceri Matei, Senior Software Developer*
>> www.cloudfounders.com
>>  | eduard.matei at cloudfounders.com
>>
>>
>>
>> *CloudFounders, The Private Cloud Software Company*
>>
>> Disclaimer:
>> This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed.
>> If you are not the named addressee or an employee or agent responsible for delivering this message to the named addressee, you are hereby notified that you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If you have received this email in error we request you to notify us by reply e-mail and to delete all electronic files of the message. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
>> E-mail transmission cannot be guaranteed to be secure or error free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the content of this message, and shall have no liability for any loss or damage suffered by the user, which arise as a result of e-mail transmission.
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

*Eduard Biceri Matei, Senior Software Developer*
www.cloudfounders.com
 | eduard.matei at cloudfounders.com



*CloudFounders, The Private Cloud Software Company*

Disclaimer:
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed.
If you are not the named addressee or an employee or agent responsible
for delivering this message to the named addressee, you are hereby
notified that you are not authorized to read, print, retain, copy or
disseminate this message or any part of it. If you have received this
email in error we request you to notify us by reply e-mail and to
delete all electronic files of the message. If you are not the
intended recipient you are notified that disclosing, copying,
distributing or taking any action in reliance on the contents of this
information is strictly prohibited.
E-mail transmission cannot be guaranteed to be secure or error free as
information could be intercepted, corrupted, lost, destroyed, arrive
late or incomplete, or contain viruses. The sender therefore does not
accept liability for any errors or omissions in the content of this
message, and shall have no liability for any loss or damage suffered
by the user, which arise as a result of e-mail transmission.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150115/0a9ab700/attachment.html>


More information about the OpenStack-dev mailing list