- openstack-discuss - lists.openstack.org

Re: Fwd: openstack queens magnum error
by Feilong Wang 14 Dec '18

14 Dec '18

Seems your master node can't talk to your keystone, can you confirm that? On 13/12/18 11:08 AM, Ignazio Cassano wrote: > Hello Feilong, this forwarding contains logs. > Thanks > > ---------- Forwarded message --------- > From: *Ignazio Cassano* <ignaziocassano(a)gmail.com > <mailto:ignaziocassano@gmail.com>> > Date: Mer 12 Dic 2018 10:06 > Subject: openstack queens magnum error > To: OpenStack Operators <openstack-operators(a)lists.openstack.org > <mailto:openstack-operators@lists.openstack.org>> > > > Hello Everyone, > > I installed queens on centos with magnum and I am trying to create a > swarm cluster with one muster and one node. The image I used is > fedora-atomic 27 update 04 > > The stack generated end with an error and magnum conductor reports: > ec 12 09:51:17 tst2-osctrl01 magnum-conductor: 2018-12-12 09:51:17.304 > 17964 WARNING magnum.drivers.heat.template_def > [req-bfa19294-5671-47a0-b0ac-9e544f0e5e38 - - - - -] stack does not > have output_key api_address > Dec 12 09:51:17 tst2-osctrl01 magnum-conductor: 2018-12-12 > 09:51:17.305 17964 WARNING magnum.drivers.heat.template_def > [req-bfa19294-5671-47a0-b0ac-9e544f0e5e38 - - - - -] stack does not > have output_key swarm_masters > Dec 12 09:51:17 tst2-osctrl01 magnum-conductor: 2018-12-12 > 09:51:17.306 17964 WARNING magnum.drivers.heat.template_def > [req-bfa19294-5671-47a0-b0ac-9e544f0e5e38 - - - - -] stack does not > have output_key swarm_nodes > Dec 12 09:51:17 tst2-osctrl01 magnum-conductor: 2018-12-12 > 09:51:17.306 17964 WARNING magnum.drivers.heat.template_def > [req-bfa19294-5671-47a0-b0ac-9e544f0e5e38 - - - - -] stack does not > have output_key discovery_url > Dec 12 09:51:17 tst2-osctrl01 magnum-conductor: 2018-12-12 > 09:51:17.317 17964 ERROR magnum.drivers.heat.driver > [req-bfa19294-5671-47a0-b0ac-9e544f0e5e38 - - - - -] Cluster error, > stack status: CREATE_FAILED, stack_id: > 306bd83a-7878-4d94-8ed0-1d297eec9768, reason: Resource CREATE failed: > WaitConditionFailure: > resources.swarm_nodes.resources[0].resources.node_wait_condition: > swarm-agent service failed to start. > > > > I connected to the master node for verifyng if swarm agent is running. > In the cloud init log I found: > requests.exceptions.ConnectionError: > HTTPConnectionPool(host='10.102.184.190', port=5000): Max retries > exceeded with url: /v3//auth/tokens (Caused by > NewConnectionError('<urllib3.connection.HTTPConnection object at > 0x7f0814d4d250>: Failed to establish a new connection: [Errno 110] > Connection timed out',)) > Cloud-init v. 0.7.9 running 'modules:final' at Wed, 12 Dec 2018 > 08:45:31 +0000. Up 55.54 seconds. > 2018-12-12 08:47:45,858 - util.py[WARNING]: Failed running > /var/lib/cloud/instance/scripts/part-005 [1] > /var/lib/cloud/instance/scripts/part-006: line 13: > /etc/etcd/etcd.conf: No such file or directory > /var/lib/cloud/instance/scripts/part-006: line 26: > /etc/etcd/etcd.conf: No such file or directory > /var/lib/cloud/instance/scripts/part-006: line 38: > /etc/etcd/etcd.conf: No such file or directory > 2018-12-12 08:47:45,870 - util.py[WARNING]: Failed running > /var/lib/cloud/instance/scripts/part-006 [1] > Configuring docker network ... > Configuring docker network service ... > Removed > /etc/systemd/system/multi-user.target.wants/docker-storage-setup.service. > New size given (1280 extents) not larger than existing size (4863 > extents) > ERROR: There is not enough free space in volume group atomicos to > create data volume of size MIN_DATA_SIZE=2G. > 2018-12-12 08:47:46,206 - util.py[WARNING]: Failed running > /var/lib/cloud/instance/scripts/part-010 [1] > + systemctl stop docker > + echo 'starting services' > starting services > + systemctl daemon-reload > + for service in etcd docker.socket docker swarm-manager > + echo 'activating service etcd' > activating service etcd > + systemctl enable etcd > Failed to enable unit: Unit file etcd.service does not exist. > + systemctl --no-block start etcd > Failed to start etcd.service: Unit etcd.service not found. > + for service in etcd docker.socket docker swarm-manager > + echo 'activating service docker.socket' > activating service docker.socket > + systemctl enable docker.socket > > > 1) Seems etcd service is not installed , > > 2) the instance required to contact controller on port 5000 (is it > correct ?) > > > Please help me. > > Regards > Ignazio > > -- Cheers & Best regards, Feilong Wang (王飞龙) -------------------------------------------------------------------------- Senior Cloud Software Engineer Tel: +64-48032246 Email: flwang(a)catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington --------------------------------------------------------------------------

3 5

Queens magnum external network
by Ignazio Cassano 14 Dec '18

14 Dec '18

Hi All, I am testing magnum on Centos openstack queens. I saw magnum external network must speak with some controller services ....for example 5000 for keystone and 8004 for heat . Are there any other port to open for external network? Regards Ignazio

1 0

[cyborg][ResMgmt]Cyborg Ecosystem Building Planning For 2019
by Zhipeng Huang 14 Dec '18

14 Dec '18

Hi team, After the wrapping up of KubeCon this week, I have some initial thoughts on eco building for next year 1. Kubernetes We will be working on a proposal for heterogeneous resource support in k8s. This is not a new idea and we have been floating the idea around for almost a year, and based upon the discussion we had through the week, it seems a good time to do it now. Detailed proposal will come out early next year I hope for public review and discussion. To have k8s integration out is on the direction of having more form of compute with acceleration support. We will also kickstart the resource management sig activity which has been earlier this year (via email mostly) 2. RISC-V I would propose we start to talk and work with the FireSIM team to try out how could we better support RISC-V arch from a cloud perspective. 3. Edge I'm interested in have raspberry pi support with our driver gathering info from libgpio and other interfaces. It would be great to have ARM embedded devs to join the discussion :) Look forward to feedbacks.

1 0

NTP failed on deploy
by Lior Fellus 14 Dec '18

14 Dec '18

I'm doing a fresh deploy of OpenStack using Fuel, with 3 controllers and 4 compute nodes. Network verification passes without issues, but during the OpenStack installation phase on the new compute I am trying to add I get this message: *Failed to deploy node '...': Task sync_time failed on node * I can ping the NTP servers (fuel server) from the node so connectivity shouldn't be an issue. we are using Mirantis Mitaka. Thanks,

1 0

Neutron performance Ocata
by Matteo Dacrema 14 Dec '18

14 Dec '18

Hi all, I’ve recently upgraded Openstack from Mitaka to Ocata and now I’m facing an increased latency in instance creation. Before the upgrade I was able to launch sequentially 400 VMs in 5 minutes and now I need at least 15 minutes. Attached the script I use to measure timings. Here you can find the comparison between Mitaka and Ocata. I also noticed that in Ocata DHCP agent is slower and sometimes when I launch more than 30 VMs sequentially some of them doesn’t took an address. Time of instance creation increase with the number of ports in the network. https://docs.google.com/spreadsheets/d/1eHjRxkw_5BmJ2p0rBJg9suGKmu_1YrSY05f… <https://docs.google.com/spreadsheets/d/1eHjRxkw_5BmJ2p0rBJg9suGKmu_1YrSY05f…> I’ve enabled the osprofiler inside nova and neutron api and attached you can see that the most of the time is spent on wsgi component of neutron. The time is spent on getting port list. I’ve also verified with cli listing only ports and I can confirm is a port list related issue. I’m trying to profile the neutron-server with cProfile but I’ve some problems with multi-threading tracing. Does anyone have faced this issue? Thank you! Regards Matteo -------------------------------------------- This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

2 1

[TripleO] - Blueprint - tripleo-routed-networks-templates
by Harald Jensås 14 Dec '18

14 Dec '18

Hello TripleO Team, I have been working on the tripleo-routed-networks-templates blueprint for some time. There is a number of patches in need of reviews. Link # https://review.openstack.org/#/q/topic:bp/tripleo-routed-networks-templates… ) Most of the patches are based on top of eachoter starting with: - https://review.openstack.org/582180 One of the hurdles with all this is having an environment with a network infrastructure to run tests, routers, dhcp-relay server etc. For some time I have been collaborating with Ben Nemec on routed- networks support in OVB. And it is working really well! \o/ If anyone wants to try this out, I have set up a github repository[1] with some scripts to set up the OVB environment in rdocloud. The ansible playbook in that repo will Deploy the undercloud (including all the THT patches), build images and import the baremetal nodes. Overcloud environment files, templates and a deploy script is there in the repo as well. For the overcloud there are two set's of templates and accompanying deploy command in the git repo. The deploy command and templates with suffix '_pre_update' will deploy 1 Controller and 1 Compute node without subnets, a single leaf. By deploying using these first, and then deploying the using the templates and deploy command without suffix we can verify that the existing non-*neutron routed networks* is migrated to *neutron routed networks* with multiple subnets with network segment association during stack update. The commands to run Tempest is also in the Readme. Today's `tempest run --smoke` results: ====== Totals ====== Ran: 99 tests in 486.0000 sec. - Passed: 90 - Skipped: 9 - Expected Fail: 0 - Unexpected Success: 0 - Failed: 0 Sum of execute time for each test: 1028.2578 sec. I hope that we can merge these changes soon, so that they are available in early for downstream QA. Thank you! -- Cheers, Harald [1] https://github.com/hjensas/ooo-bp-tripleo-routed-networks-templates-testing

1 0

[cinder] [tempest] Ideas for test cases
by Sofia Enriquez 13 Dec '18

13 Dec '18

Hello cinder guys, I'm working on increasing the coverage of Cinder Tempest [1]. Since I'm relatively new using cinder retype+migrate functionality, I'm looking for possible ideas of test cases. Thanks, Sofi [1]: https://review.openstack.org/#/c/614022/ -- Sofia Enriquez Associate Software Engineer <https://red.ht/sig>

2 2

[senlin] Senlin Monthly(ish) Newsletter Nov/Dec 2018
by Duc Truong 13 Dec '18

13 Dec '18

HTML: https://dkt26111.wordpress.com/2018/12/13/senlin-monthlyish-newsletter-nove… This is the November/December edition of the Senlin monthly(ish) newsletter. The goal of the newsletter is to highlight happenings in the Senlin project. If you have any feedback or questions regarding the contents, please feel free to reach out to me in the #senlin IRC channel. News ---- * The Senlin Project update video and slides from the Berlin summit have been posted: https://www.openstack.org/videos/summits/berlin-2018/senlin-project-update-1 * Autoscaling forum was held at the Berlin Summit. Notes have been captured here: https://etherpad.openstack.org/p/autoscaling-integration-and-feedback * A new autoscaling SIG has been proposed: http://lists.openstack.org/pipermail/openstack-discuss/2018-November/000284… * The weekly meeting was changed to have alternating times for even and odd weeks to allow for participation from developers in US / Europe: http://eavesdrop.openstack.org/#Senlin_Team_Meeting * This week is an even numbered week, so the meeting will be held on Friday at 530 UTC. Blueprint Status ---------------- * Fail fast locked resource - https://blueprints.launchpad.net/senlin/+spec/fail-fast-locked-resource - Working on documentation and release notes. * Multiple detection modes - https://blueprints.launchpad.net/senlin/+spec/multiple-detection-modes - Working on documentation and release notes. * Fail-fast on cooldown for scaling operations - https://blueprints.launchpad.net/senlin/+spec/scaling-action-acceptance - Completed * Action update - https://blueprints.launchpad.net/senlin/+spec/action-update - Basic functionality has been merged. - Currently working on force action update. Community Goal Status --------------------- * Python 3 - All patches by Python 3 goal champions for zuul migration, documentation and unit test changes have been merged. * Upgrade Checkers - The basic framework for the senlin-status upgrade check has been merged (thanks to rajathere and mriedem). It only has a placeholder check for now. If you like to help out with this task and implement the Senlin specific checks, please let me know. Reviews Needed -------------- * Health policy tempest tests: https://review.openstack.org/#/c/620716/ * Convert requests response from byte to string: https://review.openstack.org/#/c/624248/ * Fix senlin client to work with latest openstacksdk: https://review.openstack.org/#/c/624528/ * Force action update: https://review.openstack.org/#/c/624534/ * Fix senlin-dashboard bug when creating openstacksdk connection (Needs https://review.openstack.org/#/c/624528/ to merge first): https://review.openstack.org/#/c/620767/ * Update health manager to not spawn multiple tasks: https://review.openstack.org/#/c/621504/

1 0

[all]Forum summary: Expose SIGs and WGs
by Rico Lin 13 Dec '18

13 Dec '18

Dear all Here is the summary for forum `Expose SIGs and WGs` (etherpad [1] ). This concept still under development, so this is an open discussion and we need more feedbacks. Here are some general agreements on actions or ideas that we think it's worth to find the answer. *Set up guidelines for SIGs/WGs/Teams for interaction specific to this around tracking cross-project work* We tend to agree that we kind of lack for a guideline or a sample for SIGs/WGs, since all SIGs/WGs formed for different interest, we won't try to unify tools (unless that's what everyone agrees on) or rules for all groups. What we can do is to give more help to groups and provide a clear way for how they can set up cross-project works if they want to. Also, we can provide information on how to reach to users, ops, and developers and bridge them up. And we can even do a more general guideline or sample on how other SIGs/WGs are doing with their own workflow. Like self-healing SIG working on getting user story and feedback and use them to general better document/guideline for other users. Also, public cloud WG working on collect issues from public cloud providers and bring those issues to projects. Those IMO are great examples that we should put them down somewhere for cross SIGs/WGs consideration. As a further idea, we can even discuss if it's a common interest to have a SIG to help on SIGs. *A workflow for tracking:* This kind of follow above item. If we're going to set up a workflow, what we can add in to help people live an easier life? This is also an idea that no one in the room thinks it's a bad one, so it means in long term, it might worth our time to provide more clear information on what exactly workflow that we suggest everyone use. *Discuss SIG spec repo*: The discussion here is how can we monitoring SIGs/WGs health and track tasks. When talking about tasks we not just talking about bugs, but also features that's been considered as essential tasks for SIGs/WGs. We need a place to put them down in a trackable way (from a user story to a patch for implementation). *Ask foundation about have project update for SIGs/WGs*: One action we can start right now is to let SIGs/WGs present a project update (or even like a session but give each group 5 mins to present). This should help group getting more attention. And even capable to send out messages like what's the most important features or bug fixes they need from project teams, or what's the most important tasks that are under planning or working on. Fortunately, we got Melvin Hillsman (UC) volunteer on this task. We also have some real story (Luzi's story) for people to get a better understanding of why current workflow can look like for someone who tries to help. The thing that we also wish to do is to clear the message here. We think most of the tools are already there, so we shouldn't need to ask project teams to do any huge change. But still, we found there are definitely some improvements that we can do to better bridge users, ops, and developers. You might find some information here didn't give you a clear answer. And that's because of we still under open discussion for this. And I assume we gonna keep finding actions from discussions that we can do right away. We will try to avoid that we have to do the exact same session with the same argument over and over again. So please give your feedback, any idea, or give us your help if you also care about this. [1] https://etherpad.openstack.org/p/expose-sigs-and-wgs -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

8 13

[glance]functional-py35 fails due to subunit.praser error
by Abhishek Kekane 13 Dec '18

13 Dec '18

[glance]functional-py35 fails due to subunit.praser error Hi All, >From 7th December 2018, all of sudden functional-py35 check job is failing on current master. Earlier this job was failing intermittently (2-3 times from 100). I have tried to dig into the issue and find out that it is failing obly for ubuntu and working fine with Fedora, centos etc. Below are some of the findings: 1. Even if I add failure tests [1] in black-list file every time other tests reported as failure. 2. When I compared failed tests results [1] with passed tests results [2] I have found that passing tests shows 'Pass 741 Skip 7' whereas failing tests shows 'Pass 627 Failure 9 Skip 5' (it does not execute all the tests in case of failure) 3. I have compared version of subunit, oslo.service, oslo.log, tox with failed [3] and passed [4] logs and they are same. 4. I came across a bug in cinder [5] which talks about excessive logs might cause this error and I have tried to ignore the deprecation warning in tox.ini by adding 'PYTHONWARNINGS=ignore::DeprecationWarning' but it still shows DeprecationWarning in the logs. 5. I have tried executing tests by setting '--concurrency' to 1 but still the error persists. 6. Whenver subunit.parser error occurs the tests remains 'inprogress' state and sometimes time-outs. I have no clues so far to tackle this issue, kindly provide your inputs for the same. [1] http://logs.openstack.org/89/617889/4/check/openstack-tox-functional-py35/4… [2] http://logs.openstack.org/89/617889/4/check/openstack-tox-functional-py35/3… [3] http://logs.openstack.org/89/617889/4/check/openstack-tox-functional-py35/4… [4] http://logs.openstack.org/89/617889/4/check/openstack-tox-functional-py35/3… [5] https://bugs.launchpad.net/cinder/+bug/1728640 Thanks & Best Regards, Abhishek Kekane

1 0