- openstack-discuss - lists.openstack.org

[kolla] - Ceph bootstrap monitor is not properly configured
by Tyler Bishop 01 Mar '19

01 Mar '19

Trying to deploy a new cluster using the bootstrapping for ceph but running into issues with the admin keys being incorrectly deployed. Successful ansible deploy up until: TASK [ceph : Getting ceph mgr keyring] ****************************************************************************************************************************** failed: [osctlr.home.visualbits.net -> osctlr.home.visualbits.net] (item= osctlr.home.visualbits.net) => {"changed": false, "item": " osctlr.home.visualbits.net", "msg": "Failed to call command: ['docker', 'exec', 'ceph_mon', 'ceph', '--format', 'json', 'auth', 'get-or-create', ' mgr.osctlr.home.visualbits.net', 'mds', 'allow *', 'mon', 'allow profile mgr', 'osd', 'allow *'] returncode: 1 output: stdout: \"\", stderr: \"[errno 1] error connecting to the cluster\n\""} Errors in log from ceph: TASK [ceph : Getting ceph mgr keyring] ****************************************************************************************************************************** failed: [osctlr.home.visualbits.net -> osctlr.home.visualbits.net] (item= osctlr.home.visualbits.net) => {"changed": false, "item": " osctlr.home.visualbits.net", "msg": "Failed to call command: ['docker', 'exec', 'ceph_mon', 'ceph', '--format', 'json', 'auth', 'get-or-create', ' mgr.osctlr.home.visualbits.net', 'mds', 'allow *', 'mon', 'allow profile mgr', 'osd', 'allow *'] returncode: 1 output: stdout: \"\", stderr: \"[errno 1] error connecting to the cluster\n\""} keyrings look proper: (openstack) [root@osctlr ~]# md5sum /etc/kolla/ceph-mon/ceph.client.admin.keyring 4658c01282c791bce9c75678df9e21c9 /etc/kolla/ceph-mon/ceph.client.admin.keyring (openstack) [root@osctlr ~]# md5sum /var/lib/docker/volumes/ceph_mon_config/_data/ceph.client.admin.keyring 4658c01282c791bce9c75678df9e21c9 /var/lib/docker/volumes/ceph_mon_config/_data/ceph.client.admin.keyring I've removed the docker container, volume and kolla config directories multple times with the same error. I can't even run ceph status from the container bash itself. Any ideas?

2 2

RE: [nova] NUMA live migration - mostly how it's tested
by Zhong, Luyao 01 Mar '19

01 Mar '19

Hi all, There was something wrong with the live migration when using 'dedicated' cpu_policy in my test. Attached file contains the details. (I remove the original mail message otherwise the email message body will be so big that it will be held)

1 0

nimble PyPI status
by Devin Shanahan 28 Feb '19

28 Feb '19

Dear maintainer of nimble: I am reaching out since you are listed as a maintainer of nimble <https://pypi.org/project/nimble/> on PyPI. I am currently working on a project and we are exploring releasing our distribution under the same name. Our project is a library of data science and machine learning tools. It has been under development and used internally at our company for 7 years, and we are now considering providing it as an open-source project. Nimble is a metaphor for our team's vision for the project, and we would very much prefer to use that exact name on PyPI over a similar, but more complicated name, that could cause confusion. Since PyPI shows that nimble has not had a new release or been updated since August 10, 2016, I would like to know if you would be willing to relinquish the name, nimble, so that we could provide our project in that namespace on PyPI. It's a simple process and if you are willing to do this, we would be extremely grateful. I also completely understand if this is not something that you are willing to do, or if I have misinterpreted the status of nimble and it is in fact in active development. I would really appreciate hearing your reply either way, whether or not you would be willing to consider transferring ownership of nimble over to us. Thank you very much for your time. Devin

3 2

Router not allowing traffic in reverse
by John Carew 28 Feb '19

28 Feb '19

I have setup OpenStack with OVS. I have a single Hyper-V server running the controller and three CentOS instances(10.0.0.x) on a private subnet. I created a router in OpenStack with SNAT disabled, as I only want it to route traffic between the private subnet(10.0.0.x) and the external subnet(172.16.1.x)/internet. All of the instances can ping each other along with the external network(172.16.1.x). From the external network, I can ping the interface of the ovs router on the external network. I can not though ping inside the private network. A trace route stops at the IP of the OVS router. With wireshark, I do not see anything coming from the external pc’s IP. If I trace route it, I see packets making all the way to the OVS router and then stop. Since I can ping one way, and not the other; I believe there is something in the router/OVS that is stopping the packets to route into the private subnet. What do I need to look at? (I have disabled all firewalls on all OSes involved.) Here is a sudo design of the setup. Does respond to ping VM OVS Router priv int OVS Router ext int External PC 10.0.0.14 -> 10.0.0.1 -> 172.16.1.1 -> 172.16.1.10 Does not respond to ping External PC OVS Router ext int OVS Router priv int VM 172.16.1.10 -> 172.16.1.1 -> 10.0.0.1 -> 10.0.0.14

1 0

devstack doc for using the nova fake virt driver
by Matt Riedemann 28 Feb '19

28 Feb '19

I realize some people might not be aware of this capability in devstack so I wrote a docs patch [1] after someone was asking about it in the nova channel, hopefully it's useful to others. [1] https://review.openstack.org/#/c/640205/ -- Thanks, Matt

1 0

NDSU Capstone Introduction!
by Urbano Moreno, Eduardo 28 Feb '19

28 Feb '19

Hello OpenStack community, I just wanted to go ahead and introduce myself, as I am a part of the NDSU Capstone group! My name is Eduardo Urbano and I am a Jr/Senior at NDSU. I am currently majoring in Computer Science, with no minor although that could change towards graduation. I am currently an intern at an electrical supply company here in Fargo, North Dakota known as Border States. I am an information security intern and I am enjoying it so far. I have learned many interesting security things and have also became a little paranoid of how easily someone can get hacked haha. Anyways, I am so excited to be on board and be working with OpenStack for this semester. So far I have learned many new things and I can't wait to continue on learning. Thank you! -Eduardo

6 6

[neutron] Issue in neutron-tempest-iptables_hybrid gate job
by Slawomir Kaplonski 28 Feb '19

28 Feb '19

Hi Neutrinos, Just FYI, we have bug in os-vif [1] which cause failures of (at least) neutron-tempest-iptables_hybrid. So if Zuul is failing on Your patch because of this job, please don’t recheck it until patch [2] will be merged and new os-vif will be released. [1] https://bugs.launchpad.net/os-vif/+bug/1817919 [2] https://review.openstack.org/#/c/639702/ — Slawek Kaplonski Senior software engineer Red Hat

3 4

[ops] [nova] Wrong reported memory hypervisor usage
by Massimo Sgaravatto 28 Feb '19

28 Feb '19

In the dashboard of my OpenStack Ocata cloud I see that the reported memory usage is wrong for the hypervisors. As far as I understand that information should correspond to the "used_now" field of the "nova host-describe" command. And indeed considering an hypervisor: # nova host-describe cld-blu-01.cloud.pd.infn.it +-----------------------------+----------------------------------+-----+-----------+---------+ | HOST | PROJECT | cpu | memory_mb | disk_gb | +-----------------------------+----------------------------------+-----+-----------+---------+ | cld-blu-01.cloud.pd.infn.it | (total) | 8 | 32722 | 241 | | cld-blu-01.cloud.pd.infn.it | (used_now) | 19 | 16259 | 23 | | cld-blu-01.cloud.pd.infn.it | (used_max) | 19 | 38912 | 95 | | cld-blu-01.cloud.pd.infn.it | b08eede75d5e4be4b0fe21e68fa9c688 | 1 | 2048 | 20 | | cld-blu-01.cloud.pd.infn.it | 7890b3e262264529a19f9743cf2f14bc | 16 | 32768 | 50 | | cld-blu-01.cloud.pd.infn.it | 4d8187cffa6a4085ad4a357b8a5afc03 | 2 | 4096 | 25 | +-----------------------------+----------------------------------+-----+-----------+---------+ So for the memory it reports 16259 for used_now, while as far as far I understand it should be 38912 + the memory used by the hypervisor Am I missing something ? Thanks, Massimo

3 4

[release] Release countdown for week R-5, March 4-8
by Sean McGinnis 28 Feb '19

28 Feb '19

We are getting close to the end of the cycle. Please review some important deadlines and information as we approach feature freeze and the cycle ending activities. Development Focus ----------------- Teams should be focused on implementing planned work. Work should be wrapping up on client libraries to meet the client lib deadline Thursday, the 7th. This is also Feature Freeze, so please review anything to determine if should still be targeted for Stein or will need to be deferred to Train. General Information ------------------- The final client library release is on Thursday the 7th. Releases will only be allowed for critical fixes in libraries after this point as we stabilize requirements and give time for any unforeseen impacts from lib changes to trickle through. If release critical library or client library releases are needed for Stein past the freeze dates, you must request a Feature Freeze Exception (FFE) from the requirements team before we can do a new release to avoid having something released in Rocky that is not actually usable. This is done by posting to the openstack-discuss mailing list with a subject line similar to: [$PROJECT][requirements] FFE requested for $PROJECT_LIB Include justification/reasoning for why a FFE is needed for this lib. If/when the requirements team OKs the post-freeze update, we can then process a new release. Including a link to the FFE in the release request is not required, but would be helpful in making sure we are clear to do a new release. When requesting these library releases, you should also include the stable branching request with the review (as an example, see the "branches" section here: http://git.openstack.org/cgit/openstack/releases/tree/deliverables/pike/os-…) Cycle-trailing projects are reminded that all reviews to the requirements project will have a procedural -2 unless it recieves a FFE until stable/stein is branched. This is a good point to start to think about communicating your project's updates for Stein. There are two venues for this that should be kept in mind: 1. The Prelude section of reno release notes is rendered as the top level overview for the release. Any important overall messaging for Stein changes can be added there to make sure the consumers of your release notes see them. 2. Through the releases repo, teams should be adding cycle-highlights to their deliverables. This is the direct input into marketing that helps shape what is communicated in press releases and other release activity at the end of the cycle. These are not per-deliverable, but rolled up per-team. So any significant work your team has done that would be good for marketing, please include these by the RC1 deadline. Upcoming Deadlines & Dates -------------------------- Stein-3 milestone: March 7 RC1 deadline: March 21 -- Sean McGinnis (smcginnis)

1 0

Re: [placement][tripleo][infra] zuul job dependencies for greater good?
by Bogdan Dobrelya 28 Feb '19

28 Feb '19

> On Mon, 2019-02-25 at 19:42 -0500, Clark Boylan wrote: >> On Mon, Feb 25, 2019, at 12:51 PM, Ben Nemec wrote: >> > > snip > >> That said, I wouldn't push too hard in either direction until someone >> crunched the numbers and figured out how much time it would have saved >> to not run long tests on patch sets with failing unit tests. I feel like >> it's probably possible to figure that out, and if so then we should do >> it before making any big decisions on this. > > For numbers the elastic-recheck tool [0] gives us fairly accurate tracking of which issues in the system cause tests to fail. You can use this as a starting point to potentially figure out how expensive indentation errors caught by the pep8 jobs ends up being or how often unittests fail. You probably need to tweak the queries there to get that specific though. > > Periodically I also dump node resource utilization by project, repo, and job [1]. I haven't automated this because Tobiash has written a much better thing that has Zuul inject this into graphite and we should be able to set up a grafana dashboard for that in the future instead. > > These numbers won't tell a whole story, but should paint a fairly accurate high level picture of the types of things we should look at to be more node efficient and "time in gate" efficient. Looking at these two really quickly myself it seems that job timeouts are a big cost (anyone looking into why our jobs timeout?). > > [0] http://status.openstack.org/elastic-recheck/index.html > [1] http://paste.openstack.org/show/746083/ > > Hope this helps, > Clark Here is some numbers [0] extracted via elastic-recheck console queries. It shows 6% of wasted failures because of tox issues in general, and 3% for tripleo projects in particular. My final take is, given some middle-ground solution, like I illustrated earlier this sub-thread, it might be worth it, and the effort for boosting up the total throughput of openstack CI system by a 6% is not so bad idea. [0] http://paste.openstack.org/show/746503/ -- Best regards, Bogdan Dobrelya, Irc #bogdando

3 5