- openstack-discuss - lists.openstack.org

[dev][tc] Part 2: Evaluating projects in relation to OpenStack cloud vision
by Chris Dent 14 Feb '19

14 Feb '19

This a "part 2" or "other half" of evaluating OpenStack projects in relation to the technical vision. See the other threads [1][2] for more information. In the conversations that led up to the creation of the vision document [3] one of the things we hoped was that the process could help identify ways in which existing projects could evolve to be better at what they do. This was couched in two ideas: * Helping to make sure that OpenStack continuously improves, in the right direction. * Helping to make sure that developers were working on projects that leaned more towards interesting and educational than frustrating and embarrassing, where choices about what to do and how to do it were straightforward, easy to share with others, so well-founded in agreed good practice that argument would be rare, and so few that it was easy to decide. Of course, to have a "right direction" you first have to have a direction, and thus the vision document and the idea of evaluating how aligned a project is with that. The other half, then, is looking at the projects from a development standpoint and thinking about what aspects of the project are: * Things (techniques, tools) the project contributors would encourage others to try. Stuff that has worked out well. * Things—given a clean slate, unlimited time and resources, the benefit of hindsight and without the weight of legacy—the project contributors would encourage others to not repeat. And documenting those things so they can be carried forward in time some place other than people's heads, and new projects or refactorings of existing projects can start on a good foot. A couple of examples: * Whatever we might say about the implementation (in itself and how it is used), the concept of a unified configuration file format, via oslo_config, is probably considered a good choice, and we should keep on doing that. * On the other hand, given hindsight and improvements in commonly available tools, using a homegrown WSGI (non-)framework (unless you are Swift) plus eventlet may not have been the way to go, yet because it is what's still there in nova, it often gets copied. It's not clear at this point whether these sorts of things should be documented in projects, or somewhere more central. So perhaps we can just talk about it here in email and figure something out. I'll followup with some I have for placement, since that's the project I've given the most attention. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001417.… [2] http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002524… [3] https://governance.openstack.org/tc/reference/technical-vision.html -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

5 11

[kolla] State of SELinux support
by Jason Anderson 14 Feb '19

14 Feb '19

Hey all, With CVE-2019-5736<https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-5736> dropping today, I thought it would be a good opportunity to poke about the current state of SELinux support in Kolla. The docs<https://docs.openstack.org/kolla-ansible/rocky/user/security.html> have said it is a work in progress since the Mitaka release at least. I did find a spec<https://blueprints.launchpad.net/kolla/+spec/enable-selinux> that was marked as completed, but I am not aware that there is yet any support and I see that the baremetal role still forces SELinux to "permissive" by default. Is anybody currently working on this or is there an update spec/blueprint to track the development here? I am no SELinux expert by any means but this feels like an important thing to address, particularly if Docker has made it easier to label bind mounts<https://docs.docker.com/storage/bind-mounts/#configure-the-selinux-label>. Thanks! Jason Anderson Cloud Computing Software Developer Consortium for Advanced Science and Engineering, The University of Chicago Mathematics & Computer Science Division, Argonne National Laboratory

2 1

[kolla][mariadb] Multinode deployment fails due to bootstrap_mariadb or mariadb errors
by Giuseppe Sannino 14 Feb '19

14 Feb '19

Hi all, need your help. I'm trying to deploy Openstack "Queens" via kolla on a multinode system (1 controller/kolla host + 1 compute). I tried with both binary and source packages and I'm using "ubuntu" as base_distro. The first attempt of deployment systematically fails here: TASK [mariadb : Running MariaDB bootstrap container] ******************************************************************************************************************************************************************************************************** fatal: [xx.yy.zz.136]: FAILED! => {"changed": true, "msg": "Container exited with non-zero return code 1"} Looking at the bootstrap_mariadb container logs I can see: ---------- Neither host 'xxyyzz' nor 'localhost' could be looked up with '/usr/sbin/resolveip' Please configure the 'hostname' command to return a correct hostname. ---------- Any idea ? Thanks a lot /Giuseppe

3 3

[blazar] Question about Interaction between Ironic and Blazar
by Tzu-Mainn Chen 14 Feb '19

14 Feb '19

Hi! I'm working with both Ironic and Blazar, and came across a strange interaction that I was wondering if the Blazar devs were aware of. I had four Ironic nodes registered, and only node A had an instance running on it. I tried adding node B - which was available - to the freepool and got this error: 2019-02-13 09:42:28.560 220255 ERROR oslo_messaging.rpc.server ERROR: Servers [[{u'uuid': u'298e83a4-7d5e-4aae-b89a-9dc74b4278af', u'name': u'instance-00000011'}]] found for host a00696d5-32ba-475e-9528-59bf11cffea6 This was strange, because the instance in question was running on node A, and not node B. After some investigation, the cause was identified as the following: https://bugs.launchpad.net/nova/+bug/1815793 But in the meantime, my question is: have other people using Blazar and Ironic run into this issue? It would seem to imply that Ironic nodes can only be added to the freepool if no instances are created, which poses a long-term maintenance issue. Is there a workaround? Thanks, Tzu-Mainn Chen

2 1

[tc][all][self-healing-sig] Service-side health checks community goal for Train cycle
by Jean-Philippe Evrard 14 Feb '19

14 Feb '19

Hello everyone, As you might have seen on the ML, two of the 3 top contenders for the Train cycle community got some traction. Let's here talk about the last one: The service-side health checks. While people were interested in this goal previously, nobody really came forward on the pre-work. Last week, I met a few of my colleagues to see what we can do together. Matt (irc: mattoliverau), Adam (irc: aspiers), and I discussed about the different ways to implement this new API, with the help of many in #openstack-sdk. Long story short, the current framework might be "good enough" for extension already, as we could have extra "backends" (basically "tests"), to increase the coverage of this healthcheck endpoint. While the immediate next step would be to work on the v2 prototype that Graham started (see link [1], anyone is welcome to help there!), the next step would be far easier if it was crowd sourced: We need to know which service is already using that oslo middleware, which service doesn't want to use it, and which service is already ready for healtchecks. When we'll have a lay of the land, we'll know where the energy will be spent in this community goal: Would that be bringing oslo.middleware to <insert amount here> services or bringing common "backends" that can be used by each service (like DB/MQ/cache checks). I would be very happy if you could have a look at this ethercal [2], and add/edit your project capabilities there. Thank you in advance. Jean-Philippe Evrard (evrardjp) [1]: https://review.openstack.org/#/c/617924/ [2]: https://ethercalc.openstack.org/di0mxkiepll8

4 5

[tripleo][ci] WARNING: All CI affected by a DPDK version bump
by Felix Enrique Llorente Pastora 14 Feb '19

14 Feb '19

Hi All, All the jobs from tripleo CI are currently failing at check, gate and promotions. There is a compatibility issue between OVS and DPDK versions we have a bypass to exclude problematic version. LP: https://bugs.launchpad.net/tripleo/+bug/1815863 Possible bypass: https://review.openstack.org/636860 BR -- Quique Llorente Openstack TripleO CI

1 0

[tc][uc] Becoming an Open Source Initiative affiliate org
by Thierry Carrez 14 Feb '19

14 Feb '19

I started a thread on the Foundation mailing-list about the OSF becoming an OSI affiliate org: http://lists.openstack.org/pipermail/foundation/2019-February/002680.html Please follow-up there is you have any concerns or questions. -- Thierry Carrez (ttx)

5 4

[dev] [mistral] Deprecating keystone_authtoken/auth_uri
by Tobias Urdin 13 Feb '19

13 Feb '19

Hello all Mistral people, This patch [1] has been up for a long time about the removal (which probably should a deprecation?) (and support for www_authenticate_uri) but it's been still for a long time and since deployment tools have moved away from using auth_uri I would guess that the Puppet OpenStack project is not the only project that has specific code to keep compatible with Mistral. Is there anybody active in Mistral that could look at it or have more information? Best regards Tobias [1] https://review.openstack.org/#/c/594187/

3 2

[dev][keystone] Key loading performance during token validation
by Lance Bragstad 13 Feb '19

13 Feb '19

Both fernet tokens and JSON Web Tokens (still under review [0]) require keys to either encrypt or sign the token payload. These keys are kept on disk by default and are treated like configuration. During the validation process, they're loaded from disk, stored as a list, and iterated over until the correct key decrypts the ciphertext/validates the signature or the iterable is exhausted [1][2]. Last week XiYuan raised some concerns about loading all public keys from disk every time we validate a token [3]. To clarify, this would be applicable to both fernet keys and asymmetric key pairs used for JWT. I spent a couple days late last week trying to recreate the performance bottleneck. There were two obvious approaches. 1. Watch for writable changes to key repositories on disk I used a third-party library for this called watchdog [4], but the inherent downside is that it is specific to file-based key repositories. For example, this might not work with a proper key manager, which has been proposed in the past [5]. 2. Encode a hash of the key used to create the token inside the token payload This would give any keystone server validating the token the ability preemptively load the correct key for validation instead of iterating over a list of all possible keys. There was some hesitation around this approach because the hash would be visible to anyone with access to the token (it would be outside of ciphertext in the case of fernet). While hashes are one-way, it would allow someone to "collect" tokens they know were issued using the same symmetric private key or signed with the same asymmetric private key. Security concerns aside, I did attempt to write up the second approach [6]. I was able to get preemptive loading to work, but I didn't notice a significant performance impact. For clarification, I was using a single keystone server with one private key for signing and 100 public keys to simulate a deployment of 100 API servers that all need to validate tokens issued by each other. At a certain point, I wondered if the loading of keys was really the issue, or if it was because we were iterating over an entire set every time we validate a token. It's also important to note that I had to turn off caching to fully test this. Keystone enables caching by default, which short-circuits the entire key loading/token provider code path after the token is issued. My question is if anyone has additional feedback on how to recreate performance issues specifically for loading files from disk, since I wasn't particularly successful in noticing a difference between repetitive key loading or iterating all keys on token validation against what we already do. [0] https://review.openstack.org/#/c/614549/ [1] https://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/token… [2] https://github.com/pyca/cryptography/blob/master/src/cryptography/fernet.py… [3] https://review.openstack.org/#/c/614549/13/keystone/token/providers/jws/cor… [4] https://pythonhosted.org/watchdog/ [5] https://review.openstack.org/#/c/363065/ [6] https://review.openstack.org/#/c/636151/

2 1

[neutron] [upgrade] No meeting on Feb. 14th
by Lujin Luo 13 Feb '19

13 Feb '19

Hi team, I will not be able to chair the meeting tomorrow either. Let's skip it and resume next week on 21th! Sorry for any inconvenience caused. Best regards, Lujin

1 0