June 2019 - openstack-discuss - lists.openstack.org

[nova] TPM passthrough
by Jim Rollenhagen 27 Jun '19

27 Jun '19

Hey y'all, We have an internal use case which requires a VM with a TPM, to be used to store a private key. Libvirt has two ways to present a TPM to a VM: passthrough or emulated. Per kashyap and the #qemu IRC channel, libvirt stores the TPM's state on disk, unencrypted. Our risk profile includes "someone walks away with a disk", so this won't work for our use case. The QEMU devs have asked for RFEs to implement vTPMs where the state never touches the disk, so I have hopes that this will be done eventually. However, I suspect that this will still take some time, especially as nobody has volunteered to actually do the work yet. So, I'd like to propose we implement TPM passthrough in Nova. My team is happy to do the work, but I'd love some guidance as to the best way to implement this so we can get a spec done (I assume it's "just another resource class"?). If Nova doesn't want this feature in, and would rather just wait for the features in QEMU, we'll carry it downstream, I guess. :) Thoughts? // jim

5 9

[dev][requirements] Upcoming changes to constraints handling in tox.ini
by Tony Breeds 27 Jun '19

27 Jun '19

Hi folks, This is a heads-up to describe 3 sets of changes you'll start seeing starting next week. 1) lower-constraints.txt handling TL;DR: Make sure projects do not specify a constraint file in install_command 2) Switch to the new canonical constraints URL on master TR;DR: Make sure you use https://releases.openstack.org/constraints/upper/master 3) Switch to the new canonical constraints URL on stable branches TR;DR: Make sure you use https://releases.openstack.org/constraints/upper/$series These will be generated from a member of the requirements team[1], and will be on the gerrit topic constraints-updates. We'll start next week to give y'all a few days to digest this email Now for slightly more details: 1) lower-constraints.txt handling If a projects has an install_command that includes a constraint file *and* the also have a lower-constraints test env what may happen is something like: given a tox.ini that contains: --- [testenv] install_command = pip install -U -c upper-constraints.txt {opts} {deps} deps = -r requirements.txt -r test-requirements.txt [testenv:lower-constraints] deps = -c lower-constrtaints.txt -r requirements.txt -r test-requirements.txt --- pyXX: virtualenv -p $python_version .tox/$testenv_name .tox/$testenv_name/bin/pip install -U -c upper-constratints.txt -r requirements.txt -rtest-requirements.txt .tox/$testenv_name/bin/pip install . lower-constraints: virtualenv -p $python_version .tox/$testenv_name .tox/$testenv_name/bin/pip install -U -c upper-constratints.txt -c lower-constrtaints.txt -r requirements.txt -rtest-requirements.txt .tox/$testenv_name/bin/pip install . pip will constrain a library to the first item it encounters, which pretty much means that the lower-constraints testenv is still testing against upper-constraints.txt[3] The fix is to move all constraints into deps rather than install_command. This impacts ~40 projects and will be fixed by hand, rather than a tool so we can preserve each projects style for tox.ini. There's a pretty good chance that this change will break the lower-constraints testenv, project teams are encouraged to help identify the incorrect values in lower-constraints, the requirements team will pick the current value in upper-constraints which almost certainly raises a value well past the minimum. https://review.opendev.org/#/c/601188/ is the change we did for keystone way back when Some projects have upper-constraints.txt in install_command but explicitly override it in lower-constratints.txt this works and wont be changed at this point but ideally all projects will use the same format to reduce the impact in switching between projects. 2) Switch to the new canonical constraints URL on master Way back at the first Denver PTG thr requirements, release and infrastructure teams discussed avoiding using gitweb urls for constraints urls. This was discussed[4] in more detail after the liberty EOL broke consumers still wanting to use tox as: https://git.openstack.org/cgit/openstack/requirements/plain/upper-constrain… 'vanished' The idea was re-raised[5] in Feb. this year where we devised a plan to create URLs that abstract the git hosting and allow us to switch between branches and tags at the right times in the OpenStack life-cycle. Other, more dynamic, ideas were discussed but over complicate things and make OpenStack slightly harder to understand or create more work. for example we could: a. Add an install_script that looks up $metadata and grab the right constraints We're trying avoid scripts like this as it very rapidly gets us to the point where we're doing "managed copy-and-paste" between projects *or* projects install scripts diverge b. Add to tox itself the ability to derive the URL from $metadata This is doable but frankly requires more time than we have right now. So what we have is: redirect 301 /constraints/upper/master http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/train http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/stein http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/rocky http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/queens http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/pike http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/ocata http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/newton http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/mitaka http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/liberty http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/kilo http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… redirect 301 /constraints/upper/juno http://git.openstack.org/cgit/openstack/requirements/plain/upper-constraint… [Yes I know we need to use opendev that's tomorrows problem] so that means that: http{s,}://releases.openstack.org/constraints/upper/master http{s,}://releases.openstack.org/constraints/upper/$series will always do the right thing based on the state of each OpenStack release. So we're going to generate changes that point to the right place. For master we have 2 options: a. Use .../train If we use train in the constraints URLs now things will work and when we branch for 'U' we'd need to update master to point to the right series name. This is very doable. The release team already generates a change on master at this point in time to update reno. We also need to update the new stable branch so that .gitreview points to the new branch. b. Use .../master If we use 'master' in the constratints URLs master will always work and when we branch for 'U' we'd need to update the stable/train branch to point to train. The release team already does this today along with the .gitreview change. So the bottom line is that using master (over train) is *slighty* less work for the release team and *slightly* less work for each project team. as such that's what we're going to do. There are several changes in the works that set the constraints url to either https://opendev.org/openstack/requirements/raw/branch/master/upper-constrai… or https://releases.openstack.org/constraints/upper/train These will work and will not break anything when the project branches, however for the sake of OpenStack wide consistency, and avoiding merge conflicts project teams are discouraged from merging them, or if the change also fixes other references to git.openstack.org just have the tox.ini updates removed. Once tox.ini is done the requirements team will look for additional places where the 'old style' urls are used and follow-up but right now there's a lot of noise to sift through 3) Switch to the new canonical constraints URL on stable branches Hopefully with all the background from 2 above this should be pretty self-explanatory. This is however much lower priority and probably wont start until the bulk of 1 and 2 is complete Yours Tony. [1] https://review.opendev.org/#/admin/groups/131,members [2] [2] Probably me actually ;P [3] There is some nuance here but this is the general rule [4] In the thread that starts with: http://lists.openstack.org/pipermail/openstack-dev/2017-September/122333.ht… [5] In the thread that starts with: http://lists.openstack.org/pipermail/openstack-discuss/2019-February/002682…

6 7

Backup solutions
by Alfredo De Luca 27 Jun '19

27 Jun '19

Hi all. Just a quick one. Other than freezer openstack project as backup solution, are there any other opensource/commercial project/software/solutions for that? Cheers -- *Alfredo*

3 3

[glance] Openstack Glance Image custom properties issue
by Hemant Sonawane 27 Jun '19

27 Jun '19

Hello, My self Hemant Sonawane. Recently I am working on openstack and I am getting some strange error when I installed openstack dashboard and tried to edit the visibility of images and update their metadata it dosent work I get *Forbidden : Redirecting to login *and *Unable to edit the image custom properties *such errors. I also tried to see the glance-api logs and found this *"2019-06-26 13:11:51,705.705 1 WARNING glance.api.v2.images [-] Could not find schema properties file schema-image.json. Continuing without custom properties"* Is there any glance policy in horizon? that might cause this error I think. The detail log file attached for your ready reference. So please look into my concern and let me know if there is any solution to resolve my issue. Thanks and Regards, Hemant Sonawane

1 0

Reply： [nova]Bug #1829696: qemu-kvm process takes 100% CPU usage whenrunning redhat/centos 7.6 guest OS
by 钱熙 26 Jun '19

26 Jun '19

Thanks very much. I will upgrade the virt tools software on the host and check if it still happens. Later, I hope I could update you guys. Thanks again. Best regards QianXi------------------ 原始邮件 ------------------ 发件人: "Kashyap Chamarthy"<kchamart(a)redhat.com> 发送时间: 2019年6月21日(星期五) 凌晨0:32 收件人: "钱熙"<qianxi416(a)foxmail.com>; 抄送: "openstack-discuss"<openstack-discuss(a)lists.openstack.org>; 主题: Re: [nova]Bug #1829696: qemu-kvm process takes 100% CPU usage whenrunning redhat/centos 7.6 guest OS On Thu, Jun 20, 2019 at 10:06:11PM +0800, 钱熙 wrote: > Hi there, Hi, > > I am struggling with qemu-kvm 100% CPU usage problem. > When running redhat or centos 7.6 guest os on vm, > the cpu usage is very low on vm(almost 100% idle when no tasks run), but on the host, > qemu-kvm reports 100% cpu busy usage. > > > https://bugs.launchpad.net/nova/+bug/1829696 > I opened the bug above, however it did not find some interest. > > > After searching some related bugs report, > I suspect that it is due to the clock settings in vm's domain xml. > My settings are as follows: > <clock offset='utc'> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='pit' tickpolicy='delay'/> > <timer name='hpet' present='no'/> > </clock> > And details about the bug, please see https://bugs.launchpad.net/nova/+bug/1829696 I don't think it is related to clock settings at all. > It shows that only the version 7.6 of redhat or centos affected by this bug behavior. > In my cluster, it is OK for versions from redhat or centos 6.8 to 7.5. > > > Any clue or suggestion? Please see DanPB's response in comment#2; I agree with it. (Note that I've changed the bug component from 'openstack-nova' --> 'qemu') https://bugs.launchpad.net/qemu/+bug/1829696/comments/2 -- /kashyap

2 1

[goal][python3] Train unit tests weekly update (goal-11)
by Corey Bryant 26 Jun '19

26 Jun '19

This is the goal-11 weekly update for the "Update Python 3 test runtimes for Train" goal [1]. There are 11 weeks remaining for completion of Train community goals [2]. == What's the Goal? == To ensure (in the Train cycle) that all official OpenStack repositories with Python 3 unit tests are exclusively using the 'openstack-python3-train-jobs' Zuul template or one of its variants (e.g. 'openstack-python3-train-jobs-neutron') to run unit tests, and that tests are passing. This will ensure that all official projects are running py36 and py37 unit tests in Train. For complete details please see [1]. == Ongoing Work == Open patches needing reviews: https://review.openstack.org/#/q/topic:python3-train+is:open Failing patches: https://review.openstack.org/#/q/topic:python3-train+status:open+(+label:Ve… Patch automation scripts needing review: https://review.opendev.org/#/c/666934 Some notes on 2 issues I came across this week: 1) I've updated the patch automation scripts to drop py34,py35,py36 and add py37 to default tox targets as discussed with smcginnis, fungi, and gmann in #openstack-tc this week. 2) While trying to proposed and urge the move to py37-only default tox targets, some projects have decided to keep both py36 and py37 targets and some projects have decided to run with the generic py3-only target. == Completed Work == Merged patches: https://review.openstack.org/#/q/topic:python3-train+is:merged == How can you help? == Please take a look at the failing patches and help fix any failing unit tests for your project(s). Python 3.7 unit tests will be self-testing in Zuul. If you're interested in helping submit patches, please let me know. == Reference Material == [1] Goal description: https://governance.openstack.org/tc/goals /train/python3-updates.html [2] Train release schedule: https://releases.openstack.org/train/schedule.html (see R-5 for "Train Community Goals Completed") Storyboard: https://storyboard.openstack.org/#!/story/2005924 Porting to Python 3.7: https://docs.python.org/3/whatsnew/3.7.html#porting-to-python-3-7 Python Update Process: https://opendev.org/openstack/governance/src/branch/master/resolutions/2018… Train runtimes: https://opendev.org/openstack/governance/src/branch/master/reference/runtim… Thanks, Corey

1 0

[stable][sahara] Changes to the stable team
by Tony Breeds 26 Jun '19

26 Jun '19

Hello all, In the middle of last week Telles asked[1] for a couple of the current Sahara core team (Jeremy and Luigi) to be added to the stable group. Upon review there isn't enough data (either reviews or backports) to verify that Jeremy and Luigi understand the stable policy. So I'd like to try an experiment and I've added both to the stable core team but I ask neither +W a backport and ping me (or another stable team member) for the final approval. I'd like to suggest dropping: * Elise Gafford egafford(a)redhat.com * Sergey Lukjanov slukjanov(a)mirantis.com * Sergey Reshetnyak sreshetniak(a)mirantis.com * Vitaly Gridnev gridnevvvit(a)gmail.com from the sahara-stable-maint group given[2] is empty. Yours Tony. [1] http://eavesdrop.openstack.org/irclogs/%23openstack-stable/%23openstack-sta… [2] https://review.opendev.org/#/q/(project:openstack/python-saharaclient+OR+pr…

2 2

[nova] Spec Review Day: Tuesday July 2nd
by Eric Fried 26 Jun '19

26 Jun '19

Esteemed Nova contributors and maintainers- With about a month until spec freeze [1], we need to fish or cut bait [2] on open specs. To that end, I have recycled the Spec Review Day etherpad [3] and populated it with all open Train specs (those with a file under specs/train/approved) [4]. I seeded their initial status in the etherpad, including noting from whom the next action is expected. Authors: Look for your specs, especially if they say "Action: author", and update them ASAP. (Having done so, you can toggle to "Action: reviewer".) Reviewers and Cores: Scan the list for Action: "reviewer" and/or "approver" and dive in. (You may wish to start by intersecting with [5], which narrows [4] down to specs you've already reviewed at some point.) You do not need to wait until, or limit your activity to, Tuesday. Especially if you're going to be unavailable then. Thanks, efried [1] https://wiki.openstack.org/wiki/Nova/Train_Release_Schedule [2] https://en.wiktionary.org/wiki/fish_or_cut_bait [3] https://etherpad.openstack.org/p/nova-spec-review-day [4] https://review.opendev.org/#/q/project:openstack/nova-specs+status:open+pat… [5] https://review.opendev.org/#/q/project:openstack/nova-specs+status:open+pat…

1 1

[stackalytics] Reported numbers seem inaccurate
by Mark Goddard 26 Jun '19

26 Jun '19

Hi, I've heard a couple of people say they don't feel the numbers reported by Stackalytics accurately reflect reality. I've been trying to gather a few stats for my Kolla update session in Denver, and am finding the same. I'll try to give some concrete examples. Reviews for all kolla deliverables in Stein [1]. Here the company stats don't reflect the individual stats. Also, the total reviews in the 'Kolla Official' module does not equal the sum of the reviews of its submodules (kolla, kolla-ansible, kolla-cli). If I look at the contribution summary for Kolla Official in the last 90 days [2], they are actually greater than those for the last 180 days [3]! There are also similar issues with commit metrics, and none seem to match what I see in git. Thanks, Mark [1] https://www.stackalytics.com/?metric=marks&module=kolla-group [2] https://www.stackalytics.com/report/contribution/kolla-group/90 [3] https://www.stackalytics.com/report/contribution/kolla-group/180

10 11

[nova] FYI for out-of-tree virt drivers: deprecating non-update-provider-tree compat
by Matt Riedemann 26 Jun '19

26 Jun '19

I have a change up [1] to deprecate the backward compatibility code in the ResourceTracker for virt drivers that do not implement the update_provider_tree interface. Starting in Train you'll get a warning in the nova-compute logs every time the update_available_resource periodic runs (we could only log on startup, that's up for debate in the change) until the driver is updated and in the U release we'll drop that compatibility code. The only out-of-tree virt driver I checked was nova-lxd and it does *not* implement update_provider_tree so for those that care about that driver you should implement the interface. The patch has a link to the reference docs on implementing that interface for your driver (it's pretty simple). [1] https://review.opendev.org/#/c/667442/ -- Thanks, Matt

1 0