October 2020 - openstack-discuss - lists.openstack.org

Nova missing instance
by Grant Morley 29 Oct '20

29 Oct '20

Hi all, A customer had contacted us stating their instance was not working and was unresponsive. They attempted to reboot it and it failed to reboot and went into an error status. I have had a look at the nova logs and nova seems to think the instance has "disappeared" and comes up with an error stating: "Instance cannot be found" I have attached the full logs in the pastebin link. Has anyone come across this before? https://pastebin.com/LWzvsc8u Regards, -- Grant Morley Cloud Engineer, Civo Ltd Unit H-K, Gateway 1000, Whittle Way Stevenage, Herts, SG1 2FP, UK Visit us at www.civo.com <https://www.civo.com/> Signup for an account now > <https://www.civo.com/signup>

2 1

[tripleo][ci] Docker.io Service Announcement: Pull Rate Limit Enforcement, Changes to Image Retention Policies Begin
by Wesley Hayutin 29 Oct '20

29 Oct '20

Greetings, FYI... docker.io will be unavailable to tripleo jobs on Monday Nov 2 2020. Documentation is available here: https://hackmd.io/YH9xtBmOQbSVqPu7fjgoog Reviews are welcome and appreciated https://review.opendev.org/#/q/topic:new-ci-job+(status:open+OR+status:merg… Thanks! ------ from docker.io ------ Hello: You are receiving this email because of a policy change to Docker products and services you use. On Monday, November 2, 2020 at 9am Pacific Standard Time, Docker will begin enforcing rate limits on container pulls for Anonymous and Free users. Anonymous (unauthenticated) users will be limited to 100 container image pulls every six hours, and Free (authenticated) users will be limited to 200 container image pulls every six hours, when enforcement is fully implemented. Docker Pro and Team subscribers can pull container images from Docker Hub without restriction, as long as the quantities are not excessive or abusive. In addition, we are pausing enforcement of the changes to our image-retention policies until mid-2021, when we anticipate incorporating them into usage-based pricing. Two months ago, we announced an update to Docker image-retention policies. As originally stated, this change, which was set to take effect on November 1, 2020, would result in the deletion of images for free Docker account users after six months of inactivity. Today’s announcement means Docker will not enforce image expiration on November 1, 2020. For more details, please read the blog post <http://email.notify.docker.com/c/eJxNjsFuhSAURL9Gdxi46EMWLNq89j-ucFVSBCMY49…> summarizing the changes or review the documentation for rate limits <http://email.notify.docker.com/c/eJxNjkEOgyAURE8jOwl8kOqCRRvbe3z8WEhRG8WY3r…> . Details about Docker subscription levels and differentiators are available on the Docker Pricing Page <http://email.notify.docker.com/c/eJxNjcsOgyAURL9GlgTu5VEWLNrY_gfBq5KqGKUh_n…> . Thank you for using Docker. Jean-Laurent de Morlhon VP, Engineering Docker

1 0

Windows client can't connect to internet
by liu 29 Oct '20

29 Oct '20

Dear OpenStack community, I just launch a windows client in my OpenStack platform, but it can't connect to the internet with the float IP. Before that use a ubuntu client I can access the internet. Any ideas about why this happened. Thank you for your help. Liu

2 2

[tripleo][ci] Fwd: Service Announcement: Pull Rate Limit Enforcement, Changes to Image Retention Policies Begin November 2, 2020
by Wes Hayutin 29 Oct '20

29 Oct '20

FYI... Reviews are welcome and appreciated https://review.opendev.org/#/q/topic:new-ci-job+(status:open+OR+status:merg… Thanks! ---------- Forwarded message --------- From: Docker <noreply(a)notify.docker.com> Date: Wed, Oct 28, 2020 at 10:08 PM Subject: Service Announcement: Pull Rate Limit Enforcement, Changes to Image Retention Policies Begin November 2, 2020 To: <weshayutin(a)gmail.com> [image: Docker] <http://email.notify.docker.com/c/eJxNjUEOgyAQAF-DR7LsosCBQ43tPxChkKo0imn8fe…> Hello: You are receiving this email because of a policy change to Docker products and services you use. On Monday, November 2, 2020 at 9am Pacific Standard Time, Docker will begin enforcing rate limits on container pulls for Anonymous and Free users. Anonymous (unauthenticated) users will be limited to 100 container image pulls every six hours, and Free (authenticated) users will be limited to 200 container image pulls every six hours, when enforcement is fully implemented. Docker Pro and Team subscribers can pull container images from Docker Hub without restriction, as long as the quantities are not excessive or abusive. In addition, we are pausing enforcement of the changes to our image-retention policies until mid-2021, when we anticipate incorporating them into usage-based pricing. Two months ago, we announced an update to Docker image-retention policies. As originally stated, this change, which was set to take effect on November 1, 2020, would result in the deletion of images for free Docker account users after six months of inactivity. Today’s announcement means Docker will not enforce image expiration on November 1, 2020. For more details, please read the blog post <http://email.notify.docker.com/c/eJxNjsFuhSAURL9Gdxi46EMWLNq89j-ucFVSBCMY49…> summarizing the changes or review the documentation for rate limits <http://email.notify.docker.com/c/eJxNjkEOgyAURE8jOwl8kOqCRRvbe3z8WEhRG8WY3r…>. Details about Docker subscription levels and differentiators are available on the Docker Pricing Page <http://email.notify.docker.com/c/eJxNjcsOgyAURL9GlgTu5VEWLNrY_gfBq5KqGKUh_n…>. Thank you for using Docker. Jean-Laurent de Morlhon VP, Engineering Docker © Docker, Inc., 2020 318 Cambridge Ave Palo Alto, CA 94306 Docker Privacy Policy <http://email.notify.docker.com/c/eJxNjssKgzAURL_G7AzxxiS6yKLF9j_y1ND4QG1D_r…>

1 0

Nova iSCSI oddness
by Grant Morley 29 Oct '20

29 Oct '20

Hi all, We are seeing some oddness on a couple of our compute hosts that seems to be related to iSCSI. On a couple of our hosts I am seeing this error in the nova compute logs: 2020-10-27 18:56:14.814 31490 WARNING os_brick.initiator.connectors.iscsi [req-8613ae69-1661-49cf-8bdc-6fec875d01ba - - - - -] Couldn't find iscsi sessions because iscsiadm err: iscsiadm: could not read session targetname: 5 iscsiadm: could not find session info for session1707 That seems to also stop any instance on the compute host from being able to reboot. Reboots seem to get accepted but the instance never completes and gets stuck in the reboot state: 2020-10-27 19:11:58.891 48612 INFO nova.compute.manager [-] [instance: c8079e85-4777-4615-9d5a-3d1151e11984] During sync_power_state the instance has a pending task (reboot_started_hard). Skip. 2020-10-27 19:11:58.891 48612 INFO nova.compute.manager [-] [instance: 31128f26-910d-411f-98e0-c95dd36f4f0f] During sync_power_state the instance has a pending task (reboot_started_hard). Skip. Does anyone know of a way to resolve this without rebooting the entire compute host? I can't see any other issues other than the fact there is this iSCSI error which in turn seems to stop nova from processing anything for any instance. Any advice would be much appreciated. Regards, -- Grant Morley Cloud Engineer, Civo Ltd Unit H-K, Gateway 1000, Whittle Way Stevenage, Herts, SG1 2FP, UK Visit us at www.civo.com <https://www.civo.com/> Signup for an account now > <https://www.civo.com/signup>

4 9

[tripleo] TripleO at Wallaby PTG wrapup
by Marios Andreou 29 Oct '20

29 Oct '20

Hello all, thanks to everyone that participated and especially thanks to all who prepared and lead our PTG sessions. The number of participants was variable however the maximum attendees (that I noticed at least) per day were Monday:43, Tuesday:48, Wednesday:39. We had three full days with 18 sessions in total. I've included a condensed listing below [3] for convenience and you can find the full schedule at [1] with session descriptions. All sessions were recorded - hopefully we will get access to those soon (I don't have any info about that yet). If you have problems accessing any of the etherpads reach out to me I made backups. Our "team photo" from Monday is available at [2] :) Thanks again TripleO for a great PTG! regards, marios [1] https://etherpad.opendev.org/p/tripleo-ptg-wallaby [2] https://i.imgur.com/apqiYhk.png [3] DAY 1: retrospective & community topics Etherpad: https://etherpad.opendev.org/p/tripleo-ptg-retrospective Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-community-process (Deployment) Composable Network v2 (sans Heat) Proposer: Emilien, Harald & Rabi Etherpad link: https://etherpad.opendev.org/p/composable-networks-v2 (Edge) Topic: State of DCN Proposer: James Slagle Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-edge (PIDONE) Updates from PIDONE Proposer: dciabrin/bandini Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-pidone (Ceph) Topic: review the cephadm integration specs https://review.opendev.org/#/c/757644/ and https://review.opendev.org/#/c/723108/ and https://review.opendev.org/759395 Etherpad link: https://etherpad.opendev.org/p/tripleo-ceph Presenter: [fmount, gfidente, fultonj] DAY 2: (Upgrades) Lessons learned upgrading from Queens to Train, then looking forward at Train to Wallaby Etherpad link: https://etherpad.opendev.org/p/tripleo-PTG-wallaby-upgrades Presenter: Jesse (odyssey4me), Lukas (lbezdick), Jose (jfrancoa), Sergii (holser) (CI) Topic: Dependency pipeline, CentOS Stream and CentOS 9 / Fedora 33 Slides: https://filedn.com/lHMSkrniM7gFSkf96rR711z/TripleODependencyPipeline.pdf Presenter: (rlandy) (Deployment) Topic: Drop heat-monolith local package requirement for undercloud Proposer: Alex Schultz (mwhahaha) Slides: https://docs.google.com/presentation/d/13ds5h9foCHEy3duElGs6mZ3mQwKcaggX6Vj… Blueprint: https://blueprints.launchpad.net/tripleo/+spec/switch-to-heat-all-container (Security) Topic: Updates for Security Etherpad Link: https://etherpad.opendev.org/p/tripleo-PTG-wallaby-security Presenter: Ade (alee) (Upgrades) Topic: Update should only be role inclusion, module and action plugins but we can't do the refactor without any upstream development framework or.... some broader coverage CI. Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-refactor-update (CI) Topic: 0/ so long docker.io Etherpad link: https://hackmd.io/YH9xtBmOQbSVqPu7fjgoog?both Presenter: sshnaidm, chkumar, weshay, marios, DAY 3: (Deployment) Topic: Move to Ansible 2.10.x Proposer: Kevin Carter Etherpad link: https://etherpad.opendev.org/p/tripleo-ansible-210 (Upgrades) Topics: Run all roles in parallel during update by default for massive speed improvment. Etherpad link: https://etherpad.opendev.org/p/tripleo-ptg-wallaby-all-roles-in-parallel (Deployment) Topic: config-download Ansible Collection Etherpad link: https://etherpad.opendev.org/p/ptg-tripleo-config-download-collection Proposer: Luke Short (ekultails) (Deployment) Topic: Removal of Swift from the Undercloud Proposer: Kevin Carter (cloudnull) Etherpad link: https://etherpad.opendev.org/p/tripleo-swift-removal-undercloud (Ceph) Integration Discussion - Continued Etherpad link: https://etherpad.opendev.org/p/tripleo-ceph Presenter: [fmount, gfidente, fultonj] (Networking) Topic: BGP Integration https://review.opendev.org/#/c/758249/ "Spec for installing and configuring FRRouter" Proposer: Dan Sneddon (Cloudops) Topic: Moving deployment logic into ansible roles Proposer: efoley, all (discussion)

1 0

[operators][telemetry] Quick Telemetry survey
by Matthias Runge 29 Oct '20

29 Oct '20

Hi there, you may have seen, not much has been happening lately in OpenStack Telemetry. However, there have been interesting talks at the recent OpenInfra Summit. I would like to gather some data on current Telemetry usage in order to be able to focus a bit better[1]. There is not much value in keeping projects alive, if nobody uses them. At the same time, we may want to increase focus/contribution on often used projects. In order to help with the data, please fill in the form[1] and also feel free to forward this form to interested parties. Thank you, Matthias [1] https://forms.gle/EmncAxggRcG8r31b9

3 3

[devstack][cinder failure]:internal server error(HTTP 500)
by James 29 Oct '20

29 Oct '20

Hi:    I am new to openstack installing on one VM. The environment includes Win10,virtualbox,Ubuntu 18.04; The version of  devstack and openstack  is  Train;  the VM  includes 2vCPU,4G ram, 60G disk .    The process was interrupted by cinder error.[internal server error]        The stack.log shows as stack.log; the c-api log shows as c-api log. stack.log +[38;5;242mlib/cinder:create_volume_types:531 (B[m openstack --os-region-name=RegionOne volume type create --property volume_backend_name=lvmdriver-1 lvmdriver-1 2020-10-29 07:57:11.863 | Internal Server Error (HTTP 500) 2020-10-29 07:57:12.182 | +[38;5;242mlib/cinder:create_volume_types:1 (B[m exit_trap 2020-10-29 07:57:12.232 | +[38;5;242m./stack.sh:exit_trap:536 (B[m local r=1 2020-10-29 07:57:12.259 | ++[38;5;242m./stack.sh:exit_trap:537 (B[m jobs -p 2020-10-29 07:57:12.316 | +[38;5;242m./stack.sh:exit_trap:537 (B[m jobs= 2020-10-29 07:57:12.353 | +[38;5;242m./stack.sh:exit_trap:540 (B[m [[ -n '' ]] 2020-10-29 07:57:12.385 | +[38;5;242m./stack.sh:exit_trap:546 (B[m '[' -f /tmp/tmp.8Y7a6hpwLX ']' 2020-10-29 07:57:12.408 | +[38;5;242m./stack.sh:exit_trap:547 (B[m rm /tmp/tmp.8Y7a6hpwLX 2020-10-29 07:57:12.468 | +[38;5;242m./stack.sh:exit_trap:551 (B[m kill_spinner 2020-10-29 07:57:12.499 | +[38;5;242m./stack.sh:kill_spinner:446 (B[m '[' '!' -z '' ']' 2020-10-29 07:57:12.536 | +[38;5;242m./stack.sh:exit_trap:553 (B[m [[ 1 -ne 0 ]] 2020-10-29 07:57:12.581 | +[38;5;242m./stack.sh:exit_trap:554 (B[m echo 'Error on exit' 2020-10-29 07:57:12.581 | Error on exit 2020-10-29 07:57:12.615 | +[38;5;242m./stack.sh:exit_trap:556 (B[m type -p generate-subunit 2020-10-29 07:57:12.635 | +[38;5;242m./stack.sh:exit_trap:557 (B[m generate-subunit 1603957090 1142 fail 2020-10-29 07:57:14.084 | +[38;5;242m./stack.sh:exit_trap:559 (B[m [[ -z /opt/stack/logs ]] 2020-10-29 07:57:14.110 | +[38;5;242m./stack.sh:exit_trap:562 (B[m /opt/stack/devstack/tools/worlddump.py -d /opt/stack/logs 2020-10-29 07:57:17.223 | nova-compute: no process found 2020-10-29 07:57:17.264 | neutron-dhcp-agent: no process found 2020-10-29 07:57:17.338 | neutron-l3-agent: no process found 2020-10-29 07:57:17.409 | neutron-linuxbridge-agent: no process found 2020-10-29 07:57:17.471 | neutron-metadata-agent: no process found 2020-10-29 07:57:17.603 | +[38;5;242m./stack.sh:exit_trap:571 C-api logSTRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION {{(pid=12249) _check_effective_sql_mode /usr/local/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py:307}} 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault [req-7d67a008-3478-4da0-8b29-259571c1fb94 req-d823c701-e76a-489b-8ec9-fad9f9504c7b admin admin] Caught error: <class 'oslo_db.exception.DBConnectionError'> (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: (Background on this error at: http://sqlalche.me/e/e3q8) DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: (Background on this error at: http://sqlalche.me/e/e3q8) 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault Traceback (most recent call last): 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault File "/opt/stack/cinder/cinder/api/middleware/fault.py", line 85, in __call__ 10月 29 10:56:07 controller devstack(a)c-api.service[12246]: ERROR cinder.api.middleware.fault return req.get_response(self.application)Thanks for your help,firtly.   发自我的iPhone

1 0

[infra][ci][all] Devstack error
by Slawek Kaplonski 29 Oct '20

29 Oct '20

Hi, Recently I see pretty often devstack based jobs failing with error "ERROR] / opt/stack/devstack/functions-common:230 Failure retrieving default route device". I reported it at [1] as Devstack bug but maybe it's something related to some changes in Ubuntu used on the gate. Did You saw something similar also? Maybe You already know why it's like that and have any ideas about how to fix it? Thx in advance for any info. [1] https://bugs.launchpad.net/devstack/+bug/1902002 -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[heat] propose to move meeting time to Wednesday at 1400 UTC in #heat
by Rico Lin 29 Oct '20

29 Oct '20

Hi all as our current members are changed, I would like to propose we move our meeting time [1] one hour later. >From Weekly on Wednesday at *1300* UTC in #heat to Weekly on Wednesday at *1400* UTC in #heat Which I believe will be easier for US forks to join. [1] http://eavesdrop.openstack.org/#Heat_(Orchestration)_Team_Meeting -- May The Force of OpenStack Be With You, *Rico Lin*irc: ricolin

2 1