- openstack-discuss - lists.openstack.org

[Neutron] Metering agent issues
by Eric K. Miller 17 Oct '19

17 Oct '19

Hi, We have a Stein deployment using Kolla Ansible 8.0.1 (the latest) and have enabled the deployment of the neutron metering agent. We have separate network nodes and compute nodes and use DVR. DVR is working properly, where ingress and egress traffic to floating IPs binded to fixed IPs flows directly to our upstream routers, not through the network nodes, whereas SNAT traffic flows through the network nodes. I configured the agent using the simplest of rules to test the agent (as cloud admin): openstack network meter create --description Shared --share bandwidth openstack network meter rule create --egress --remote-ip-prefix 0.0.0.0/0 bandwidth openstack network meter rule create --ingress --remote-ip-prefix 0.0.0.0/0 bandwidth None of the compute nodes appeared to be metering. So I checked the metering agent log (see below my signature for a snippet of the respective log entries), which shows that no routers are being found. Note that the Debug flag is set to True (in metering_agent.ini). On the primary network node, the "get_traffic_counters" method returns a list of all routers for every project, and runs a respective meter retrieval for each, such as the following, for each router: 2019-10-17 15:29:30.437 6 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-e0d2ee4f-76ec-4043-93f7-a47fae488d59', 'iptables', '-t', 'filter', '-L', 'neutron-meter-l-fb2c912d-9ce', '-n', '-v', '-x', '-w', '10', '-Z'] execute_rootwrap_daemon /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/agent/linux/util s.py:103 However, a single metering report is being created. Only one debug row appears in the log, and only one "bandwidth metric" measure is added to Gnocchi: 2019-10-17 15:38:15.494 6 DEBUG neutron.services.metering.agents.metering_agent [-] Send metering report: {'label_id': u'fb2c912d-9ced-433b-914e-df6526e034d1', 'time': 30, 'pkts': 0, 'tenant_id': u'96b2e5246a7744219e2af51f0e7b91d6', 'first_update': 1571300562, 'bytes': 0, 'host': 'networknode002', 'last_update': 1571344695} _metering_notification /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/services/meterin g/agents/metering_agent.py:103 No errors are being reported in the logs. Am I missing something? Thanks! Eric 2019-10-17 15:25:12.418 6 DEBUG oslo_concurrency.lockutils [-] Lock "metering-agent" acquired by "neutron.services.metering.agents.metering_agent._invoke_driver" :: waited 0.000s inner /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockuti ls.py:327 2019-10-17 15:25:12.418 6 DEBUG neutron.services.metering.drivers.iptables.iptables_driver [-] neutron.services.metering.drivers.iptables.iptables_driver.IptablesMeter ingDriver method get_traffic_counters called with arguments (<neutron_lib.context.ContextBase object at 0x7f661dfa9c90>, []) {} wrapper /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_log/helpers.py:66 2019-10-17 15:25:12.418 6 DEBUG oslo_concurrency.lockutils [-] Lock "metering-agent" released by "neutron.services.metering.agents.metering_agent._invoke_driver" :: held 0.000s inner /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockuti ls.py:339 2019-10-17 15:25:27.421 6 DEBUG neutron.services.metering.agents.metering_agent [-] Get router traffic counters _get_traffic_counters /var/lib/kolla/venv/lib/python2.7/site-packages/neutron/services/meterin g/agents/metering_agent.py:215 2019-10-17 15:25:27.421 6 DEBUG oslo_concurrency.lockutils [-] Lock "metering-agent" acquired by "neutron.services.metering.agents.metering_agent._invoke_driver" :: waited 0.000s inner /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockuti ls.py:327 2019-10-17 15:25:27.421 6 DEBUG neutron.services.metering.drivers.iptables.iptables_driver [-] neutron.services.metering.drivers.iptables.iptables_driver.IptablesMeter ingDriver method get_traffic_counters called with arguments (<neutron_lib.context.ContextBase object at 0x7f661dfa9c90>, []) {} wrapper /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_log/helpers.py:66 2019-10-17 15:25:27.421 6 DEBUG oslo_concurrency.lockutils [-] Lock "metering-agent" released by "neutron.services.metering.agents.metering_agent._invoke_driver" :: held 0.000s inner /var/lib/kolla/venv/lib/python2.7/site-packages/oslo_concurrency/lockuti ls.py:339

1 0

[tc] Feedback on Airship pilot project
by Jeremy Stanley 17 Oct '19

17 Oct '19

Hi TC members, The Airship project will start its confirmation process with the OSF Board of Directors at the Board meeting[1] Tuesday next week. A draft of the slide deck[2] they plan to present is available for reference. Per the confirmation guidelines[3], the OSF Board of directors will take into account the feedback from representative bodies of existing confirmed Open Infrastructure Projects (OpenStack, Zuul and Kata) when evaluating Airship for confirmation. Particularly worth calling out, guideline #4 "Open collaboration" asserts the following: Project behaves as a good neighbor to other confirmed and pilot projects. If you (our community at large, not just TC members) have any observations/interactions with the Airship project which could serve as useful examples for how these projects do or do not meet this and other guidelines, please provide them on the etherpad[4] ASAP. If possible, include a citation with links to substantiate your feedback. If a TC representative can assemble this feedback and send it to the Board (for example, to the foundation mailing list) for consideration before the meeting next week, that would be appreciated. Apologies for the short notice. [1] http://lists.openstack.org/pipermail/foundation/2019-October/002800.html [2] https://www.airshipit.org/collateral/AirshipConfirmation-Review-for-the-OSF… [3] https://wiki.openstack.org/wiki/Governance/Foundation/OSFProjectConfirmatio… [4] https://etherpad.openstack.org/p/openstack-tc-airship-confirmation-feedback -- Jeremy Stanley

3 4

[monasca] Ussuri planning meeting summary
by Witek Bedyk 17 Oct '19

17 Oct '19

Hello everyone, yesterday we held a planning meeting for the next release cycle (Ussuri). You can find the summary of the meeting in the etherpad [1]. Please discuss with your teams and vote for the proposed topics in this spreadsheet [2] until next Wednesday. We meet next week again to agree on priorities during the regular Team Meeting hour, Wednesday, 15 UTC [3]. Thanks Witek [1] https://etherpad.openstack.org/p/monasca-planning-ussuri [2] https://docs.google.com/spreadsheets/d/17PLO8PJr28jXuPaFJ7efZCwgdgMpinZyDRT… [3] https://global.gotomeeting.com/join/817873693

1 0

[nova][gate] Hold off on rechecks until https://review.opendev.org/689152/ merges
by Matt Riedemann 17 Oct '19

17 Oct '19

The gate is blocked for nova changes until [1] is merged so hold off on rechecking anything until then. [1] https://review.opendev.org/#/c/689152/ -- Thanks, Matt

1 0

[openstacksdk][RelMgmt] release-post job for openstack/releases for ref refs/heads/master failed
by Sean McGinnis 17 Oct '19

17 Oct '19

----- Forwarded message from zuul(a)openstack.org ----- Date: Thu, 17 Oct 2019 10:19:24 +0000 From: zuul(a)openstack.org To: release-job-failures(a)lists.openstack.org Subject: [Release-job-failures] release-post job for openstack/releases for ref refs/heads/master failed Reply-To: openstack-discuss(a)lists.openstack.org Build failed. - tag-releases https://zuul.opendev.org/t/openstack/build/571c0f0e43cb469d8727121b77a3c92b : FAILURE in 3m 14s - publish-tox-docs-static https://zuul.opendev.org/t/openstack/build/None : SKIPPED _______________________________________________ Release-job-failures mailing list Release-job-failures(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures ----- End forwarded message ----- The tagging of the new stable/queens OpenStackSDK release failed. It appears to be due to the .gitreview file in that branch still referring to the python-openstacksdk name. That caused the automation to fail on trying to interact with that repo, but it has since been renamed to openstack/openstacksdk. Since this wasn't able to process the release, even though the patch had merged, I think the fix is actually fairly easy. I've proposed a backport of the repo name changes with: https://review.opendev.org/#/c/689131/ If we were to do another release to pick up that change, I believe it would actually fail validation because it would not find the current 0.11.4 tag and therefore think that two releases were needed instead of just one. So since it was not able to process anything to actually perform the release, I believe the easy path forward is to just revert the release patch of this last release: https://review.opendev.org/#/c/689132/ Then wait for the .gitreview patch to merge, then propose the 0.11.4 release again with the hash updated to include that rename fix. Please let me know if there are any questions or if I've overlooked anything. Thanks, Sean

1 0

CentOS 8 nodes available now
by Ian Wienand 17 Oct '19

17 Oct '19

Hello, I'm happy to say that CentOS 8 images are now live in OpenDev infra. Using a node label of "centos-8" will get you started. --- The python environment setup on these images is different to our other images. Firstly, some background: currently during image build we go to some effort to: a) install the latest pip/virtualenv/setuptools b) ensure standard behaviour: /usr/bin/python -> python2 /usr/bin/pip -> python2 install /usr/bin/pip3 -> python3 install /usr/bin/virtualenv -> creates python2 environment by default; python3 virtualenv package in sync This means a number of things; hosts always have python2, and because we overwrite the system pip/virtualenv/setuptools we put these packages on "hold" (depending on the package manager) so jobs don't re-install them and create (even more of) a mess. This made sense in the past, when we had versions of pip/setuptools in distributions that couldn't understand syntax in requirements files (and other bugs) and didn't have the current fantastic job inheritance and modularity that Zuul (v3) provides. However, it also introduces a range of problems for various users, and has a high maintenance overhead. Thus these new images are, by default, python3 only, and have upstream pip and virtualenv packages installed. You will have a default situation: /usr/bin/python -> not provided /usr/bin/pip -> not provided, use /usr/bin/pip3 or "python3 -m pip" /usr/bin/virtualenv -> create python3 environment; provided by upstream python3-virtualenv package /usr/bin/pyvenv -> not provided (is provided by Ubuntu python3-venv), use /usr/bin/pyvenv-3 or "python3 -m venv". Ergo, the "standard behaviour" is not so standard any more. I would suggest if you wish to write somewhat portable jobs/roles etc., you do the following: * in general don't rely on "unversioned" calls of tools at all (python/pip/virtualenv) -- they can all mean different things on different platforms. * scripts should always be #!/usr/bin/python3 * use "python3 -m venv" for virtual environments (if you really need "virtualenv" because of one of the features it provides, use "-m virtualenv") * use "python3 -m pip" to install global pip packages; but try not too -- mixing packages and pip installs never works that well. * if you need python2 for some reason, use a bindep file+role to install it (don't assume it is there) --- For any Zuul admins, note that to use python3-only images similar to what we make, you'll need to set "python-path" to python3 in nodepool so that Ansible calls the correct remote binary. Keep an eye on [1] which will automate this for Ansible >=2.8 after things are merged and released. --- Most of the job setup has been tested (network configs, setting mirrors, adding swap etc.) but there's always a chance of issues with a new platform. Please bring up any issues in #openstack-infra and we'll be sure to get them fixed. --- If you're interested in the images, they are exported at https://nb01.openstack.org/images/ although they are rather large, because we pre-cache a lot. If you'd like to build your own, [2] might help with: DISTRO=centos-minimal DIB_RELEASE=8 --- Thanks, -i [1] https://review.opendev.org/#/c/682797/ [2] https://opendev.org/openstack/project-config/src/branch/master/tools/build-…

5 5

[placement][ptg] Ussuri Placement Topics
by Tetsuro Nakamura 17 Oct '19

17 Oct '19

Hi Placementers, We won't have a specific meeting space for Shanghai PTG, but I'd like to have retrospective on Stein, and gather and note work items we have. Please put your ideas on the etherpad [1]. [1] https://etherpad.openstack.org/p/placement-shanghai-ptg -- Tetsuro Nakamura <tetsuro.nakamura.bc(a)hco.ntt.co.jp> NTT Network Service Systems Laboratories TEL:0422 59 6914(National)/+81 422 59 6914(International) 3-9-11, Midori-Cho Musashino-Shi, Tokyo 180-8585 Japan

1 0

[Octavia] Amphora build issues
by Eric K. Miller 16 Oct '19

16 Oct '19

Hi, It seems that every build I have attempted of an amphora fails in some way. I have tried CentOS 7, Ubuntu Bionic, Xenial, and Trusty. Note that we are running Stein. I will concentrate on Ubuntu issues for now. I first create a fresh VM that is used to install the diskimage-create tool, then run (after sudo'ing to root): apt update apt -y upgrade apt-get -y install qemu qemu-system-common uuid-runtime curl kpartx git jq python-pip debootstrap libguestfs-tools pip install 'networkx==2.2' pip install argparse Babel dib-utils PyYAML git clone -b stable/stein https://github.com/openstack/octavia.git git clone https://git.openstack.org/openstack/diskimage-builder.git cd diskimage-builder pip install -r requirements.txt cd ../octavia/diskimage-create/ pip install -r requirements.txt # And finally, I run the diskimage-create script, specifying the image's OS, so ONE of these, depending on the OS: ./diskimage-create.sh -d bionic # or to use Xenial: ./diskimage-create.sh -d xenial # Note that when selecting Trusty, diskimage-create.sh error's, and so never finishes successfully. # Somewhat expected since it is quite old and unsupported. ./diskimage-create.sh -d trusty The amphorae launch when creating a load balancer, but the amphora agent fails to start, and thus is not responsive on TCP Port 9443. The log from inside the amphora is below. Has anyone successfully created an image? Am I missing something? Thanks! Eric Amphora agent fails to start inside amphora - this is logged when running the agent from the command line: 2019-10-16 03:41:04.835 1119 INFO octavia.common.config [-] /usr/local/bin/amphora-agent version 5.1.0.dev20 2019-10-16 03:41:04.835 1119 DEBUG octavia.common.config [-] command line: /usr/local/bin/amphora-agent --config-file /etc/octavia/amphora-agent.conf setup_logging /opt/amphora-agent-venv/lib/python3.5/site-packages/octavia/common/confi g.py:779 2019-10-16 03:41:05.036 1124 INFO octavia.amphorae.backends.health_daemon.health_daemon [-] Health Manager Sender starting. 2019-10-16 03:41:05.084 1119 CRITICAL octavia [-] Unhandled error: FileNotFoundError: [Errno 2] No such file or directory 2019-10-16 03:41:05.084 1119 ERROR octavia Traceback (most recent call last): 2019-10-16 03:41:05.084 1119 ERROR octavia File "/usr/local/bin/amphora-agent", line 8, in <module> 2019-10-16 03:41:05.084 1119 ERROR octavia sys.exit(main()) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/octavia/cmd/agent.p y", line 89, in main 2019-10-16 03:41:05.084 1119 ERROR octavia AmphoraAgent(server_instance.app, options).run() 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/app/base.p y", line 72, in run 2019-10-16 03:41:05.084 1119 ERROR octavia Arbiter(self).run() 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/arbiter.py ", line 60, in __init__ 2019-10-16 03:41:05.084 1119 ERROR octavia self.setup(app) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/arbiter.py ", line 95, in setup 2019-10-16 03:41:05.084 1119 ERROR octavia self.log = self.cfg.logger_class(app.cfg) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/glogging.p y", line 200, in __init__ 2019-10-16 03:41:05.084 1119 ERROR octavia self.setup(cfg) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/glogging.p y", line 227, in setup 2019-10-16 03:41:05.084 1119 ERROR octavia self.error_log, cfg, self.syslog_fmt, "error" 2019-10-16 03:41:05.084 1119 ERROR octavia File "/opt/amphora-agent-venv/lib/python3.5/site-packages/gunicorn/glogging.p y", line 449, in _set_syslog_handler 2019-10-16 03:41:05.084 1119 ERROR octavia facility=facility, socktype=socktype) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/usr/lib/python3.5/logging/handlers.py", line 806, in __init__ 2019-10-16 03:41:05.084 1119 ERROR octavia self._connect_unixsocket(address) 2019-10-16 03:41:05.084 1119 ERROR octavia File "/usr/lib/python3.5/logging/handlers.py", line 823, in _connect_unixsocket 2019-10-16 03:41:05.084 1119 ERROR octavia self.socket.connect(address) 2019-10-16 03:41:05.084 1119 ERROR octavia FileNotFoundError: [Errno 2] No such file or directory

3 11

OpenStack Train is officially released!
by Sean McGinnis 16 Oct '19

16 Oct '19

The official OpenStack Train release announcement has been sent out: http://lists.openstack.org/pipermail/openstack-announce/2019-October/002024… Thanks to all who were part of making the Train series a success! This marks the official opening of the releases repo for Ussuri, and freezes are now lifted. Train is now a full stable branch. Thanks! Sean

4 3

[PTL] PTG Team Photos
by Kendall Nelson 16 Oct '19

16 Oct '19

Hello Everyone! We are excited to see you in a few weeks at the PTG and wanted to share that we will be taking team photos again! Here is an ethercalc signup for the available time slots [1]. We will be providing time on Thursday Morning/Afternoon and Friday morning to come as a team to get your photo taken. Slots are only ten minutes so its *important that everyone be on time*! The location is TBD at this point, but it will likely be in the prefunction space near registration. Thanks, -Kendall Nelson (diablo_rojo) [1] https://ethercalc.openstack.org/lnupu1sx6ljl

1 1

[nova][ptg] Ussuri scope containment
by Eric Fried 16 Oct '19

16 Oct '19

Nova developers and maintainers- Every cycle we approve some number of blueprints and then complete a low percentage [1] of them. Which blueprints go unfinished seems to be completely random (notably, it appears to have nothing to do with our declared cycle priorities). This is especially frustrating for consumers of a feature, who (understandably) interpret blueprint/spec approval as a signal that they can reasonably expect the feature to land [2]. The cause for non-completion usually seems to fall into one of several broad categories: == Inadequate *developer* attention == - There's not much to be done about the subset of these where the contributor actually walks away. - The real problem is where the developer thinks they're ready for reviewers to look, but reviewers don't. Even things that seem obvious to experienced reviewers, like failing CI or "WIP" in the commit title, will cause patches to be completely ignored -- but unseasoned contributors don't necessarily understand even that, let alone more subtle issues. Consequently, patches will languish, with each side expecting the other to take the next action. This is a problem of culture: contributors don't understand nova reviewer procedures and psychology. == Inadequate *reviewer* attention == - Upstream maintainer time is limited. - We always seem to have low review activity until the last two or three weeks before feature freeze, when there's a frantic uptick and lots gets done. - But there's a cultural rift here as well. Getting maintainers to care about a blueprint is hard if they don't already have a stake in it. The "squeaky wheel" concept is not well understood by unseasoned contributors. The best way to get reviews is to lurk in IRC and beg. Aside from not being intuitive, this can also be difficult logistically (time zone pain, knowing which nicks to ping and how) as well as interpersonally (how much begging is enough? too much? when is it appropriate?). == Multi-release efforts that we knew were going to be multi-release == These may often drag on far longer than they perhaps should, but I'm not going to try to address that here. ======== There's nothing new or surprising about the above. We've tried to address these issues in various ways in the past, with varying degrees of effectiveness. I'd like to try a couple more. (A) Constrain scope, drastically. We marked 25 blueprints complete in Train [3]. Since there has been no change to the core team, let's limit Ussuri to 25 blueprints [4]. If this turns out to be too few, what's the worst thing that happens? We finish everything, early, and wish we had done more. If that happens, drinks are on me, and we can bump the number for V. (B) Require a core to commit to "caring about" a spec before we approve it. The point of this "core liaison" is to act as a mentor to mitigate the cultural issues noted above [5], and to be a first point of contact for reviews. I've proposed this to the spec template here [6]. Thoughts? efried [1] Like in the neighborhood of 60%. This is anecdotal; I'm not aware of a good way to go back and mine actual data. [2] Stuff happens, sure, and nobody expects 100%, but 60%? Come on, we have to be able to do better than that. [3] https://blueprints.launchpad.net/nova/train [4] Recognizing of course that not all blueprints are created equal, this is more an attempt at a reasonable heuristic than an actual expectation of total size/LOC/person-hours/etc. The theory being that constraining to an actual number, whatever the number may be, is better than not constraining at all. [5] If you're a core, you can be your own liaison, because presumably you don't need further cultural indoctrination or help begging for reviews. [6] https://review.opendev.org/685857

11 21

[oslo] On PTO rest of the week
by Ben Nemec 15 Oct '19

15 Oct '19

Hey, I'm not working for the rest of the week. \o/ I also realized that I haven't announced that here. /o\ Things are pretty quiet in Oslo right now so I don't anticipate that I'll be needed in the interim, but I should have internet the whole time I'm out and I'll _probably_ be checking email. If something comes up that can't wait until Monday just holler and hopefully I'll see it. Thanks. -Ben

1 0

[tripleo] Deprecating paunch CLI?
by Emilien Macchi 15 Oct '19

15 Oct '19

With our long-term goal to simplify TripleO and focus on what people actually deploy and how they operate their clouds, it appears that the Paunch CLI hasn't been a critical piece in our project and I propose that we deprecate it; create an Ansible module to call Paunch as a library only. I've been playing with it a little today: https://review.opendev.org/#/c/682093/ https://review.opendev.org/#/c/682094/ Here is how you would call paunch: - name: Start containers for step {{ step }} paunch: config: "/var/lib/tripleo-config/hashed-container-startup-config-step_{{ step }}.json" config_id: "tripleo_step{{ step }}" action: apply container_cli: "{{ container_cli }}" managed_by: "tripleo-{{ tripleo_role_name }}" A few benefits: - Deployment tasks in THT would call the new module instead of a shell command - More Pythonic and clean for Ansible, to interact with the actual task during the run - Removing some code in Paunch, make it easier to maintain for us For now, the Ansible module only covers "paunch apply", we will probably cover "delete" and "cleanup" eventually. Please let me know if you have any questions or concerns, -- Emilien Macchi

9 18

OpenStack COA Exam Update
by Jimmy McArthur 15 Oct '19

15 Oct '19

We wanted to circle back on this email thread <http://lists.openstack.org/pipermail/openstack-discuss/2019-January/001911.…> and provide an update on the Certified OpenStack Administrator (COA) exam. We’ve listened to community feedback and so has the OpenStack ecosystem. We are excited to collaborate with Mirantis who has stepped up to donate resources, including the administration of the vendor-neutral OpenStack certification exam now running on OpenStack Rocky. We are planning to continue COA exam sales starting this Thursday, October 17. If you’re interested in becoming a COA, you will be able to buy an exam through the OpenStack website <http://openstack.org/coa> or through one of the many OpenStack training partners in the marketplace <https://www.openstack.org/marketplace/training/>. We are excited to continue growing the market of certified OpenStack professionals and appreciate the community’s patience as we identified a solution moving forward. Please reach out if you have any questions and we will continue to update openstack.org/coa as the certification program evolves. Thanks, Jimmy

1 0

[ironic] [tripleo] IPA images without RPM and YUM/DNF?
by Dmitry Tantsur 15 Oct '19

15 Oct '19

(adding TripleO because of potential effect) Hi all, I'm working on making ironic-python-agent images smaller than they currently are. The proposed patches already reduce the default image (as built by IPA-builder) size from around 420 MiB to around 380 MiB. My next idea is to get rid of RPM and YUM databases (in case of a CentOS/RHEL image). They amount for nearly 100 MiB of the uncompressed image: $ du -sh var/lib/rpm 91M var/lib/rpm $ du -sh var/lib/yum 6.6M var/lib/yum How important for anyone is the ability to install/inspect packages inside a ramdisk? Dmitry

3 2

[Ops] ops meetups team meeting 2019-10-15 - minutes
by Chris Morgan 15 Oct '19

15 Oct '19

We had a brief meeting for the OpenStack Ops Meetups today on IRC, minutes linked below. The preparations for ops events at the Shanghai summit continue. There will be an "ops war stories" session during the forum, and then a mini ops meetup on day 4 (thursday) agenda still tbd, please make suggestions here : https://etherpad.openstack.org/p/PVG-OPS-Forum-Brainstorming Another bit of news is that Bloomberg intends to offer to host the next OpenStack Ops Meetup. If accepted this would be an event in our London headquarters on January 7th and 8th. More news will be shared here as and when available and via the ops notifications twitter account ( https://twitter.com/osopsmeetup) Today's meeting minutes: 10:40 AM Minutes: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… 10:40 AM Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… 10:40 AM Log: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… Cheers, Chris - on behalf of the OpenStack Ops Meetups team -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[ops] Shanghai Meetup
by Erik McCormick 15 Oct '19

15 Oct '19

Greetings Operators! We have been allocated a half-day session on Thursday afternoon as part of the PTG. We have room for 50 people and would like to use it as a mini Ops Meetup. For those who haven't attended one before, these are working sessions like the forum or PTG rather than formal presentations. Sessions don't need to be of fixed length for this, especially since it's a fairly short period of time. How much time we spend on a topic will be dictated by the cadence of the discussion and interest of the attendees. Please take a few minutes to add topic suggestions here, and +1 others that you would like to talk about. https://etherpad.openstack.org/p/PVG-OPS-Forum-Brainstorming Thanks, and see you in Shanghai! -Erik

1 0

Request to update email of Launchpad Account
by Pooja Jadhav 15 Oct '19

15 Oct '19

Hi Team, I am Pooja Jadhav. I have a Launchpad account with pooja.jadhav(a)nttdata.com but now I have left the company so I am not able to access this email any more, Hence I am not entitled to do forget password as well. So I am requesting community to suggest me alternative way to solve this problem through which I can put my Personal Email Id/ Current Corporate Id. Looking forward for your reply. Thanks & Regards, Pooja Jadhav

2 1

OSF booth at KubeCon+CloudNativeCon in San Diego
by denise＠openstack.org 15 Oct '19

15 Oct '19

Hello Everyone, We wanted to let you know that the OpenStack Foundation will have a booth at the upcoming KubeCon+CloudNativeCon event in San Diego, CA on November 18-21, 2019. We are in booth #S23 and we will be featuring the OpenStack Foundation in addition to all the projects - Airship, Kata Containers, StarlingX, OpenStack and Zuul. At the booth we will have stickers and educational collateral about each project to distribute. We would like to invite you to help us in the following areas: Represent your project by staffing the OSF booth If you can spare 1 hour/day (or any time at all!) to represent your specific project in the OSF booth Here is the [ link ]( https://docs.google.com/spreadsheets/d/1mZzK0GHm9OQ0IL9njTWLxPLuAeR6jqmyeM6… ) to the google doc to sign up Project Demos in the OSF booth If you are interested in delivering a project-specific demo in the OSF booth, please contact [ Denise ]( http://denise@openstack.org ) Looking forward to seeing all of you in San Diego! Best regards, OSF Marketing team Denise, Claire, Allison and Ashlee

1 0

[nova][ptg] Team dinner in Shanghai
by Stephen Finucane 15 Oct '19

15 Oct '19

We haven't done one of these in a while (formally, anyway), so I think it would be a good idea to take advantage of the lack of fussy eaters [1] present and organise a team dinner. For anyone that hasn't joined one of these before, it's a good opportunity for people that regularly work on nova to spend some time with their fellow nova contributors IRL (in real life) and for newer contributors to m̶e̶e̶t̶ ̶t̶h̶e̶i̶r̶ ̶h̶e̶r̶o̶e̶s̶ get to know the people reviewing their code. If you are interested, please state so in the Etherpad [2] along with days that work for you and I'll try organize a suitable venue. Stephen [1] Sorry Dan, Jay :P [2] https://etherpad.openstack.org/p/nova-shanghai-ptg

1 0

[nova][ptg] Ussuri planning etherpad for nova
by Stephen Finucane 15 Oct '19

15 Oct '19

With the PTG only a few weeks away, it's about time we started figuring out what we need to discuss there. The Ussuri PTG planning etherpad can be found at [1]. If you have something you want to discuss at the PTG then please include it there, even if you're not going to be there in person. I'm going to be away from this Friday until the summit but if we could get the bones of this in place this week, it should leave us (well, others in nova) enough time to organize the usual jumble of topics into something approaching an agenda. Stephen [1] https://etherpad.openstack.org/p/nova-shanghai-ptg

1 0

[neutron][Kolla] Failed to get DHCP offer packet at qvo/qvb in compute node
by Shuai Zhao 15 Oct '19

15 Oct '19

Hi Neutron, I've deployed Rocky-rc2 version on Debian Buster(compute node), kernel Linux 4.19 Now the issue: The VM running on the Host(Debian Buster) could not get IP when Booting. I use tcpdump to get the packet on tap, qbr, qvb and qvo. *The DHCP broadcast packet could be dumped at tap and qbr, but not at qvo/qvb.* So the DHCP failed. All the firewall policy is neutron automatic generated. The firewall policy is never changed. (neutron-openvswitch-agent)[root@** /]# iptables -S | grep tapba5cd56c-46 -A neutron-openvswi-FORWARD -m physdev --physdev-out tapba5cd56c-46 --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain -A neutron-openvswi-FORWARD -m physdev --physdev-in tapba5cd56c-46 --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain -A neutron-openvswi-INPUT -m physdev --physdev-in tapba5cd56c-46 --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-openvswi-oba5cd56c-4 -A neutron-openvswi-sg-chain -m physdev --physdev-out tapba5cd56c-46 --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-iba5cd56c-4 -A neutron-openvswi-sg-chain -m physdev --physdev-in tapba5cd56c-46 --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-oba5cd56c-4 (neutron-openvswitch-agent)[root@*** /]#* iptables -S | grep neutron-openvswi-oba5cd56c-4* -N neutron-openvswi-oba5cd56c-4 -A neutron-openvswi-INPUT -m physdev --physdev-in tapba5cd56c-46 --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-openvswi-oba5cd56c-4 -A neutron-openvswi-oba5cd56c-4 -s 0.0.0.0/32 -d 255.255.255.255/32 -p udp -m udp --sport 68 --dport 67 -m comment --comment "Allow DHCP client traffic." -j RETURN -A neutron-openvswi-oba5cd56c-4 -j neutron-openvswi-sba5cd56c-4 -A neutron-openvswi-oba5cd56c-4 -p udp -m udp --sport 68 --dport 67 -m comment --comment "Allow DHCP client traffic." -j RETURN -A neutron-openvswi-oba5cd56c-4 -p udp -m udp --sport 67 --dport 68 -m comment --comment "Prevent DHCP Spoofing by VM." -j DROP -A neutron-openvswi-oba5cd56c-4 -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN -A neutron-openvswi-oba5cd56c-4 -j RETURN -A neutron-openvswi-oba5cd56c-4 -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP -A neutron-openvswi-oba5cd56c-4 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback -A neutron-openvswi-sg-chain -m physdev --physdev-in tapba5cd56c-46 --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-oba5cd56c-4 Pls help to give some advices about that. Thanks a lot!

1 1

Writing a Train blog post?
by Allison Price 14 Oct '19

14 Oct '19

Hi everyone, Are you planning on writing a blog post about OpenStack Train ahead of (or after) the release this Wednesday, October 16? If so, please respond and let me know so we can include it in Train promotional activities, including the Open Infrastructure Community Newsletter. Thanks! Allison Allison Price OpenStack Foundation allison(a)openstack.org

1 0

[neutron][ptg] Ussuri planning etherpad for Neutron
by Slawek Kaplonski 14 Oct '19

14 Oct '19

Hi, At [1] there is Ussuri PTG planning etherpad. Please add Your topics to it, even if You are not planning to be there and You want team to discuss about it. I plan to start preparing agenda during the week of October 28th so would be great if You could add Your topics to it before this date. But of course any “last minute” topics are always welcome :) [1] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning — Slawek Kaplonski Senior software engineer Red Hat

1 0

[neutron] Team dinner in Shanghai
by Slawek Kaplonski 14 Oct '19

14 Oct '19

Hi neutrinos, We are planning to organise some team dinner during PTG in Shanghai. If You are interested to go for such dinner, please write it in etherpad [1] together with days which works the best for You. [1] https://etherpad.openstack.org/p/Shanghai-Neutron-Planning — Slawek Kaplonski Senior software engineer Red Hat

1 0

[tc] Weekly update
by Rico Lin 14 Oct '19

14 Oct '19

Hello friends, Here's what needs attention for the OpenStack TC this week. 1. We have our meeting last Thursday [1], so please work on actions [2]. And notice that some actions might require TCs to votes/helps on (like in [4]). 2. We have three potential goals for Ussuri [7] now. Please provide any if you found any suitable goal idea for Ussuri (or V) cycle. 3. For TC PTG, please propose topics before October 17, so we will have two weeks to discuss, finalize, and prepare. 4. Rico will help with chair's responsibility during JP's time off [5]. 5. Some recently started mailing list with [tc] or [all] tags: - [all][tc] What happened in OpenStack Governance recently [3] - [tc] Status update on naming our next releases [4] - [tc] Time off for JP! [5] - [nova][all] Roadmap for dropping Python 2 support [6] Thank you everyone! [1] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-10-10-14.00.log.html [2] http://eavesdrop.openstack.org/meetings/tc/2019/tc.2019-10-10-14.00.txt [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010113.… [4] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010106.… [5] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010061.… [6] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010109.… [7] https://etherpad.openstack.org/p/PVG-u-series-goals Regards, JP & Rico

1 0

[qa][ptg] Ussuri PTG Planning for QA
by Ghanshyam Mann 14 Oct '19

14 Oct '19

Hello Everyone, This is the etherpad[1] to collect the Ussuri cycle PTG topic ideas for QA. Please start adding your item/topic you want to discuss in PTG. Even you are not making to PTG physically, still, add your topic which you want us to discuss or give a thought. Anyone is welcome to add the cross-project testing topics they want to discuss related to QA. [1] https://etherpad.openstack.org/p/shanghai-ptg-qa -gmann

1 0

[all][qa][forum] Etherpad for Users / Operators adoption of QA tools / plugins sessions at Shanghai Summit
by Ghanshyam Mann 14 Oct '19

14 Oct '19

Hello Everyone, I've created the etherpad for the QA feedback sessions[1] which is scheduled on Monday, November 4, 1:20pm-2:00pm. I have added a few basic feedback questions there and If you have any additional items to add or modify, please feel free to do that. In case, you are not able to attend this session, you can still write your feedback with your irc/name contact. Etherpad: https://etherpad.openstack.org/p/PVG-forum-qa-ops-user-feedback [1] https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24401… -gmann

1 0

[all][tc] What happened in OpenStack Governance recently
by Rico Lin 14 Oct '19

14 Oct '19

Hello everyone, Here are a few things that happened recently: - *We've got the last volley of new PTLs!* Lucian Petrut for winstackers and Nicholas Bock for Designate. Congratulations! *- Rico Lin is now the Vice-chair of the TC.* - As mentioned in [1], *we will have two `Meet the project leaders` events* during the Shanghai summit. It will be nice if OpenStack PTLs, SIG Chairs, TC members, core reviewers, UC members, interested in join. You can signup in [2] to let others know you're coming. And if you think you might be part of this or you would like to meet any of them. Yes! you should come! -* We're open for goal idea*, so add it in [3] if you have any. We also looking for V cycle goal as well. And here's some backlog [4] if you're interested to be a champion of it. - At this point you should already know, *the newest OpenStack User Survey results are already out *[8]. So analysis it and make decisions accordingly (assume those questions are essential for team dicisions). - *The Shanghai PTG schedule is finalized*. Please check [7] for more detail on each forum. We hope we can have users, operators, and developers all together to collaborate and make more successful and valuable outcome from each forum. So please join! - Big thanks to our release team, *Train final releases for cycle-with-rc projects is right around the corner* [9]. - Our official meeting happened on October 10, as announced on the ML. We decided the following: - As one of meeting actions, a ML `[tc] Status update on naming our next releases` [6] is out (Thanks to Thierry). If you would like to give your review/feedback before proposals (mentioned in that mail) gets a majority of TC votes, now is the time. As always your review and feedback matter. - Summit and PTG are near, currently, Summit presentations and Forum sessions [7] are released. At this point, teams (which plan to join PTG) should start planning for their PTG topics and formats. So please help with teams to be prepared. For TC PTG, please propose topics before October 17, so we will have two weeks to discuss, finalize, and prepare. - V cycle goal discussion will be more asynchronously, more ML for U and V cycle goal process will be out soon. - Thanks for the effort of swift team, current python 3 support for swift is good. This makes OpenStack more ready for python 3 first. - There will be a forum [5] for large scale SIG, so if you're interested, please join (we hope to have more large scale users join to provide hands and feedbacks) so we can make OpenStack a better place for large scale. [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/009947.… [2] https://etherpad.openstack.org/p/meet-the-project-leaders [3] https://etherpad.openstack.org/p/PVG-u-series-goals [4] https://etherpad.openstack.org/p/community-goals [5] https://www.openstack.org/summit/shanghai-2019/summit-schedule/events/24405… [6] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010106.… [7] https://www.openstack.org/summit/shanghai-2019/summit-schedule#track_groups… [8] http://lists.openstack.org/pipermail/openstack-discuss/2019-September/00950… [9] https://review.opendev.org/#/c/687991 Regards, JP & Rico

1 0

[oslo] New courtesy ping list for Ussuri
by Ben Nemec 14 Oct '19

14 Oct '19

As we discussed at the beginning of the cycle, I'll be clearing the current ping list in the next few weeks. This is to prevent courtesy pinging people who are no longer active on the project. If you wish to continue receiving courtesy pings at the start of the Oslo meeting please add yourself to the new list on the agenda template [0]. Note that the new list is above the template, called "Courtesy ping list for Ussuri". If you add yourself again to the end of the existing list I'll assume you want to be left on though. :-) Thanks. -Ben 0: https://wiki.openstack.org/wiki/Meetings/Oslo#Agenda_Template

1 1

[watcher] Thesis on improving Watcher and collaborating with OpenStack community
by info＠dantalion.nl 14 Oct '19

14 Oct '19

Hello everyone, I am a Dutch student at the Amsterdam University of Applied Sciences (AUAS) and have recently finished my thesis. My thesis was written on improvements that were made to OpenStack Watcher between February and Juli of 2019. Specifically, many of these improvements were written to aid CERN in deploying Watcher. In addition, the thesis describes methods of collaboration and engaging in communities as well as evaluating strengths and weaknesses of communties. Since the thesis primarily resolves around OpenStack I would like to share it with the community as well. Please find the thesis attached to this email. Any feedback, remarks, future advice or other responses are appreciated. Kind regards, Corne Lukken (Dantali0n)

2 1

[nova][all] Roadmap for dropping Python 2 support
by Stephen Finucane 14 Oct '19

14 Oct '19

The time has come. Train is almost out the door and we're already well into Ussuri planning and work. We agreed some time ago that this cycle was the right one to drop support for Python 2.7 [1] and to that effect I've proposed a patch to do just this in nova [2]. However, I have noticed a large number of the third party CIs are failing on this patch. This is because they are still testing with Python 2 and the patch marks nova as only supporting Python 3. As you can see in that patch, the effort to switch things over is not that significant, and I'd ask that any owners of third party CIs prioritise work to switch these things over in the next few weeks leading up to the PTG so we can merge this change as soon as possible. Please reach out on IRC if you have any concerns or questions. Cheers, Stephen [1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecatio… [2] https://review.opendev.org/#/c/687954/

1 0

[tripleo] TripleO CI Summary: Sprint 37
by Rafael Folco 14 Oct '19

14 Oct '19

Greetings, The TripleO CI team has just completed Sprint 37 / Unified Sprint 16 (Sep 19 thru Oct 09). The following is a summary of completed work during this sprint cycle: - Started Train release branching prep work and bootstrapped a centos8 nodepool node. - Designed and implemented tests for verifying changes in the promotion server. - Added multi-arch support w/ manifests to container push in the promoter code. - Designed a test strategy for building and running jobs in zuul on ceph-ansible and podman repositories against pull requests on Github. The planned work for the next sprint [1] are: - Complete the manifest implementation with a test strategy for not breaking promotion workflow. - Improve tests for verifying a full promotion workflow running on the staging environment. - Implement CI jobs in zuul to build and run tests against ceph-ansible and podman pull requests in github. - Close-out Train release branching preparation work. - Address required changes for building a CentOS8 node for upcoming distro release support across TripleO CI jobs. The Ruck and Rover for this sprint are Rafael Folco (rfolco) and Marios Andreou (marios). Please direct questions or queries to them regarding CI status or issues in #tripleo, ideally to whomever has the ‘|ruck’ suffix on their nick. Ruck/rover notes are being tracked in etherpad [2]. Thanks, rfolco [1] https://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-17 [2] https://etherpad.openstack.org/p/ruckroversprint17

1 0

[neutron] Bug deputy report - week of October 7th
by Slawek Kaplonski 14 Oct '19

14 Oct '19

Hi, I was on bug deputy last week. It was really quiet week. Below is summary of new bugs reported: Critical test_update_firewall_calls_get_dvr_hosts_for_router failure on rocky - https://bugs.launchpad.net/neutron/+bug/1847019 - patch is already proposed https://review.opendev.org/#/c/687085/ Medium Loading neutron-lib internationalized file - https://bugs.launchpad.net/neutron/+bug/1847586 - fix proposed already https://review.opendev.org/687861 Low '--sql' option of neutron-db-manage does not work - https://bugs.launchpad.net/neutron/+bug/1847210 Undecided l3-agent stops processing router updates - https://bugs.launchpad.net/neutron/+bug/1847203 - I asked some additional questions there but would be also good if some L3 experts could take a look into that, Incomplete cannot reuse floating IP as port's fixed-ip - https://bugs.launchpad.net/neutron/+bug/1847763 - I think it’s for Calico project but lets wait until reporter clarify that RFEs [RFE] create option in neutron.conf to disable designate+neutron consistency - https://bugs.launchpad.net/neutron/+bug/1847068 Others [RPC] digging RPC timeout for client and server - https://bugs.launchpad.net/oslo.messaging/+bug/1847747 - More oslo_messaging issue IMO, neutron added only as impacted project. — Slawek Kaplonski Senior software engineer Red Hat

1 0

[tc] Status update on naming our next releases
by Thierry Carrez 14 Oct '19

14 Oct '19

Hi, Following last week TC meeting, I took the action to summarize the situation and way forward of naming releases. Naming our U release was a bit of a painful process, mostly due to the subjectivity of the process, combined with a difficult combination of letter vs. naming criteria. It triggered proposals to change the naming process and/or criteria, as we expect similar difficulties with the rest of the alphabet. We did a first round of proposals that covered the V-Z part of the alphabet. A Condorcet poll was run to select the best options. The two best ones were then run in a new Condorcet poll against "keep things the same", and the result was a draw. It was a good indication that the TC membership found none of the proposals on the table was significantly better than keeping things the same, and therefore no change was effected. That said, it does not mean we should stop proposing new models, as the current system is flawed: its subjectivity combined with a popularity contest creates problems, and its criteria (strongly tied to event locations) will not work well in the future as we work to reduce the number of global events we run while increasing the number of local events. The way forward is as follows: proposals can still be made, but they should address "any foreseeable future". That means they need to explain how they will name the V-Z releases, but also how they will roll over past Z. TC members should rollcall-vote +1 on those proposals if they think they are better than keeping things the same. They can rollcall-vote -1 on the proposals for which they think keeping things the same would be better. If one proposal gets a majority of votes (seven +1s), then after the usual grace period of 3 calendar days, it should be approved (unless a competing proposals gathers *more* positive votes in the mean time). There is no deadline for proposing, even after one such proposal is approved. Those things can always be changed in the future. However I personally don't think we should change naming systems too often, because they are only fun if they become some sort of tradition. We currently have three proposals up: Cities with 100,000+ inhabitants, TC-only poll: https://review.opendev.org/#/c/677745/ Vancouver, then words present in movie quotes about "release": https://review.opendev.org/#/c/684688/ Vancouver, then cities with 100,000+ inhabitants, community poll: https://review.opendev.org/#/c/687764/ Cheers, -- Thierry Carrez (ttx)

1 0

[kolla] Support for removing previously enabled services..
by Laurent Dumont 14 Oct '19

14 Oct '19

Hey everyone, I'm pretty sure I know the answer but are there any support within Kolla itself to disable Services that we're previously enabled. For example, I was testing the Skydive Agent/Analyzer combo till I realized that it was using about 90-100% of the CPUs or computes and controllers. [image: image.png] Re-running Kolla with reconfigure but with Service set to "No" didn't remove the containers. I had to remove the containers after the reconfigure finished. This is Kolla 8.0.1 with a Stein install. Thanks!

4 4

[goals][IPv6-Only Deployments and Testing] Update
by Ghanshyam Mann 14 Oct '19

14 Oct '19

Hello Everyone, Below is the updated on IPv6 goal. All the projects have the ipv6 job patch proposed now. Next step is to review then as per mentioned guidelines below or help in debugging the failure. As stable/train is already cut for all the projects, we will keep merging the remaining projects listed below in Ussuri release. If your project is listed below, check the project patch and help in review/debug failure. Summary: The projects waiting for IPv6 job patch to merge: If patch is failing, help me to debug that otherwise review and merge. * Barbican * Tricircle * Vitrage * Zaqar * Glance * Monasca * Neutron stadium projects (added a more generic job for all. need debugging as few tests failing- https://review.opendev.org/#/c/686043/) * Qinling * Sahara * Searchlight * Senlin * Tacker * Ec2-Api * Freezer * Heat * Ironic * Karbor * kuryr-kubernetes (not yet ready for IPv6. as per IRC chat with dulek, IPv6 support is planned for ussuri cycle - https://review.opendev.org/#/c/682531/) * Magnum * Masakari * Mistral * Octavia (johnsom is working on this) Storyboard: ========= - https://storyboard.openstack.org/#!/story/2005477 IPv6 missing support found: ===================== 1. https://review.opendev.org/#/c/673397/ 2. https://review.opendev.org/#/c/673449/ 3. https://review.opendev.org/#/c/677524/ There are few more but need to be tracked. How you can help: ============== - Each project needs to look for and review the ipv6 job patch. - Verify it works fine on ipv6 and no ipv4 used in conf etc - Any other specific scenario needs to be added as part of project IPv6 verification. - Help on debugging and fix the bug in IPv6 job is failing. Everything related to this goal can be found under this topic: Topic: https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+(stat… How to define and run new IPv6 Job on project side: ======================================= - I prepared a wiki page to describe this section - https://wiki.openstack.org/wiki/Goal-IPv6-only-deployments-and-testing Review suggestion: ============== - Main goal of these jobs will be whether your service is able to listen on IPv6 and can communicate to any other services either OpenStack or DB or rabbitmq etc on IPv6 or not. So check your proposed job with that point of view. If anything missing, comment on patch. - One example was - I missed to configure novnc address to IPv6- https://review.opendev.org/#/c/672493/ - base script as part of 'devstack-tempest-ipv6' will do basic checks for endpoints on IPv6 and some devstack var setting. But if your project needs more specific verification then it can be added in project side job as post-run playbooks as described in wiki page[1]. [1] https://wiki.openstack.org/wiki/Goal-IPv6-only-deployments-and-testing

2 1

[image-encryption] No meeting today
by Josephine Seifert 13 Oct '19

13 Oct '19

Hi, unfortunately neither Markus (mhen) nor me can hold the meeting today. We will have our next meeting next monday. greetings Josephine (Luzi)

1 0

monitoring
by Ali Ebrahimpour 13 Oct '19

13 Oct '19

hi guys i want to install monitoring in my horizon Ui and i'm confused in setting up ceilometer or gnocchi or aodh or monasca in my project because all of them where deprecated. i setup openstack with ansible <https://docs.openstack.org/project-deploy-guide/openstack-ansible/stein/>and i want to monitor the usage of cpu and ram and etc in my dashboard and i also want to know how much resources each customer used for one hour and day. Thanks in advance for your precise guidance.

1 0

[Trove] [Qinling] PTL on vacation
by Lingxian Kong 13 Oct '19

13 Oct '19

Hi all, I will be away from 15 Oct to 15 Nov. - Best regards, Lingxian Kong Catalyst Cloud

1 0

[Indian OpenStack User Group} Need Volunteer Mentors for Bangalore Friday Oct 18 10AM-4 PM
by prakash RAMCHANDRAN 12 Oct '19

12 Oct '19

Hi all, We have 150+ students and 3 classrooms & need technical mentors who can deliver following contentInformation about training OpenStack Docs: OpenStack Upstream Institute Training Content https://docs.openstack.org/upstream-training/upstream-training-content.html | | | | OpenStack Docs: OpenStack Upstream Institute | | | If you are in Bangalore (India) and your company allows you to fulfill your Corporate Social Responsibility (CSR) or you are a motivated Professional ready to help Engineering Students please respond to highlighted link and RSVP so that Event Co-coordinators can reach you to seek your help. Alternate is you can contact madhuri.rai07 or ganesh.hiregaoudar or digambarpat all AT gamailDOTcom. Once again appreciate all the help from tech folks from Bangalore India, who have been instrumental in supporting OpenStack for last decade. ThanksPrakash RamchandranEvent Coordinator

1 0

[kolla][tacker][glance] Deployment of Tacker Train (VNF CSAR packages issues)
by Radosław Piliszek 12 Oct '19

12 Oct '19

Hello Tackers! Some time ago I reported a bug in Kolla-Ansible Tacker deployment [1] Eduardo (thanks!) did some debugging to discover that you started requiring internal Glance configuration for Tacker to make it use the local filesystem via the filestore backend (internally in Tacker, not via the deployed Glance) [2] This makes us, Koalas, wonder how to approach a proper production deployment of Tacker. Tacker docs have not been updated regarding this new feature and following them may result in broken Tacker deployment (as we have now). We are especially interested in how to deal with multinode Tacker deployment. Do these new paths require any synchronization? [1] https://bugs.launchpad.net/kolla-ansible/+bug/1845142 [2] https://review.opendev.org/#/c/684275/2/ansible/roles/tacker/templates/tack… Kind regards, Radek

2 2

[keystone] Keystone Team Update - Week of 7 October 2019
by Colleen Murphy 11 Oct '19

11 Oct '19

# Keystone Team Update - Week of 7 October 2019 ## News ### RC2 We ended up cutting a second RC in order to remove the policy.v3cloudsample.json file[1] and to include placeholder schema migrations[2]. That RC should become the Train release next week[3]. [1] https://review.opendev.org/687639 [2] https://review.opendev.org/687775 [3] http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010074.… ## Office Hours When there are topics to cover, the keystone team holds office hours on Tuesdays at 17:00 UTC. There won't be office hours next week. Add topics you would like to see covered during office hours to the etherpad: https://etherpad.openstack.org/p/keystone-office-hours-topics ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 17 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 32 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ## Milestone Outlook https://releases.openstack.org/train/schedule.html Next week is the Train release! ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[neutron][security group][IPv6] IPv6 ICMPv6 port security in security group
by Xing Zhang 11 Oct '19

11 Oct '19

Hi all, When using neutron on CentOS 7 with OVSHybridIptablesFirewallDriver, create a vm with IPv4/IPv6 dual stack port, then remove all security group, we can get response with ping dhcp or router using IPv6 address in vm, while IPv4 can't. IPv6 works different with IPv4 in some cases and some useful function must work with ICMPv6 like NDP, NS, NA. Checking these two links below, neutron only drop IPv6 RA from vm, and allow all ICMPv6 ICMPv6 Type 128 Echo Request and Type 129 Echo Reply are allowed by default. Should we try to restrict ICMPv6 some types or there are some considerations and just follow ITEF 4890? IETF 4890 [section 4.3.2. Traffic That Normally Should Not Be Dropped] mentioned that: As discussed in Section 3.2 <https://tools.ietf.org/html/rfc4890#section-3.2>, the risks from port scanning in an IPv6 network are much less severe, and it is not necessary to filter IPv6 Echo Request messages. [section 3.2. Probing] However, the very large address space of IPv6 makes probing a less effective weapon as compared with IPv4 provided that addresses are not allocated in an easily guessable fashion. https://github.com/openstack/neutron/commit/a8a9d225d8496c044db7057552394af… https://www.ietf.org/rfc/rfc4890.txt Commands are: neutron port-update --no-security-groups 0307f016-0cc8-468b-bf3e-36ebe50e13ac ping6 from vm to dhcp ip6tables rules in compute node: PS: seems rules for type 131/135/143 are included in the rule # ip6tables-save | grep 08a0812a -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 143 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 134 -m comment --comment "Drop IPv6 Router Advts from VM Instance." -j DROP -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback full rules are at Ref #3 REF #1 ml2_config.ini [securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver Ref #2 Chain neutron-openvswi-o08a0812a-9 (2 references) pkts bytes target prot opt in out source destination 0 0 RETURN icmpv6 * * :: ff02::/16 ipv6-icmptype 131 /* Allow IPv6 ICMP traffic. */ 1 72 RETURN icmpv6 * * :: ff02::/16 ipv6-icmptype 135 /* Allow IPv6 ICMP traffic. */ 2 152 RETURN icmpv6 * * :: ff02::/16 ipv6-icmptype 143 /* Allow IPv6 ICMP traffic. */ 5 344 neutron-openvswi-s08a0812a-9 all * * ::/0 ::/0 0 0 DROP icmpv6 * * ::/0 ::/0 ipv6-icmptype 134 /* Drop IPv6 Router Advts from VM Instance. */ 5 344 RETURN icmpv6 * * ::/0 ::/0 /* Allow IPv6 ICMP traffic. */ 0 0 RETURN udp * * ::/0 ::/0 udp spt:546 dpt:547 /* Allow DHCP client traffic. */ 0 0 DROP udp * * ::/0 ::/0 udp spt:547 dpt:546 /* Prevent DHCP Spoofing by VM. */ 0 0 RETURN all * * ::/0 ::/0 state RELATED,ESTABLISHED /* Direct packets associated with a known session to the RETURN chain. */ 0 0 DROP all * * ::/0 ::/0 state INVALID /* Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack. */ 0 0 neutron-openvswi-sg-fallback all * * ::/0 ::/0 /* Send unmatched traffic to the fallback chain. */ Ref #3 # ip6tables-save | grep 08a0812a -A neutron-openvswi-PREROUTING -m physdev --physdev-in qvb08a0812a-9e -m comment --comment "Set zone for 812a-9ef7-45e3-9d81-9463dd80e63e" -j CT --zone 4104 -A neutron-openvswi-PREROUTING -i qvb08a0812a-9e -m comment --comment "Set zone for 812a-9ef7-45e3-9d81-9463dd80e63e" -j CT --zone 4104 -A neutron-openvswi-PREROUTING -m physdev --physdev-in tap08a0812a-9e -m comment --comment "Set zone for 812a-9ef7-45e3-9d81-9463dd80e63e" -j CT --zone 4104 :neutron-openvswi-i08a0812a-9 - [0:0] :neutron-openvswi-o08a0812a-9 - [0:0] :neutron-openvswi-s08a0812a-9 - [0:0] -A neutron-openvswi-FORWARD -m physdev --physdev-out tap08a0812a-9e --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain -A neutron-openvswi-FORWARD -m physdev --physdev-in tap08a0812a-9e --physdev-is-bridged -m comment --comment "Direct traffic from the VM interface to the security group chain." -j neutron-openvswi-sg-chain -A neutron-openvswi-INPUT -m physdev --physdev-in tap08a0812a-9e --physdev-is-bridged -m comment --comment "Direct incoming traffic from VM to the security group chain." -j neutron-openvswi-o08a0812a-9 -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 130 -j RETURN -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -j RETURN -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 136 -j RETURN -A neutron-openvswi-i08a0812a-9 -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN -A neutron-openvswi-i08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 134 -j RETURN -A neutron-openvswi-i08a0812a-9 -d 20ff::c/128 -p udp -m udp --sport 547 --dport 546 -j RETURN -A neutron-openvswi-i08a0812a-9 -d fe80::/64 -p udp -m udp --sport 547 --dport 546 -j RETURN -A neutron-openvswi-i08a0812a-9 -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP -A neutron-openvswi-i08a0812a-9 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 131 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 135 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -s ::/128 -d ff02::/16 -p ipv6-icmp -m icmp6 --icmpv6-type 143 -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -j neutron-openvswi-s08a0812a-9 -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m icmp6 --icmpv6-type 134 -m comment --comment "Drop IPv6 Router Advts from VM Instance." -j DROP -A neutron-openvswi-o08a0812a-9 -p ipv6-icmp -m comment --comment "Allow IPv6 ICMP traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -p udp -m udp --sport 546 --dport 547 -m comment --comment "Allow DHCP client traffic." -j RETURN -A neutron-openvswi-o08a0812a-9 -p udp -m udp --sport 547 --dport 546 -m comment --comment "Prevent DHCP Spoofing by VM." -j DROP -A neutron-openvswi-o08a0812a-9 -m state --state RELATED,ESTABLISHED -m comment --comment "Direct packets associated with a known session to the RETURN chain." -j RETURN -A neutron-openvswi-o08a0812a-9 -m state --state INVALID -m comment --comment "Drop packets that appear related to an existing connection (e.g. TCP ACK/FIN) but do not have an entry in conntrack." -j DROP -A neutron-openvswi-o08a0812a-9 -m comment --comment "Send unmatched traffic to the fallback chain." -j neutron-openvswi-sg-fallback -A neutron-openvswi-s08a0812a-9 -s 20ff::c/128 -m mac --mac-source FA:16:3E:7C:D8:C0 -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN -A neutron-openvswi-s08a0812a-9 -s fe80::f816:3eff:fe7c:d8c0/128 -m mac --mac-source FA:16:3E:7C:D8:C0 -m comment --comment "Allow traffic from defined IP/MAC pairs." -j RETURN -A neutron-openvswi-s08a0812a-9 -m comment --comment "Drop traffic without an IP/MAC allow rule." -j DROP -A neutron-openvswi-sg-chain -m physdev --physdev-out tap08a0812a-9e --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-i08a0812a-9 -A neutron-openvswi-sg-chain -m physdev --physdev-in tap08a0812a-9e --physdev-is-bridged -m comment --comment "Jump to the VM specific chain." -j neutron-openvswi-o08a0812a-9

2 1

Port creation times out for some VMs in large group
by Albert Braden 11 Oct '19

11 Oct '19

When I create 100 VMs in our prod cluster: openstack server create --flavor s1.tiny --network it-network --image cirros-0.4.0-x86_64 --min 100 --max 100 alberttest Most of them build successfully in about a minute. 5 or 10 will stay in BUILD status for 5 minutes and then fail with " BuildAbortException: Build of instance <UUID> aborted: Failed to allocate the network(s), not rescheduling." If I build smaller numbers, I see less failures, and no failures if I build one at a time. This does not happen in dev or QA; it appears that we are exhausting a resource in prod. I tried reducing various config values in dev but am not able to duplicate the issue. The neutron servers don't appear to be overloaded during the failure. What config variables should I be looking at? Here are the relevant log entries from the HV: 2019-09-26 10:10:43.001 57008 INFO os_vif [req-dea54d9a-3f3e-4d47-b901-a4f41b1947a8 d28c3871f61e4c8c8f8c7600417f7b14 e9621e3b105245ba8660f434ab21016c - default 4fb72165eee4468e8033cdc7d506ddf0] Successfully plugged vif VIFBridge(active=False,address=fa:16:3e:8b:45:07,bridge_name='brq49cbe55d-51',has_traffic_filtering=True,id=18f4e419-b19c-4b62-b6e4-152ec78e72bc,network=Network(49cbe55d-5188-4183-b5ad-e65f9b46f8f2),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap18f4e419-b1') 2019-09-26 10:15:44.029 57008 WARNING nova.virt.libvirt.driver [req-dea54d9a-3f3e-4d47-b901-a4f41b1947a8 d28c3871f61e4c8c8f8c7600417f7b14 e9621e3b105245ba8660f434ab21016c - default 4fb72165eee4468e8033cdc7d506ddf0] [instance: dc58f154-00f9-4c45-8986-94b10821cbc9] Timeout waiting for [('network-vif-plugged', u'18f4e419-b19c-4b62-b6e4-152ec78e72bc')] for instance with vm_state building and task_state spawning.: Timeout: 300 seconds More logs and data: http://paste.openstack.org/show/779524/

4 17

[keystone][edge][k8s] Keystone - StarlingX integration feedback
by Ildiko Vancsa 11 Oct '19

11 Oct '19

Hi, I wanted to point you to a thread that’s just started on the edge-computing mailing list: http://lists.openstack.org/pipermail/edge-computing/2019-October/000642.html The mail contains information about a use case that StarlingX has to use Keystone integrated with Kubernetes which I believe is valuable information to the Keystone team to see if there are any items to discuss further/fix/implement. Thanks, Ildikó

2 1

monitoring openstack
by Ali Ebrahimpour 11 Oct '19

11 Oct '19

hi guys i want to install monitoring in my horizon Ui and i'm confused in setting up ceilometer or gnocchi or aodh or monasca in my project because all of them where deprecated. i setup openstack with ansible <https://docs.openstack.org/project-deploy-guide/openstack-ansible/stein/>and i want to monitor the usage of cpu and ram and etc in my dashboard and i also want to know how much resources each customer used for one hour and day. Thanks in advance for your precise guidance.

2 1

[nova] rescue instances with volumes
by Ionut Biru 11 Oct '19

11 Oct '19

Hello guys, How do you guys rescue instances that are booted from volumes or have volumes attached to them? If i use nova rescue on instances booted from volume, api returns that it's not compatible and if i rescue an instance that has a volume attached, the drive is not available in the OS. -- Ionut Biru - https://fleio.com

2 1

[Openstack] FOSDEM 2020 Virtualization & IaaS Devroom CfP
by Piotr Kliczewski 11 Oct '19

11 Oct '19

We are excited to announce that the call for proposals is now open for the Virtualization & IaaS devroom at the upcoming FOSDEM 2020, to be hosted on February 1st 2020. This year will mark FOSDEM’s 20th anniversary as one of the longest-running free and open source software developer events, attracting thousands of developers and users from all over the world. FOSDEM will be held once again in Brussels, Belgium, on February 1st & 2nd, 2020. This devroom is a collaborative effort, and is organized by dedicated folks from projects such as OpenStack, Xen Project, oVirt, QEMU, KVM, and Foreman. We would like to invite all those who are involved in these fields to submit your proposals by December 1st, 2019. About the Devroom The Virtualization & IaaS devroom will feature session topics such as open source hypervisors and virtual machine managers such as Xen Project, KVM, bhyve, and VirtualBox, and Infrastructure-as-a-Service projects such as KubeVirt, Apache CloudStack, OpenStack, oVirt, QEMU and OpenNebula. This devroom will host presentations that focus on topics of shared interest, such as KVM; libvirt; shared storage; virtualized networking; cloud security; clustering and high availability; interfacing with multiple hypervisors; hyperconverged deployments; and scaling across hundreds or thousands of servers. Presentations in this devroom will be aimed at developers working on these platforms who are looking to collaborate and improve shared infrastructure or solve common problems. We seek topics that encourage dialog between projects and continued work post-FOSDEM. Important Dates Submission deadline: 1 December 2019 Acceptance notifications: 10 December 2019 Final schedule announcement: 15th December 2019 Devroom: 1st February 2020 Submit Your Proposal All submissions must be made via the Pentabarf event planning site[1]. If you have not used Pentabarf before, you will need to create an account. If you submitted proposals for FOSDEM in previous years, you can use your existing account. After creating the account, select Create Event to start the submission process. Make sure to select Virtualization and IaaS devroom from the Track list. Please fill out all the required fields, and provide a meaningful abstract and description of your proposed session. Submission Guidelines We expect more proposals than we can possibly accept, so it is vitally important that you submit your proposal on or before the deadline. Late submissions are unlikely to be considered. All presentation slots are 30 minutes, with 20 minutes planned for presentations, and 10 minutes for Q&A. All presentations will be recorded and made available under Creative Commons licenses. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example: "If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>." In the Submission notes field, please also confirm that if your talk is accepted, you will be able to attend FOSDEM and deliver your presentation. We will not consider proposals from prospective speakers who are unsure whether they will be able to secure funds for travel and lodging to attend FOSDEM. (Sadly, we are not able to offer travel funding for prospective speakers.) Submission Guidelines Mentored presentations will have 25-minute slots, where 20 minutes will include the presentation and 5 minutes will be reserved for questions. The number of newcomer session slots is limited, so we will probably not be able to accept all applications. You must submit your talk and abstract to apply for the mentoring program, our mentors are volunteering their time and will happily provide feedback but won't write your presentation for you! If you are experiencing problems with Pentabarf, the proposal submission interface, or have other questions, you can email our devroom mailing list[2] and we will try to help you. How to Apply In addition to agreeing to video recording and confirming that you can attend FOSDEM in case your session is accepted, please write "speaker mentoring program application" in the "Submission notes" field, and list any prior speaking experience or other relevant information for your application. Code of Conduct Following the release of the updated code of conduct for FOSDEM, we'd like to remind all speakers and attendees that all of the presentations and discussions in our devroom are held under the guidelines set in the CoC and we expect attendees, speakers, and volunteers to follow the CoC at all times. If you submit a proposal and it is accepted, you will be required to confirm that you accept the FOSDEM CoC. If you have any questions about the CoC or wish to have one of the devroom organizers review your presentation slides or any other content for CoC compliance, please email us and we will do our best to assist you. Call for Volunteers We are also looking for volunteers to help run the devroom. We need assistance watching time for the speakers, and helping with video for the devroom. Please contact devroom mailing list [2] for more information. Questions? If you have any questions about this devroom, please send your questions to our devroom mailing list. You can also subscribe to the list to receive updates about important dates, session announcements, and to connect with other attendees. See you all at FOSDEM! [1] <https://penta.fosdem.org/submission/FOSDEM17> https://penta.fosdem.org/submission/FOSDEM20 [2] iaas-virt-devroom at lists.fosdem.org

1 0

[release] Release countdown for week R-0, October 14-18
by Sean McGinnis 10 Oct '19

10 Oct '19

Development Focus ----------------- We will be releasing the coordinated OpenStack Train release next week, on Wednesday October 16th. Thanks to everyone involved in the Train cycle! We are now in pre-release freeze, so no new deliverable will be created until final release, unless a release-critical regression is spotted. Otherwise, teams attending the PTG in Shanghai should start to plan what they will be discussing there, by creating and filling team etherpads. You can find the list of etherpads at: http://ptg.openstack.org/etherpads.html General Information ------------------- On release day, the release team will produce final versions of deliverables following the cycle-with-rc release model, by re-tagging the commit used for the last RC. A patch doing just that will be proposed. PTLs and release liaisons should watch for that final release patch from the release team. While not required, we would appreciate having an ack from each team before we approve it on the 16th. Upcoming Deadlines & Dates -------------------------- Final Train release: October 16 Forum+PTG at Shanghai summit: November 4

1 1