October 2019 - openstack-discuss - lists.openstack.org

[openstack][heat-cfn] CFN Signaling with heat
by Ajay Kalambur (akalambu) 09 Oct '19

09 Oct '19

Hi I was trying the Software Deployment/Structured deployment of heat. I somehow can never get the signaling to work I see that authentication is happening but I don’t see a POST from the VM as a result stack is stuck in CREATE_IN_PROGRESS I see this message in my heat api cfn log which seems to suggest authentication is successful but it does not seem to POST. Have included debug output from VM and also the sample heat template I used. Don’t know if the template is correct as I referred some online examples to build it 2019-10-05 10:30:00.908 7 INFO heat.api.aws.ec2token [-] Checking AWS credentials.. 2019-10-05 10:30:00.909 7 INFO heat.api.aws.ec2token [-] AWS credentials found, checking against keystone. 2019-10-05 10:30:00.910 7 INFO heat.api.aws.ec2token [-] Authenticating with http://10.10.173.9:5000/v3/ec2tokens 2019-10-05 10:30:01.315 7 INFO heat.api.aws.ec2token [-] AWS authentication successful. 2019-10-05 10:30:02.326 7 INFO eventlet.wsgi.server [req-506f22c6-4062-4a84-8e85-40317a4099ed - adccd09df89e4b71b0a42f462679e75a-b1c6eb69-3877-466b-b00d-03dc051 - 0ecadd4762a34de1ac08508db4d3caa9 0ecadd4762a34de1ac08508db4d3caa9] 10.11.59.36,10.10.173.9 - - [05/Oct/2019 10:30:02] "GET /v1/?SignatureVersion=2&AWSAccessKeyId=f7874ac9898248edaae53511230534a4&StackName=test_stack&SignatureMethod=HmacSHA256&Signature=c03Q7Hb35q9tPPuYOv6YByn5YekF96p2s5zx36sX7x4%3D&Action=DescribeStackResource&LogicalResourceId=sig-vm-1 HTTP/1.1" 200 4669 1.418045 Some debugging output from my VM: [root@sig-vm-1 fedora]# sudo os-collect-config --force --one-time --debug /var/lib/os-collect-config/local-data not found. Skipping [2019-10-05 17:32:47,058] (os-refresh-config) [INFO] Starting phase pre-configure dib-run-parts Sat Oct 5 17:32:47 UTC 2019 ----------------------- PROFILING ----------------------- dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Target: pre-configure.d dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Script Seconds dib-run-parts Sat Oct 5 17:32:47 UTC 2019 --------------------------------------- ---------- dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 --------------------- END PROFILING --------------------- [2019-10-05 17:32:47,091] (os-refresh-config) [INFO] Completed phase pre-configure [2019-10-05 17:32:47,092] (os-refresh-config) [INFO] Starting phase configure dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Running /usr/libexec/os-refresh-config/configure.d/20-os-apply-config [2019/10/05 05:32:47 PM] [INFO] writing /var/run/heat-config/heat-config [2019/10/05 05:32:47 PM] [INFO] writing /etc/os-collect-config.conf [2019/10/05 05:32:47 PM] [INFO] success dib-run-parts Sat Oct 5 17:32:47 UTC 2019 20-os-apply-config completed dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Running /usr/libexec/os-refresh-config/configure.d/50-heat-config-docker-compose dib-run-parts Sat Oct 5 17:32:47 UTC 2019 50-heat-config-docker-compose completed dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Running /usr/libexec/os-refresh-config/configure.d/50-heat-config-kubelet dib-run-parts Sat Oct 5 17:32:47 UTC 2019 50-heat-config-kubelet completed dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Running /usr/libexec/os-refresh-config/configure.d/55-heat-config [2019-10-05 17:32:47,724] (heat-config) [ERROR] Skipping group Heat::Ungrouped with no hook script None [2019-10-05 17:32:47,724] (heat-config) [ERROR] Skipping group Heat::Ungrouped with no hook script None dib-run-parts Sat Oct 5 17:32:47 UTC 2019 55-heat-config completed dib-run-parts Sat Oct 5 17:32:47 UTC 2019 ----------------------- PROFILING ----------------------- dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Target: configure.d dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Script Seconds dib-run-parts Sat Oct 5 17:32:47 UTC 2019 --------------------------------------- ---------- dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 20-os-apply-config 0.345 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 50-heat-config-docker-compose 0.064 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 50-heat-config-kubelet 0.134 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 55-heat-config 0.065 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 dib-run-parts Sat Oct 5 17:32:47 UTC 2019 --------------------- END PROFILING --------------------- [2019-10-05 17:32:47,787] (os-refresh-config) [INFO] Completed phase configure [2019-10-05 17:32:47,787] (os-refresh-config) [INFO] Starting phase post-configure dib-run-parts Sat Oct 5 17:32:47 UTC 2019 Running /usr/libexec/os-refresh-config/post-configure.d/99-refresh-completed ++ os-apply-config --key completion-handle --type raw --key-default '' + HANDLE= ++ os-apply-config --key completion-signal --type raw --key-default '' + SIGNAL= ++ os-apply-config --key instance-id --type raw --key-default '' + ID=i-0000000d + '[' -n i-0000000d ']' + '[' -n '' ']' + '[' -n '' ']' ++ os-apply-config --key deployments --type raw --key-default '' ++ jq -r 'map(select(.group == "os-apply-config") | select(.inputs[].name == "deploy_signal_id") | .id + (.inputs | map(select(.name == "deploy_signal_id")) | .[].value)) | .[]' + DEPLOYMENTS= + DEPLOYED_DIR=/var/lib/os-apply-config-deployments/deployed + '[' '!' -d /var/lib/os-apply-config-deployments/deployed ']' dib-run-parts Sat Oct 5 17:32:49 UTC 2019 99-refresh-completed completed dib-run-parts Sat Oct 5 17:32:49 UTC 2019 ----------------------- PROFILING ----------------------- dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 Target: post-configure.d dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 Script Seconds dib-run-parts Sat Oct 5 17:32:49 UTC 2019 --------------------------------------- ---------- dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 99-refresh-completed 1.206 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 --------------------- END PROFILING --------------------- [2019-10-05 17:32:49,041] (os-refresh-config) [INFO] Completed phase post-configure [2019-10-05 17:32:49,042] (os-refresh-config) [INFO] Starting phase migration dib-run-parts Sat Oct 5 17:32:49 UTC 2019 ----------------------- PROFILING ----------------------- dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 Target: migration.d dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 Script Seconds dib-run-parts Sat Oct 5 17:32:49 UTC 2019 --------------------------------------- ---------- dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 dib-run-parts Sat Oct 5 17:32:49 UTC 2019 --------------------- END PROFILING --------------------- [2019-10-05 17:32:49,073] (os-refresh-config) [INFO] Completed phase migration onfig]# cat /var/run/heat-config/heat-config [{"inputs": [{"type": "String", "name": "foo", "value": "fu"}, {"type": "String", "name": "bar", "value": "barmy"}, {"type": "String", "name": "deploy_server_id", "value": "226ed96d-2335-436e-9707-95af73041e5f", "description": "ID of the server being deployed to"}, {"type": "String", "name": "deploy_action", "value": "CREATE", "description": "Name of the current action being deployed"}, {"type": "String", "name": "deploy_stack_id", "value": "test_stack/b1c6eb69-3877-466b-b00d-03dc051d1893", "description": "ID of the stack this deployment belongs to"}, {"type": "String", "name": "deploy_resource_name", "value": "other_deployment", "description": "Name of this deployment resource in the stack"}, {"type": "String", "name": "deploy_signal_transport", "value": "CFN_SIGNAL", "description": "How the server should signal to heat with the deployment output values."}, {"type": "String", "name": "deploy_signal_id", "value": "http://172.29.85.87:8000/v1/signal/arn%3Aopenstack%3Aheat%3A%3Aadccd09df89e…", "description": "ID of signal to use for signaling output values"}, {"type": "String", "name": "deploy_signal_verb", "value": "POST", "description": "HTTP verb to use for signaling outputvalues"}], "group": "Heat::Ungrouped", "name": "test_stack-config-bmekpj67pq6p", "outputs": [], "creation_time": "2019-10-05T01:14:31Z", "options": {}, "config": {"config_value_foo": "fu", "config_value_bar": "barmy"}, "id": "5c404619-ce79-48cd-b001-00ac6ff4f4e8"}, {"inputs": [{"type": "String", "name": "foo", "value": "fooooo"}, {"type": "String", "name": "bar", "value": "baaaaa"}, {"type": "String", "name": "deploy_server_id", "value": "226ed96d-2335-436e-9707-95af73041e5f", "description": "ID of the server being deployed to"}, {"type": "String", "name": "deploy_action", "value": "CREATE", "description": "Name of the current action being deployed"}, {"type": "String", "name": "deploy_stack_id", "value": "test_stack/b1c6eb69-3877-466b-b00d-03dc051d1893", "description": "ID of the stack this deployment belongs to"}, {"type": "String", "name": "deploy_resource_name", "value": "deployment", "description": "Name of this deployment resource in the stack"}, {"type": "String", "name": "deploy_signal_transport", "value": "CFN_SIGNAL", "description": "How the server should signal to heat with the deployment output values."}, {"type": "String", "name": "deploy_signal_id", "value": "http://172.29.85.87:8000/v1/signal/arn%3Aopenstack%3Aheat%3A%3Aadccd09df89e…", "description": "ID of signal to use for signaling output values"}, {"type": "String", "name": "deploy_signal_verb", "value": "POST", "description": "HTTP verb to use for signaling outputvalues"}], "group": "Heat::Ungrouped", "name": "test_stack-config-bmekpj67pq6p", "outputs": [], "creation_time": "2019-10-05T01:14:31Z", "options": {}, "config": {"config_value_foo": "fooooo", "config_value_bar": "baaaaa"}, "id": "f4dea0c1-73c9-4ce4-aa04-c76ef9b08859"}][root@sig-vm-1 heat-config]# [root@sig-vm-1 heat-config]# cat /etc/os-collect-config.conf [DEFAULT] command = os-refresh-config collectors = ec2 collectors = cfn collectors = local [cfn] metadata_url = http://172.29.85.87:8000/v1/ stack_name = test_stack secret_access_key = npa^GWsPtbRL7D*MYObOI*kV0i1yqKOG access_key_id = f7874ac9898248edaae53511230534a4 path = sig-vm-1.Metadata Here is my basic sample temple heat_template_version: 2013-05-23 description: > This template demonstrates how to use OS::Heat::StructuredDeployment to override substitute get_input placeholders defined in OS::Heat::StructuredConfig config. As there is no hook on the server to act on the configuration data, these deployment resource will perform no actual configuration. parameters: flavor: type: string default: 'a061cb6c-99e7-4bdb-93e4-f0037ee3e947' image: type: string default: 3be29d9f-2ce6-4b95-b80c-0dbca7acfdfe public_net_id: type: string default: 67ae0e17-6258-4fb6-8b9b-0f29f6adb9db private_net_id: type: string description: Private network id default: 995fc046-1c58-468a-b81c-e42c06fc8966 private_subnet_id: type: string description: Private subnet id default: 7598c805-3a9b-4c27-be5b-dca4d89f058c password: type: string description: SSH password default: lab123 resources: the_sg: type: OS::Neutron::SecurityGroup properties: name: the_sg description: Ping and SSH rules: - protocol: icmp - protocol: tcp port_range_min: 22 port_range_max: 22 config: type: OS::Heat::StructuredConfig properties: config: config_value_foo: {get_input: foo} config_value_bar: {get_input: bar} deployment: type: OS::Heat::StructuredDeployment properties: signal_transport: CFN_SIGNAL config: get_resource: config server: get_resource: sig-vm-1 input_values: foo: fooooo bar: baaaaa other_deployment: type: OS::Heat::StructuredDeployment properties: signal_transport: CFN_SIGNAL config: get_resource: config server: get_resource: sig-vm-1 input_values: foo: fu bar: barmy server1_port0: type: OS::Neutron::Port properties: network_id: { get_param: private_net_id } security_groups: - default fixed_ips: - subnet_id: { get_param: private_subnet_id } server1_public: type: OS::Neutron::FloatingIP properties: floating_network_id: { get_param: public_net_id } port_id: { get_resource: server1_port0 } sig-vm-1: type: OS::Nova::Server properties: name: sig-vm-1 image: { get_param: image } flavor: { get_param: flavor } networks: - port: { get_resource: server1_port0 } user_data_format: SOFTWARE_CONFIG user_data: get_resource: cloud_config cloud_config: type: OS::Heat::CloudConfig properties: cloud_config: password: { get_param: password } chpasswd: { expire: False } ssh_pwauth: True

2 2

[requirements][heat] remove salt from requirements (used by heat-agents tests only)
by Matthew Thode 09 Oct '19

09 Oct '19

Salt has been harsh to deal with. Upstream adding and maintaining caps has caused it to be held back. This time it's pyyaml, I'm not going to hold back the version of pyyaml for one import of salt. In any case, heat-agents uses salt in one location and may not even be using the one we define via constraints in any case. File: heat-config-salt/install.d/50-heat-config-hook-salt Installs salt from package then runs heat-config-salt/install.d/hook-salt.py In heat-config-salt/install.d/hook-salt.py is defined the only import of salt I can find and likely uses the package version as it's installed after tox sets things up. Is the heat team ok with this? -- Matthew Thode

2 2

[manila] Proposal to add dviroel to the core maintainers team
by Goutham Pacha Ravi 08 Oct '19

08 Oct '19

Dear Zorillas and other Stackers, I would like to formalize the conversations we've been having amongst ourselves over IRC and in-person. At the outset, we have a lot of incoming changes to review, but we have limited core maintainer attention. We haven't re-jigged our core maintainers team as often as we'd like, and that's partly to blame. We have some relatively new and enthusiastic contributors that we would love to encourage to become maintainers! We've mentored contributors 1-1, n-1 before before adding them to the maintainers team. We would like to do more of this!** In this spirit, I would like your inputs on adding Douglas Viroel (dviroel) to the core maintainers team for manila and its associated projects (manila-specs, manila-ui, python-manilaclient, manila-tempest-plugin, manila-test-image, manila-image-elements). Douglas has been an active contributor for the past two releases and has valuable review inputs in the project. While he's been around here less longer than some of us, he brings a lot of experience to the table with his background in networking and shared file systems. He has a good grasp of the codebase and is enthusiastic in adding new features and fixing bugs in the Ussuri cycle and beyond. Please give me a +/-1 for this proposal. ** If you're interested in helping us maintain Manila by being part of the manila core maintainer team, please reach out to me or any of the current maintainers, we would love to work with you and help you grow into that role! Thanks, Goutham Pacha Ravi (gouthamr)

9 10

Neutron dhcp-agent scalability techniques
by fsbiz＠yahoo.com 08 Oct '19

08 Oct '19

Hi folks, We have a rather large flat network consisting of over 300 ironic baremetal nodesand are constantly having the baremetals timing out during their PXE boot due tothe dhcp agent not able to respond in time. Looking for inputs on successful DHCP scaling techniques that would help mitigate this. thanks,Fred.

3 6

[ops] ops meetups team meeting 2019-10-8
by Chris Morgan 08 Oct '19

08 Oct '19

Minutes from todays meeting are here: 10:56 AM Minutes: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… 10:56 AM Minutes (text): http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… 10:56 AM Log: http://eavesdrop.openstack.org/meetings/ops_meetup_team/2019/ops_meetup_tea… The Ops Community attending the upcoming Summit in Shanghai will have one Forum session (ops war stories). On day 4 we will also have a 3 hours session for further ops related topic discussion. Details still to be arranged. Chris - on behalf of the openstack ops meetups team -- Chris Morgan <mihalis68(a)gmail.com>

1 0

[cinder] Train release status
by Brian Rosmaita 08 Oct '19

08 Oct '19

You may have noticed that RC-2 was released yesterday. We aren't planning to do an RC-3 unless a critical bugfix is approved for backport. Here's the timeline: - Now through Friday 11 October: RC-3, etc. are cut as necessary - The "final RC" is whatever RC-n exists on 11 October - The coordinated release date is 16 October * Any bugfixes caught after 11 Oct can be merged into stable/train, but it is up to the release team whether they can be included in the release. Please do some exploratory testing on the 15.0.0.0rc2 tag (which right now is the HEAD of stable/train). If you find a critical bug, please file it in Launchpad and tag it 'train-rc-potential'. Also add it to the etherpad: https://etherpad.openstack.org/p/cinder-train-backport-potential and make some noise in #openstack-cinder so we are all aware of it. cheers, brian

1 0

Re: cinder 15.0.0.0rc2 (train)
by Thomas Goirand 08 Oct '19

08 Oct '19

On 10/7/19 4:40 PM, no-reply(a)openstack.org wrote: > Hello everyone, > > A new release candidate for cinder for the end of the Train > cycle is available! You can find the source code tarball at: > > https://tarballs.openstack.org/cinder/ Hi, For the 2nd time, could we *please* re-add the tag: [release-announce] when announcing for a release? I don't mind if it's sent to -discuss instead of the announce this, but this breaks mail filters... Cheers, Thomas Goirand (zigo)

2 3

[nova] Request to include routed networks support in the Ussuri cucly goals
by Miguel Lavalle 08 Oct '19

08 Oct '19

Hi Stackers, I want to request the inclusion of the support for Neutron Routed Networks in the Nova goals for the Ussuri cycle. As many of you might know, Routed networks is a feature in Neutron that enables the creation of very large virtual networks that avoids the performance penalties of large L2 broadcast domains ( https://www.openstack.org/videos/summits/barcelona-2016/scaling-up-openstac…) This functionality can be very helpful for large deployers who have the need to have one or a few large virtual networks shared by all their users and has been available in Neutron since very soon after the Barcelona Summit in 2016. But it is really useless until there is code in Nova that can schedule VMs to compute hosts based on the segments topology of the routed networks. Without it, VMs can land in compute hosts where their traffic cannot be routed by the underlying network infrastructure. I would like the Nova team to consider the following when making a decision about this request: 1. Work for Routed Networks was approved as a priority for the Ocata cycle, although it wasn't concluded: https://specs.openstack.org/openstack/nova-specs/priorities/ocata-prioritie… and https://specs.openstack.org/openstack/nova-specs/specs/pike/index.html 2. The are several large deployers that need this feature. Verizon Media, my employer, is one of them. Others that come to mind include GoDaddy and Cern. And I am sure there are others. 3. There is a WIP patch to implement the functionality some of the functionality: https://review.opendev.org/#/c/656885. We, at Verizon Media, are proposing to take over this work and finish its implementation by the end of U. What we are requesting is Nova core reviewers bandwidth to help us merge the code I will be attending the PTG in Shanghai and will make myself available to discuss this further in person any day and any time. Hopefully, we can get this feature lined up very soon Best regards Miguel

5 5

[OSSA-2019-005] Octavia Amphora-Agent not requiring Client-Certificate (CVE-2019-17134)
by Daniel 'f0o' Preussker 08 Oct '19

08 Oct '19

===================================================================== OSSA-2019-005: Octavia Amphora-Agent not requiring Client-Certificate ===================================================================== :Date: October 07, 2019 :CVE: CVE-2019-17134 Affects ~~~~~~~ - Octavia: >=0.10.0 <2.1.2, >=3.0.0 <3.2.0, >=4.0.0 <4.1.0 Description ~~~~~~~~~~~ Daniel Preussker reported a vulnerability in amphora-agent, running within Octavia Amphora Instances which allows unauthenticated access from the management network. This leads to information disclosure and also allows changes to the configuration of the Amphora via simple HTTP requests because cmd/agent.py gunicorn cert_reqs option is incorrectly set to True instead of ssl.CERT_REQUIRED. Patches ~~~~~~~ - https://review.opendev.org/686547 (Ocata) - https://review.opendev.org/686546 (Pike) - https://review.opendev.org/686545 (Queens) - https://review.opendev.org/686544 (Rocky) - https://review.opendev.org/686543 (Stein) - https://review.opendev.org/686541 (Train) Credits ~~~~~~~ - Daniel Preussker (CVE-2019-17134) References ~~~~~~~~~~ - https://storyboard.openstack.org/#!/story/2006660 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17134 Notes ~~~~~ - The stable/ocata and stable/pike branches are under extended maintenance and will receive no new point releases, but patches for them are provided as a courtesy.

1 0

[nova] Stepping down from core reviewer
by Kenichi Omichi 08 Oct '19

08 Oct '19

Hello, Today my job description is changed and I cannot have enough time for regular reviewing work of Nova project. So I need to step down from the core reviewer. I spend 6 years in the project, the experience is amazing. OpenStack gave me a lot of chances to learn technical things deeply, make friends in the world and bring me and my family to foreign country from our home country. I'd like to say thank you for everyone in the community :-) My personal private cloud is based on OpenStack, so I'd like to still keep contributing for the project if I find bugs or idea. Thanks Kenichi Omichi ---

11 10