January 2026 - openstack-discuss - lists.openstack.org

[Nova] Proposal: NetApp FlexGroup Backend for Ceph-like Ephemeral Storage Performance on NFS
by Borra, Jayaanand 08 Jan '26

08 Jan '26

Hi everyone, Happy New Year to all! I hope you all had a great holiday season. I'm reaching out to share a proposal we've been working on that addresses a performance challenge many enterprises face with ephemeral storage in Nova. Context: Currently, OpenStack deployments running on NFS-based storage (like NetApp FlexGroups) face a significant performance penalty compared to Ceph RBD due to the QCOW2-over-NFS approach. The hypervisor manages Copy-on-Write operations, which introduces: * Double filesystem metadata overhead (QCOW2 on top of NFS) * CPU tax on compute nodes for CoW management * Slower instance provisioning compared to Ceph's instant cloning We are planning to implement solution that leverages NetApp's FlexClone technology through the ONTAP REST API to achieve Ceph-like performance while maintaining the operational simplicity of NFS. Essentially, we're offloading the cloning operation to the storage array itself, making instance creation a metadata-only operation (sub-second) with zero data copying. The implementation involves creating a new NetAppNfs image backend class in nova/virt/libvirt/imagebackend.py that calls the ONTAP REST API's /storage/file/clone endpoint instead of traditional file copy operations. I'm wondering if anyone has insights into: * Similar efforts: Has anyone explored or implemented storage-side cloning for other NFS vendors? * Design concerns: Any architectural red flags with this approach in the Nova driver? * Security practices: Best approach for managing storage credentials and oslo.privsep<vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/code/electron-browser/workbench/workbench.html> integration? * Community interest: Would this be valuable for other NFS deployments? This will enable high-performance ephemeral storage from, so I'm curious if others have encountered this limitation and whether there's are any known limitation with this approach. Thanks in advance for any guidance or suggestions! Best regards, Jayaanand.

1 0

SECURITY BUG
by Umbrella Corporation 08 Jan '26

08 Jan '26

i found a vulnerability in your git repo so if i reported bugs and vulnerabilities can i get a reward bounty ?

3 3

OpenStack 2026.2 'H' Release Name Poll
by Allison Price 08 Jan '26

08 Jan '26

Hi everyone, Happy New Year! To kick off 2026, I wanted to share the poll to vote for the OpenStack 2026.2 release name, starting with the letter H. The poll will close this Thursday, January 8 at 1700 UTC. https://civs1.civs.us/cgi-bin/vote.pl?id=E_e4708c67d8d7e171&akey=e3b1514078… If you have any questions / comments, please let me know. Cheers, Allison

1 1

[designate] Question about NS record update restriction in multi-region PRIMARY/SECONDARY setup
by Sylvain 08 Jan '26

08 Jan '26

Hi everyone, I hope this email finds you well. I'm reaching out to the community because I'm encountering a limitation with Designate that I don't fully understand, and I'm hoping someone can shed some light on the reasoning behind it or suggest alternative approaches. My Setup I'm running a multi-region Designate deployment with: - 3 independent Designate instances (one per region) - Each instance has its own pool with a single nameserver: - Region 1: ns.region1.net. - Region 2: ns.region2.net. - Region 3: ns.region3.net. This default configuration works perfectly for most of our use cases where zones are region-specific. What I'm Trying to Achieve For certain domains, I need to set up a PRIMARY/SECONDARY configuration across all three regions: - Region 1: PRIMARY zone for example.com - Region 2: SECONDARY zone for example.com - Region 3: SECONDARY zone for example.com For this to work correctly, the PRIMARY zone needs to have NS records pointing to all three nameservers so that: 1. DNS clients can query any of the three nameservers 2. Zone transfers are properly configured between primary and secondaries The Problem When I try to update the NS recordset on the primary zone, I get the error: Updating a root zone NS record is not allowed Steps I'm following: # List recordsets to get the NS recordset ID openstack recordset list example.com. # Attempt to update the NS recordset openstack recordset set example.com. <ns-recordset-id> \ --record ns.region1.net. \ --record ns.region2.net. \ --record ns.region3.net. This happens even when authenticated as an admin user. Why the Default Workaround Doesn't Work I understand that one typical solution is to modify the pool configuration to include all three nameservers. However, this doesn't work in my case because: - I don't want to change the default pool settings since most zones should only use their local regional nameserver - Modifying the pool in Region 1 would affect *all zones* created there, not just the ones I want to replicate My Questions I'm trying to understand the design decisions here, and I'd really appreciate any insights: 1. What's the reasoning behind blocking NS record updates on root zones? I'm not seeing the security or technical concern that this protects against. 2. Is this restriction something that could potentially be made configurable via policy? For example, allowing zone owners to update their own NS records when they have legitimate multi-region or high-availability requirements. 3. Are there any alternative approaches I should be considering for this multi-region PRIMARY/SECONDARY setup that I might be missing? 4. Would the community be open to discussing a potential enhancement that makes this more flexible? I understand if there are concerns, but I'd love to understand them better. Use Case Context This is a fairly standard DNS architecture for high availability and geographic distribution. Many organizations need to run primary and secondary nameservers across different regions or datacenters, and being able to configure the NS records accordingly seems like it should be a supported use case. Thank you very much for taking the time to read this, and I appreciate any guidance or feedback the community can provide! Best regards, Sylvain.

2 2

vmware_nsxv
by Parsa Aminian 07 Jan '26

07 Jan '26

hello on kolla ansible neutron_plugin_agent: "vmware_nsxv" is showing this error : fatal: [control01]: FAILED! => {"changed": true, "msg": "Container exited with non-zero return code 1", "rc": 1, "stderr": "+ sudo -E kolla_set_configs\n2026-01-06 12:53:23.464 INFO Loading config file at /var/lib/kolla/config_files/config.json\n2026-01-06 12:53:23.465 INFO Validating config file\n2026-01-06 12:53:23.465 INFO Kolla config strategy set to: COPY_ALWAYS\n2026-01-06 12:53:23.476 INFO Copying service configuration files\n2026-01-06 12:53:23.476 INFO Copying /var/lib/kolla/config_files/neutron.conf to /etc/neutron/neutron.conf\n2026-01-06 12:53:23.489 INFO Setting permission for /etc/neutron/neutron.conf\n2026-01-06 12:53:23.501 INFO Copying /var/lib/kolla/config_files/nsx.ini to /etc/neutron/plugins/vmware/nsx.ini\n2026-01-06 12:53:23.501 INFO Setting permission for /etc/neutron/plugins/vmware/nsx.ini\n2026-01-06 12:53:23.502 INFO Copying /var/lib/kolla/config_files/ml2_conf.ini to /etc/neutron/plugins/ml2/ml2_conf.ini\n2026-01-06 12:53:23.512 INFO Setting permission for /etc/neutron/plugins/ml2/ml2_conf.ini\n2026-01-06 12:53:23.523 INFO Copying /var/lib/kolla/config_files/id_rsa to /var/lib/neutron/.ssh/id_rsa\n2026-01-06 12:53:23.524 INFO Setting permission for /var/lib/neutron/.ssh/id_rsa\n2026-01-06 12:53:23.524 INFO Writing out command to execute\n++ cat /run_command\n+ CMD='neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/vmware/nsx.ini '\n+ ARGS=\n+ sudo kolla_copy_cacerts\n+ sudo kolla_install_projects\n+ [[ ! -n '' ]]\n+ . kolla_extend_start\n++ [[ ! -d /var/log/kolla/neutron ]]\n++ mkdir -p /var/log/kolla/neutron\n+++ stat -c %a /var/log/kolla/neutron\n++ [[ 2755 != \\7\\5\\5 ]]\n++ chmod 755 /var/log/kolla/neutron\n++ [[ ! ubuntu =~ centos|rocky ]]\n++ /usr/bin/update-alternatives --display iptables\n++ KOLLA_LEGACY_IPTABLES=false\n++ [[ false == \\t\\r\\u\\e ]]\n++ sudo /usr/bin/update-alternatives --auto iptables\n++ sudo /usr/bin/update-alternatives --auto ip6tables\n++ . /usr/local/bin/kolla_neutron_extend_start\n+++ [[ -n 0 ]]\n+++ neutron-db-manage --subproject neutron upgrade head\nINFO [alembic.runtime.migration] Context impl MySQLImpl.\nINFO [alembic.runtime.migration] Will assume non-transactional DDL.\nINFO [alembic.runtime.migration] Context impl MySQLImpl.\nINFO [alembic.runtime.migration] Will assume non-transactional DDL.\nINFO [alembic.runtime.migration] Running upgrade -> kilo\nINFO [alembic.runtime.migration] Running upgrade kilo -> 354db87e3225\nINFO [alembic.runtime.migration] Running upgrade 354db87e3225 -> 599c6a226151\nINFO [alembic.runtime.migration] Running upgrade 599c6a226151 -> 52c5312f6baf\nINFO [alembic.runtime.migration] Running upgrade 52c5312f6baf -> 313373c0ffee\nINFO [alembic.runtime.migration] Running upgrade 313373c0ffee -> 8675309a5c4f\nINFO [alembic.runtime.migration] Running upgrade 8675309a5c4f -> 45f955889773\nINFO [alembic.runtime.migration] Running upgrade 45f955889773 -> 26c371498592\nINFO [alembic.runtime.migration] Running upgrade 26c371498592 -> 1c844d1677f7\nINFO [alembic.runtime.migration] Running upgrade 1c844d1677f7 -> 1b4c6e320f79\nINFO [alembic.runtime.migration] Running upgrade 1b4c6e320f79 -> 48153cb5f051\nINFO [alembic.runtime.migration] Running upgrade 48153cb5f051 -> 9859ac9c136\nINFO [alembic.runtime.migration] Running upgrade 9859ac9c136 -> 34af2b5c5a59\nINFO [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d\nINFO [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a\nINFO [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25\nINFO [alembic.runtime.migration] Running upgrade 32e5974ada25 -> ec7fcfbf72ee\nINFO [alembic.runtime.migration] Running upgrade ec7fcfbf72ee -> dce3ec7a25c9\nINFO [alembic.runtime.migration] Running upgrade dce3ec7a25c9 -> c3a73f615e4\nINFO [alembic.runtime.migration] Running upgrade c3a73f615e4 -> 659bf3d90664\nINFO [alembic.runtime.migration] Running upgrade 659bf3d90664 -> 1df244e556f5\nINFO [alembic.runtime.migration] Running upgrade 1df244e556f5 -> 19f26505c74f\nINFO [alembic.runtime.migration] Running upgrade 19f26505c74f -> 15be73214821\nINFO [alembic.runtime.migration] Running upgrade 15be73214821 -> b4caf27aae4\nINFO [alembic.runtime.migration] Running upgrade b4caf27aae4 -> 15e43b934f81\nINFO [alembic.runtime.migration] Running upgrade 15e43b934f81 -> 31ed664953e6\nINFO [alembic.runtime.migration] Running upgrade 31ed664953e6 -> 2f9e956e7532\nINFO [alembic.runtime.migration] Running upgrade 2f9e956e7532 -> 3894bccad37f\nINFO [alembic.runtime.migration] Running upgrade 3894bccad37f -> 0e66c5227a8a\nINFO [alembic.runtime.migration] Running upgrade 0e66c5227a8a -> 45f8dd33480b\nINFO [alembic.runtime.migration] Running upgrade 45f8dd33480b -> 5abc0278ca73\nINFO [alembic.runtime.migration] Running upgrade 5abc0278ca73 -> d3435b514502\nINFO [alembic.runtime.migration] Running upgrade d3435b514502 -> 30107ab6a3ee\nINFO [alembic.runtime.migration] Running upgrade 30107ab6a3ee -> c415aab1c048\nINFO [alembic.runtime.migration] Running upgrade c415aab1c048 -> a963b38d82f4\nINFO [alembic.runtime.migration] Running upgrade kilo -> 30018084ec99\nINFO [alembic.runtime.migration] Running upgrade 30018084ec99 -> 4ffceebfada\nINFO [alembic.runtime.migration] Running upgrade 4ffceebfada -> 5498d17be016\nINFO [alembic.runtime.migration] Running upgrade 5498d17be016 -> 2a16083502f3\nINFO [alembic.runtime.migration] Running upgrade 2a16083502f3 -> 2e5352a0ad4d\nINFO [alembic.runtime.migration] Running upgrade 2e5352a0ad4d -> 11926bcfe72d\nINFO [alembic.runtime.migration] Running upgrade 11926bcfe72d -> 4af11ca47297\nINFO [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c\nINFO [alembic.runtime.migration] Running upgrade 1b294093239c -> 8a6d8bdae39\nINFO [alembic.runtime.migration] Running upgrade 8a6d8bdae39 -> 2b4c2465d44b\nINFO [alembic.runtime.migration] Running upgrade 2b4c2465d44b -> e3278ee65050\nINFO [alembic.runtime.migration] Running upgrade e3278ee65050 -> c6c112992c9\nINFO [alembic.runtime.migration] Running upgrade c6c112992c9 -> 5ffceebfada\nINFO [alembic.runtime.migration] Running upgrade 5ffceebfada -> 4ffceebfcdc\nINFO [alembic.runtime.migration] Running upgrade 4ffceebfcdc -> 7bbb25278f53\nINFO [alembic.runtime.migration] Running upgrade 7bbb25278f53 -> 89ab9a816d70\nINFO [alembic.runtime.migration] Running upgrade 89ab9a816d70 -> c879c5e1ee90\nINFO [alembic.runtime.migration] Running upgrade c879c5e1ee90 -> 8fd3918ef6f4\nINFO [alembic.runtime.migration] Running upgrade 8fd3918ef6f4 -> 4bcd4df1f426\nINFO [alembic.runtime.migration] Running upgrade 4bcd4df1f426 -> b67e765a3524\nINFO [alembic.runtime.migration] Running upgrade a963b38d82f4 -> 3d0e74aa7d37\nINFO [alembic.runtime.migration] Running upgrade 3d0e74aa7d37 -> 030a959ceafa\nINFO [alembic.runtime.migration] Running upgrade 030a959ceafa -> a5648cfeeadf\nINFO [alembic.runtime.migration] Running upgrade a5648cfeeadf -> 0f5bef0f87d4\nINFO [alembic.runtime.migration] Running upgrade 0f5bef0f87d4 -> 67daae611b6e\nINFO [alembic.runtime.migration] Running upgrade b67e765a3524 -> a84ccf28f06a\nINFO [alembic.runtime.migration] Running upgrade a84ccf28f06a -> 7d9d8eeec6ad\nINFO [alembic.runtime.migration] Running upgrade 67daae611b6e -> 6b461a21bcfc\nINFO [alembic.runtime.migration] Running upgrade 6b461a21bcfc -> 5cd92597d11d\nINFO [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70\nINFO [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c\nINFO [alembic.runtime.migration] Running upgrade a9c43481023c -> 804a3c76314c\nINFO [alembic.runtime.migration] Running upgrade 804a3c76314c -> 2b42d90729da\nINFO [alembic.runtime.migration] Running upgrade 2b42d90729da -> 62c781cb6192\nINFO [alembic.runtime.migration] Running upgrade 62c781cb6192 -> c8c222d42aa9\nINFO [alembic.runtime.migration] Running upgrade c8c222d42aa9 -> 349b6fd605a6\nINFO [alembic.runtime.migration] Running upgrade 349b6fd605a6 -> 7d32f979895f\nINFO [alembic.runtime.migration] Running upgrade 7d32f979895f -> 594422d373ee\nINFO [alembic.runtime.migration] Running upgrade 594422d373ee -> 61663558142c\nINFO [alembic.runtime.migration] Running upgrade 61663558142c -> 867d39095bf4, port forwarding\nINFO [alembic.runtime.migration] Running upgrade 867d39095bf4 -> d72db3e25539, modify uniq port forwarding\nINFO [alembic.runtime.migration] Running upgrade d72db3e25539 -> cada2437bf41\nINFO [alembic.runtime.migration] Running upgrade cada2437bf41 -> 195176fb410d, router gateway IP QoS\nINFO [alembic.runtime.migration] Running upgrade 195176fb410d -> fb0167bd9639\nINFO [alembic.runtime.migration] Running upgrade fb0167bd9639 -> 0ff9e3881597\nINFO [alembic.runtime.migration] Running upgrade 0ff9e3881597 -> 9bfad3f1e780\nINFO [alembic.runtime.migration] Running upgrade 9bfad3f1e780 -> 63fd95af7dcd\nINFO [alembic.runtime.migration] Running upgrade 63fd95af7dcd -> c613d0b82681\nINFO [alembic.runtime.migration] Running upgrade c613d0b82681 -> c3e9d13c4367\nINFO [alembic.runtime.migration] Running upgrade c3e9d13c4367 -> 86274d77933e\nINFO [alembic.runtime.migration] Running upgrade 86274d77933e -> f4b9654dd40c\nINFO [alembic.runtime.migration] Running upgrade f4b9654dd40c -> a010322604bc\nINFO [alembic.runtime.migration] Running upgrade a010322604bc -> 263d454a9655\nINFO [alembic.runtime.migration] Running upgrade 263d454a9655 -> Ibac91d24da2\nINFO [alembic.runtime.migration] Running upgrade Ibac91d24da2 -> 2217c4222de6\nINFO [alembic.runtime.migration] Running upgrade 2217c4222de6 -> 18a7e90ae768\nINFO [alembic.runtime.migration] Running upgrade 18a7e90ae768 -> e4e236b0e1ff\nINFO [alembic.runtime.migration] Running upgrade e4e236b0e1ff -> e88badaa9591\nINFO [alembic.runtime.migration] Running upgrade e88badaa9591 -> d8bdf05313f4\nINFO [alembic.runtime.migration] Running upgrade d8bdf05313f4 -> dfe425060830\nINFO [alembic.runtime.migration] Running upgrade dfe425060830 -> fd6107509ccd\nINFO [alembic.runtime.migration] Running upgrade fd6107509ccd -> 1ea5dab0897a\nINFO [alembic.runtime.migration] Running upgrade 1ea5dab0897a -> 49d8622c5221\nINFO [alembic.runtime.migration] Running upgrade 49d8622c5221 -> I38991de2b4\nINFO [alembic.runtime.migration] Running upgrade I38991de2b4 -> 532aa95457e2\nINFO [alembic.runtime.migration] Running upgrade 532aa95457e2 -> f010820fc498\nINFO [alembic.runtime.migration] Running upgrade f010820fc498 -> a964d94b4677\nINFO [alembic.runtime.migration] Running upgrade a964d94b4677 -> 26d1e9f5c766\nINFO [alembic.runtime.migration] Running upgrade 26d1e9f5c766 -> 1e0744e4ffea\nINFO [alembic.runtime.migration] Running upgrade 1e0744e4ffea -> 6135a7bd4425\nINFO [alembic.runtime.migration] Running upgrade 6135a7bd4425 -> 8df53b0d2c0e\nINFO [alembic.runtime.migration] Running upgrade 8df53b0d2c0e -> 1bb3393de75d, add qos policy rule Packet Rate Limit\nINFO [alembic.runtime.migration] Running upgrade 1bb3393de75d -> c181bb1d89e4\nINFO [alembic.runtime.migration] Running upgrade c181bb1d89e4 -> ba859d649675\nINFO [alembic.runtime.migration] Running upgrade ba859d649675 -> e981acd076d3\nINFO [alembic.runtime.migration] Running upgrade e981acd076d3 -> 76df7844a8c6, add Local IP tables\nINFO [alembic.runtime.migration] Running upgrade 76df7844a8c6 -> 1ffef8d6f371, migrate RBAC registers from \"target_tenant\" to \"target_project\"\nINFO [alembic.runtime.migration] Running upgrade 1ffef8d6f371 -> 8160f7a9cebb, drop portbindingports table\nINFO [alembic.runtime.migration] Running upgrade 8160f7a9cebb -> cd9ef14ccf87\nINFO [alembic.runtime.migration] Running upgrade cd9ef14ccf87 -> 34cf8b009713\nINFO [alembic.runtime.migration] Running upgrade 34cf8b009713 -> I43e0b669096\nINFO [alembic.runtime.migration] Running upgrade I43e0b669096 -> 4e6e655746f6\nINFO [alembic.runtime.migration] Running upgrade 4e6e655746f6 -> 659cbedf30a1\nINFO [alembic.runtime.migration] Running upgrade 659cbedf30a1 -> 21ff98fabab1\nINFO [alembic.runtime.migration] Running upgrade 21ff98fabab1 -> 5881373af7f5\nINFO [alembic.runtime.migration] Running upgrade 5881373af7f5 -> fc153938cdc1\nINFO [alembic.runtime.migration] Running upgrade fc153938cdc1 -> 93f394357a27\nINFO [alembic.runtime.migration] Running upgrade 93f394357a27 -> 6f1145bff34c\nINFO [alembic.runtime.migration] Running upgrade 6f1145bff34c -> 682c319773d7\nINFO [alembic.runtime.migration] Running upgrade 682c319773d7 -> 0aefee21cd87\nINFO [alembic.runtime.migration] Running upgrade 0aefee21cd87 -> b1199a3adbef\nINFO [alembic.runtime.migration] Running upgrade b1199a3adbef -> c33da356b165\nINFO [alembic.runtime.migration] Running upgrade c33da356b165 -> 89c58a70ceba\nINFO [alembic.runtime.migration] Running upgrade 89c58a70ceba -> 054e34dbe6b4\nINFO [alembic.runtime.migration] Running upgrade 054e34dbe6b4 -> 0e6eff810791\nINFO [alembic.runtime.migration] Running upgrade 0e6eff810791 -> 175fa80908e1\nINFO [alembic.runtime.migration] Running upgrade 175fa80908e1 -> 5bcb7b31ec7d\nINFO [alembic.runtime.migration] Running upgrade 5bcb7b31ec7d -> ad80a9f07c5c\nINFO [alembic.runtime.migration] Running upgrade 7d9d8eeec6ad -> a8b517cff8ab\nINFO [alembic.runtime.migration] Running upgrade a8b517cff8ab -> 3b935b28e7a0\nINFO [alembic.runtime.migration] Running upgrade 3b935b28e7a0 -> b12a3ef66e62\nINFO [alembic.runtime.migration] Running upgrade b12a3ef66e62 -> 97c25b0d2353\nINFO [alembic.runtime.migration] Running upgrade 97c25b0d2353 -> 2e0d7a8a1586\nINFO [alembic.runtime.migration] Running upgrade 2e0d7a8a1586 -> 5c85685d616d\n+++ [[ -n 0 ]]\n+++ for service in ${NEUTRON_BOOTSTRAP_SERVICES}\n+++ neutron-db-manage --subproject vmware-nsx upgrade head\nargument --subproject: Invalid String(choices=['networking-sfc', 'neutron-dynamic-routing', 'neutron-fwaas', 'neutron-vpnaas', 'neutron', 'tap-as-a-service']) value: vmware-nsx\n", "stderr_lines": ["+ sudo -E kolla_set_configs", "2026-01-06 12:53:23.464 INFO Loading config file at /var/lib/kolla/config_files/config.json", "2026-01-06 12:53:23.465 INFO Validating config file", "2026-01-06 12:53:23.465 INFO Kolla config strategy set to: COPY_ALWAYS", "2026-01-06 12:53:23.476 INFO Copying service configuration files", "2026-01-06 12:53:23.476 INFO Copying /var/lib/kolla/config_files/neutron.conf to /etc/neutron/neutron.conf", "2026-01-06 12:53:23.489 INFO Setting permission for /etc/neutron/neutron.conf", "2026-01-06 12:53:23.501 INFO Copying /var/lib/kolla/config_files/nsx.ini to /etc/neutron/plugins/vmware/nsx.ini", "2026-01-06 12:53:23.501 INFO Setting permission for /etc/neutron/plugins/vmware/nsx.ini", "2026-01-06 12:53:23.502 INFO Copying /var/lib/kolla/config_files/ml2_conf.ini to /etc/neutron/plugins/ml2/ml2_conf.ini", "2026-01-06 12:53:23.512 INFO Setting permission for /etc/neutron/plugins/ml2/ml2_conf.ini", "2026-01-06 12:53:23.523 INFO Copying /var/lib/kolla/config_files/id_rsa to /var/lib/neutron/.ssh/id_rsa", "2026-01-06 12:53:23.524 INFO Setting permission for /var/lib/neutron/.ssh/id_rsa", "2026-01-06 12:53:23.524 INFO Writing out command to execute", "++ cat /run_command", "+ CMD='neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/vmware/nsx.ini '", "+ ARGS=", "+ sudo kolla_copy_cacerts", "+ sudo kolla_install_projects", "+ [[ ! -n '' ]]", "+ . kolla_extend_start", "++ [[ ! -d /var/log/kolla/neutron ]]", "++ mkdir -p /var/log/kolla/neutron", "+++ stat -c %a /var/log/kolla/neutron", "++ [[ 2755 != \\7\\5\\5 ]]", "++ chmod 755 /var/log/kolla/neutron", "++ [[ ! ubuntu =~ centos|rocky ]]", "++ /usr/bin/update-alternatives --display iptables", "++ KOLLA_LEGACY_IPTABLES=false", "++ [[ false == \\t\\r\\u\\e ]]", "++ sudo /usr/bin/update-alternatives --auto iptables", "++ sudo /usr/bin/update-alternatives --auto ip6tables", "++ . /usr/local/bin/kolla_neutron_extend_start", "+++ [[ -n 0 ]]", "+++ neutron-db-manage --subproject neutron upgrade head", "INFO [alembic.runtime.migration] Context impl MySQLImpl.", "INFO [alembic.runtime.migration] Will assume non-transactional DDL.", "INFO [alembic.runtime.migration] Context impl MySQLImpl.", "INFO [alembic.runtime.migration] Will assume non-transactional DDL.", "INFO [alembic.runtime.migration] Running upgrade -> kilo", "INFO [alembic.runtime.migration] Running upgrade kilo -> 354db87e3225", "INFO [alembic.runtime.migration] Running upgrade 354db87e3225 -> 599c6a226151", "INFO [alembic.runtime.migration] Running upgrade 599c6a226151 -> 52c5312f6baf", "INFO [alembic.runtime.migration] Running upgrade 52c5312f6baf -> 313373c0ffee", "INFO [alembic.runtime.migration] Running upgrade 313373c0ffee -> 8675309a5c4f", "INFO [alembic.runtime.migration] Running upgrade 8675309a5c4f -> 45f955889773", "INFO [alembic.runtime.migration] Running upgrade 45f955889773 -> 26c371498592", "INFO [alembic.runtime.migration] Running upgrade 26c371498592 -> 1c844d1677f7", "INFO [alembic.runtime.migration] Running upgrade 1c844d1677f7 -> 1b4c6e320f79", "INFO [alembic.runtime.migration] Running upgrade 1b4c6e320f79 -> 48153cb5f051", "INFO [alembic.runtime.migration] Running upgrade 48153cb5f051 -> 9859ac9c136", "INFO [alembic.runtime.migration] Running upgrade 9859ac9c136 -> 34af2b5c5a59", "INFO [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d", "INFO [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a", "INFO [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25", "INFO [alembic.runtime.migration] Running upgrade 32e5974ada25 -> ec7fcfbf72ee", "INFO [alembic.runtime.migration] Running upgrade ec7fcfbf72ee -> dce3ec7a25c9", "INFO [alembic.runtime.migration] Running upgrade dce3ec7a25c9 -> c3a73f615e4", "INFO [alembic.runtime.migration] Running upgrade c3a73f615e4 -> 659bf3d90664", "INFO [alembic.runtime.migration] Running upgrade 659bf3d90664 -> 1df244e556f5", "INFO [alembic.runtime.migration] Running upgrade 1df244e556f5 -> 19f26505c74f", "INFO [alembic.runtime.migration] Running upgrade 19f26505c74f -> 15be73214821", "INFO [alembic.runtime.migration] Running upgrade 15be73214821 -> b4caf27aae4", "INFO [alembic.runtime.migration] Running upgrade b4caf27aae4 -> 15e43b934f81", "INFO [alembic.runtime.migration] Running upgrade 15e43b934f81 -> 31ed664953e6", "INFO [alembic.runtime.migration] Running upgrade 31ed664953e6 -> 2f9e956e7532", "INFO [alembic.runtime.migration] Running upgrade 2f9e956e7532 -> 3894bccad37f", "INFO [alembic.runtime.migration] Running upgrade 3894bccad37f -> 0e66c5227a8a", "INFO [alembic.runtime.migration] Running upgrade 0e66c5227a8a -> 45f8dd33480b", "INFO [alembic.runtime.migration] Running upgrade 45f8dd33480b -> 5abc0278ca73", "INFO [alembic.runtime.migration] Running upgrade 5abc0278ca73 -> d3435b514502", "INFO [alembic.runtime.migration] Running upgrade d3435b514502 -> 30107ab6a3ee", "INFO [alembic.runtime.migration] Running upgrade 30107ab6a3ee -> c415aab1c048", "INFO [alembic.runtime.migration] Running upgrade c415aab1c048 -> a963b38d82f4", "INFO [alembic.runtime.migration] Running upgrade kilo -> 30018084ec99", "INFO [alembic.runtime.migration] Running upgrade 30018084ec99 -> 4ffceebfada", "INFO [alembic.runtime.migration] Running upgrade 4ffceebfada -> 5498d17be016", "INFO [alembic.runtime.migration] Running upgrade 5498d17be016 -> 2a16083502f3", "INFO [alembic.runtime.migration] Running upgrade 2a16083502f3 -> 2e5352a0ad4d", "INFO [alembic.runtime.migration] Running upgrade 2e5352a0ad4d -> 11926bcfe72d", "INFO [alembic.runtime.migration] Running upgrade 11926bcfe72d -> 4af11ca47297", "INFO [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c", "INFO [alembic.runtime.migration] Running upgrade 1b294093239c -> 8a6d8bdae39", "INFO [alembic.runtime.migration] Running upgrade 8a6d8bdae39 -> 2b4c2465d44b", "INFO [alembic.runtime.migration] Running upgrade 2b4c2465d44b -> e3278ee65050", "INFO [alembic.runtime.migration] Running upgrade e3278ee65050 -> c6c112992c9", "INFO [alembic.runtime.migration] Running upgrade c6c112992c9 -> 5ffceebfada", "INFO [alembic.runtime.migration] Running upgrade 5ffceebfada -> 4ffceebfcdc", "INFO [alembic.runtime.migration] Running upgrade 4ffceebfcdc -> 7bbb25278f53", "INFO [alembic.runtime.migration] Running upgrade 7bbb25278f53 -> 89ab9a816d70", "INFO [alembic.runtime.migration] Running upgrade 89ab9a816d70 -> c879c5e1ee90", "INFO [alembic.runtime.migration] Running upgrade c879c5e1ee90 -> 8fd3918ef6f4", "INFO [alembic.runtime.migration] Running upgrade 8fd3918ef6f4 -> 4bcd4df1f426", "INFO [alembic.runtime.migration] Running upgrade 4bcd4df1f426 -> b67e765a3524", "INFO [alembic.runtime.migration] Running upgrade a963b38d82f4 -> 3d0e74aa7d37", "INFO [alembic.runtime.migration] Running upgrade 3d0e74aa7d37 -> 030a959ceafa", "INFO [alembic.runtime.migration] Running upgrade 030a959ceafa -> a5648cfeeadf", "INFO [alembic.runtime.migration] Running upgrade a5648cfeeadf -> 0f5bef0f87d4", "INFO [alembic.runtime.migration] Running upgrade 0f5bef0f87d4 -> 67daae611b6e", "INFO [alembic.runtime.migration] Running upgrade b67e765a3524 -> a84ccf28f06a", "INFO [alembic.runtime.migration] Running upgrade a84ccf28f06a -> 7d9d8eeec6ad", "INFO [alembic.runtime.migration] Running upgrade 67daae611b6e -> 6b461a21bcfc", "INFO [alembic.runtime.migration] Running upgrade 6b461a21bcfc -> 5cd92597d11d", "INFO [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70", "INFO [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c", "INFO [alembic.runtime.migration] Running upgrade a9c43481023c -> 804a3c76314c", "INFO [alembic.runtime.migration] Running upgrade 804a3c76314c -> 2b42d90729da", "INFO [alembic.runtime.migration] Running upgrade 2b42d90729da -> 62c781cb6192", "INFO [alembic.runtime.migration] Running upgrade 62c781cb6192 -> c8c222d42aa9", "INFO [alembic.runtime.migration] Running upgrade c8c222d42aa9 -> 349b6fd605a6", "INFO [alembic.runtime.migration] Running upgrade 349b6fd605a6 -> 7d32f979895f", "INFO [alembic.runtime.migration] Running upgrade 7d32f979895f -> 594422d373ee", "INFO [alembic.runtime.migration] Running upgrade 594422d373ee -> 61663558142c", "INFO [alembic.runtime.migration] Running upgrade 61663558142c -> 867d39095bf4, port forwarding", "INFO [alembic.runtime.migration] Running upgrade 867d39095bf4 -> d72db3e25539, modify uniq port forwarding", "INFO [alembic.runtime.migration] Running upgrade d72db3e25539 -> cada2437bf41", "INFO [alembic.runtime.migration] Running upgrade cada2437bf41 -> 195176fb410d, router gateway IP QoS", "INFO [alembic.runtime.migration] Running upgrade 195176fb410d -> fb0167bd9639", "INFO [alembic.runtime.migration] Running upgrade fb0167bd9639 -> 0ff9e3881597", "INFO [alembic.runtime.migration] Running upgrade 0ff9e3881597 -> 9bfad3f1e780", "INFO [alembic.runtime.migration] Running upgrade 9bfad3f1e780 -> 63fd95af7dcd", "INFO [alembic.runtime.migration] Running upgrade 63fd95af7dcd -> c613d0b82681", "INFO [alembic.runtime.migration] Running upgrade c613d0b82681 -> c3e9d13c4367", "INFO [alembic.runtime.migration] Running upgrade c3e9d13c4367 -> 86274d77933e", "INFO [alembic.runtime.migration] Running upgrade 86274d77933e -> f4b9654dd40c", "INFO [alembic.runtime.migration] Running upgrade f4b9654dd40c -> a010322604bc", "INFO [alembic.runtime.migration] Running upgrade a010322604bc -> 263d454a9655", "INFO [alembic.runtime.migration] Running upgrade 263d454a9655 -> Ibac91d24da2", "INFO [alembic.runtime.migration] Running upgrade Ibac91d24da2 -> 2217c4222de6", "INFO [alembic.runtime.migration] Running upgrade 2217c4222de6 -> 18a7e90ae768", "INFO [alembic.runtime.migration] Running upgrade 18a7e90ae768 -> e4e236b0e1ff", "INFO [alembic.runtime.migration] Running upgrade e4e236b0e1ff -> e88badaa9591", "INFO [alembic.runtime.migration] Running upgrade e88badaa9591 -> d8bdf05313f4", "INFO [alembic.runtime.migration] Running upgrade d8bdf05313f4 -> dfe425060830", "INFO [alembic.runtime.migration] Running upgrade dfe425060830 -> fd6107509ccd", "INFO [alembic.runtime.migration] Running upgrade fd6107509ccd -> 1ea5dab0897a", "INFO [alembic.runtime.migration] Running upgrade 1ea5dab0897a -> 49d8622c5221", "INFO [alembic.runtime.migration] Running upgrade 49d8622c5221 -> I38991de2b4", "INFO [alembic.runtime.migration] Running upgrade I38991de2b4 -> 532aa95457e2", "INFO [alembic.runtime.migration] Running upgrade 532aa95457e2 -> f010820fc498", "INFO [alembic.runtime.migration] Running upgrade f010820fc498 -> a964d94b4677", "INFO [alembic.runtime.migration] Running upgrade a964d94b4677 -> 26d1e9f5c766", "INFO [alembic.runtime.migration] Running upgrade 26d1e9f5c766 -> 1e0744e4ffea", "INFO [alembic.runtime.migration] Running upgrade 1e0744e4ffea -> 6135a7bd4425", "INFO [alembic.runtime.migration] Running upgrade 6135a7bd4425 -> 8df53b0d2c0e", "INFO [alembic.runtime.migration] Running upgrade 8df53b0d2c0e -> 1bb3393de75d, add qos policy rule Packet Rate Limit", "INFO [alembic.runtime.migration] Running upgrade 1bb3393de75d -> c181bb1d89e4", "INFO [alembic.runtime.migration] Running upgrade c181bb1d89e4 -> ba859d649675", "INFO [alembic.runtime.migration] Running upgrade ba859d649675 -> e981acd076d3", "INFO [alembic.runtime.migration] Running upgrade e981acd076d3 -> 76df7844a8c6, add Local IP tables", "INFO [alembic.runtime.migration] Running upgrade 76df7844a8c6 -> 1ffef8d6f371, migrate RBAC registers from \"target_tenant\" to \"target_project\"", "INFO [alembic.runtime.migration] Running upgrade 1ffef8d6f371 -> 8160f7a9cebb, drop portbindingports table", "INFO [alembic.runtime.migration] Running upgrade 8160f7a9cebb -> cd9ef14ccf87", "INFO [alembic.runtime.migration] Running upgrade cd9ef14ccf87 -> 34cf8b009713", "INFO [alembic.runtime.migration] Running upgrade 34cf8b009713 -> I43e0b669096", "INFO [alembic.runtime.migration] Running upgrade I43e0b669096 -> 4e6e655746f6", "INFO [alembic.runtime.migration] Running upgrade 4e6e655746f6 -> 659cbedf30a1", "INFO [alembic.runtime.migration] Running upgrade 659cbedf30a1 -> 21ff98fabab1", "INFO [alembic.runtime.migration] Running upgrade 21ff98fabab1 -> 5881373af7f5", "INFO [alembic.runtime.migration] Running upgrade 5881373af7f5 -> fc153938cdc1", "INFO [alembic.runtime.migration] Running upgrade fc153938cdc1 -> 93f394357a27", "INFO [alembic.runtime.migration] Running upgrade 93f394357a27 -> 6f1145bff34c", "INFO [alembic.runtime.migration] Running upgrade 6f1145bff34c -> 682c319773d7", "INFO [alembic.runtime.migration] Running upgrade 682c319773d7 -> 0aefee21cd87", "INFO [alembic.runtime.migration] Running upgrade 0aefee21cd87 -> b1199a3adbef", "INFO [alembic.runtime.migration] Running upgrade b1199a3adbef -> c33da356b165", "INFO [alembic.runtime.migration] Running upgrade c33da356b165 -> 89c58a70ceba", "INFO [alembic.runtime.migration] Running upgrade 89c58a70ceba -> 054e34dbe6b4", "INFO [alembic.runtime.migration] Running upgrade 054e34dbe6b4 -> 0e6eff810791", "INFO [alembic.runtime.migration] Running upgrade 0e6eff810791 -> 175fa80908e1", "INFO [alembic.runtime.migration] Running upgrade 175fa80908e1 -> 5bcb7b31ec7d", "INFO [alembic.runtime.migration] Running upgrade 5bcb7b31ec7d -> ad80a9f07c5c", "INFO [alembic.runtime.migration] Running upgrade 7d9d8eeec6ad -> a8b517cff8ab", "INFO [alembic.runtime.migration] Running upgrade a8b517cff8ab -> 3b935b28e7a0", "INFO [alembic.runtime.migration] Running upgrade 3b935b28e7a0 -> b12a3ef66e62", "INFO [alembic.runtime.migration] Running upgrade b12a3ef66e62 -> 97c25b0d2353", "INFO [alembic.runtime.migration] Running upgrade 97c25b0d2353 -> 2e0d7a8a1586", "INFO [alembic.runtime.migration] Running upgrade 2e0d7a8a1586 -> 5c85685d616d", "+++ [[ -n 0 ]]", "+++ for service in ${NEUTRON_BOOTSTRAP_SERVICES}", "+++ neutron-db-manage --subproject vmware-nsx upgrade head", "argument --subproject: Invalid String(choices=['networking-sfc', 'neutron-dynamic-routing', 'neutron-fwaas', 'neutron-vpnaas', 'neutron', 'tap-as-a-service']) value: vmware-nsx"], "stdout": "iptables - auto mode\n link best version is /usr/sbin/iptables-nft\n link currently points to /usr/sbin/iptables-nft\n link iptables is /usr/sbin/iptables\n slave iptables-restore is /usr/sbin/iptables-restore\n slave iptables-save is /usr/sbin/iptables-save\n/usr/sbin/iptables-legacy - priority 10\n slave iptables-restore: /usr/sbin/iptables-legacy-restore\n slave iptables-save: /usr/sbin/iptables-legacy-save\n/usr/sbin/iptables-nft - priority 20\n slave iptables-restore: /usr/sbin/iptables-nft-restore\n slave iptables-save: /usr/sbin/iptables-nft-save\nRunning upgrade for neutron ...\nOK\n", "stdout_lines": ["iptables - auto mode", " link best version is /usr/sbin/iptables-nft", " link currently points to /usr/sbin/iptables-nft", " link iptables is /usr/sbin/iptables", " slave iptables-restore is /usr/sbin/iptables-restore", " slave iptables-save is /usr/sbin/iptables-save", "/usr/sbin/iptables-legacy - priority 10", " slave iptables-restore: /usr/sbin/iptables-legacy-restore", " slave iptables-save: /usr/sbin/iptables-legacy-save", "/usr/sbin/iptables-nft - priority 20", " slave iptables-restore: /usr/sbin/iptables-nft-restore", " slave iptables-save: /usr/sbin/iptables-nft-save", "Running upgrade for neutron ...", "OK"]} whats the problem ?

2 3

[tc][all] No weekly IRC meeting on January 6 2026
by Goutham Pacha Ravi 06 Jan '26

06 Jan '26

Hello Stackers, A number of TC members are still out on PTO or observing public holidays in their respective countries, so we’ll be skipping the weekly IRC meeting this week, and the next meeting will be on January 13th 2026 at 1700 UTC in OFTC’s #openstack-tc channel. Thank you, Goutham Pacha Ravi (gouthamr) Openstack TC Chair

1 0

[openstacksdk][keystoneauth1][nova] Memory leak in nova-scheduler, possibly all services?
by Clippinger, Sam 05 Jan '26

05 Jan '26

Hello everyone, We've been seeing steady memory leaks in some of our OpenStack API services. I think I've found the cause but I'd appreciate some feedback. When a service creates a client object (e.g. placementclient) to use another service's API, the call goes through Proxy.request() in the openstacksdk module. That function calls _report_stats()[1] for every request which calls three other functions for statsd, prometheus and influxdb. The prometheus function records the number of requests and response times in a dict using the full URL of the request.[2] This data is recorded unless the _prometheus_counter or _prometheus_histogram member variables are None.[2] Those member variables are set by config.CloudRegion.get_prometheus_counter()[3] and config.CloudRegion.get_prometheus_histogram()[4], which do not check any config settings. It looks like the only way to prevent this behavior is to uninstall the prometheus_client module.[5] Unfortunately prometheus_client is required by oslo.metrics[6] which is required by oslo.messaging[7]. This is causing a lot of extra memory usage in our production environments. It's worst in our build farm where we create over 500K VMs per week. nova-scheduler queries a lot of unique URLs because the Placement API uses the VM's UUID to get allocations[8] and each URL gets its own counter and histogram. Even though we run 6 copies of nova-scheduler (currently Caracal) after a few weeks each copy will reach 8 GB and the worker processes will start getting OOMkilled, putting the new VM in error state. The nova-scheduler parent process keeps running and restarts the workers, which just keep getting killed and VMs can't be reliably scheduled until nova-scheduler is restarted. This feels like two problems to me. We don't need Prometheus metrics from the OpenStack services but there's no way to turn them off. The Session class in keystoneauth1 supports a collect-timing option[9] (default False) but it's only used in one or two places that I can find. The CloudRegion methods get their config from the Adapter[10] class in keystoneauth1, which does not support collect-timing. Should it? Even if we did use these Prometheus metrics, it seems like they shouldn't be allowed to grow unbounded forever. Should the Proxy._report_stats_prometheus() function[2] limit the metrics by age or quantity? Any thoughts? I've patched our services in-house to turn off these metrics (I'll have results in a week). In the meantime, are we doing something wrong or is every OpenStack service always collecting these metrics constantly? -- Sam Clippinger 1: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/prox… 2: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/prox… 3: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/conf… 4: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/conf… 5: https://github.com/openstack/openstacksdk/blob/stable/2025.2/openstack/conf… 6: https://github.com/openstack/oslo.metrics/blob/stable/2025.2/requirements.t… 7: https://github.com/openstack/oslo.messaging/blob/stable/2025.2/requirements… 8: https://docs.openstack.org/api-ref/placement/#list-allocations 9: https://github.com/openstack/keystoneauth/blob/stable/2025.2/keystoneauth1/… 10: https://github.com/openstack/keystoneauth/blob/stable/2025.2/keystoneauth1/… ________________________________ CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be Garmin confidential and/or Garmin legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.

4 4

[puppet-cinder,glance,manila] Resigning from core team
by Alan Bishop 05 Jan '26

05 Jan '26

Hello, Due to a change in my work status, I am no longer able to fulfill my obligations as a core reviewer. With this in mind, I am resigning from the core team on the puppet-cinder, puppet-glance and puppet-manila projects. Alan Bishop

3 2

[release] Release countdown for week R-12, Jan 05 - Jan 09
by Elõd Illés 05 Jan '26

05 Jan '26

Development Focus ----------------- The Gazpacho-2 milestone is *this* week, on January 8th, 2026! 2026.1 Gazpacho-related specs should now be finalized so that teams can move to implementation ASAP. Some teams observe specific deadlines on the second milestone (mostly spec freezes): please refer to https://releases.openstack.org/gazpacho/schedule.html for details. General Information ------------------- Libraries need to be released at least once per milestone period. Today, the release team will propose releases for any library that has not been otherwise released since milestone 1. PTL's and release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well. A +1 would be appreciated, but if we do not hear anything at all by the end of the week, we will assume things are OK to proceed. Remember that non-library deliverables that follow the cycle-with-intermediary release model should have an intermediary release before milestone-2. Those who haven't will be proposed to switch to the cycle-with-rc model, which is more suited to deliverables that are released only once per cycle. This week is also the deadline to freeze the contents of the final release. All new '2026.1 Gazpacho' deliverables need to have a deliverable file in https://opendev.org/openstack/releases/src/branch/master/deliverables/gazpa… and need to have done a release by milestone-2. Changes proposing those deliverables for inclusion in 2026.1 Gazpacho have been posted, please update them with an actual release request before the milestone-2 deadline if you plan on including that deliverable in 2026.1 Gazpacho, or -1 if you need one more cycle to be ready. Upcoming Deadlines & Dates -------------------------- Gazpacho-2 Milestone: January 8th, 2026 Előd Illés irc: elodilles @ #openstack-release

1 0

[Keystone][ldap] Keystone bind request to ldap not able to interpret if the user's password is expired.
by Sharath Ck 05 Jan '26

05 Jan '26

Hi, I have configured Keystone with 389ds as backend. 389ds server is configured with password policy and pwdmustchange is set to true. However on first login attempt from keystone to request token for user in 389ds is resulting in successful bind request. Perhaps, 389ds is providing control ID - control: 2.16.840.1.113730.3.4.4 in the response. But keystone is not able to interpret the control ID and provide a token with full access. Is it not possible for keystone to respond token with minimum scope to only change password? Or any response attribute to specify password is expired and needs to be changed? Kindly help with any configuration that may interpret the password expiry of ldap user and can be used in token response. P.S: configuring user_enabled_attribute to check nsAccountLock will not work as ldap user's password would be expired but not locked. user_enabled_attribute = nsAccountLock user_enabled_mask = 0 user_enabled_invert = true user_enabled_default = true Regards, Sharath

3 2