Hello Bodo, hello Radoslaw, thanks a lot for your help.
My situation is:
In June: 3 test servers. CentOS Stream (8.4).
Victoria + Vpnaas: OK
Wallaby + Vpnaas: NOT OK
In July: 3 new (different) servers: Rocky OS (8.4) (I changed /etc/system.release to install Openstack)
Wallaby + Vpnaas: NOT OK (1), all is working except VPNaaS
Victoria + Vpnaas: NOT OK (2), all is working except VPNaaS (thanks Radoslaw for horizon !!)
Error messages for
1): Neutron-l3-agent logs says: Command: ['ipsec', 'whack', '--status'] Exit code: 33 Stdout: Stderr: whack: Pluto is not running (no "/run/pluto/pluto.ctl »)
2): More complete
2021-07-26 20:24:23.352 34 ERROR neutron.agent.linux.utils [-] Exit code: 33; Cmd: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-a63770d5-edad-425d-8330-7d12c2cbf3e4', '/var/lib/kolla/venv/bin/neutron-vpn-netns-wrapper', '--mount_paths=/etc:/var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/etc,/var/run:/var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/var/run', '--rootwrap_config=/etc/neutron/rootwrap.conf', '--cmd=ipsec,whack,--status']; Stdin: ; Stdout: 2021-07-26 20:24:23.163 39401 INFO neutron.common.config [-] Logging enabled!
2021-07-26 20:24:23.164 39401 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-vpn-netns-wrapper version 17.1.3.dev54
Command: ['mount', '--bind', '/var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/etc', '/etc'] Exit code: 0 Stdout: Stderr: 2021-07-26 20:24:23.186 39401 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/etc has been bind-mounted in /etc
Command: ['mount', '--bind', '/var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/var/run', '/var/run'] Exit code: 0 Stdout: Stderr: 2021-07-26 20:24:23.195 39401 INFO neutron_vpnaas.services.vpn.common.netns_wrapper [-] /var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/var/run has been bind-mounted in /var/run
Command: ['ipsec', 'whack', '--status'] Exit code: 33 Stdout: Stderr: whack: Pluto is not running (no "/run/pluto/pluto.ctl")
So… I tried what you say:
« docker exec -it /bin/bash »
« ps aux |grep pluto » ———> no pluto running
You could also check if the configuration has been created properly:
/var/lib/neutron/ipsec/{router-id}/etc with ipsec.conf and ipsec.secrets
Yes, configuration is created (i think…..)
[neutron@iut1r-srv-ops01-i01 /var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/etc]$ ls -l
total 8
-rw-r--r--. 1 neutron neutron 0 Jul 26 09:49 hosts
-rw-r--r--. 1 neutron neutron 1948 Jul 26 09:49 ipsec.conf
drwxr-xr-x. 11 neutron neutron 135 Jul 26 09:49 ipsec.d
-rw-------. 1 root root 107 Jul 26 09:49 ipsec.secrets
drwxr-xr-x. 3 neutron neutron 19 Jul 26 09:49 pki
-rw-r--r--. 1 neutron neutron 0 Jul 26 09:49 resolv.conf
for exemple, the file /var/lib/neutron/ipsec/a63770d5-edad-425d-8330-7d12c2cbf3e4/etc/ipsec.conf is
# Configuration for 2a1446f5-4618-4927-883a-7a8b702e13c1
config setup
nat_traversal=yes
virtual_private=%v4:10.0.1.0/24,%v4:10.0.2.0/24
conn %default
keylife=60m
keyingtries=%forever
conn 6b588921-a120-402a-bed2-38c55879a432
# NOTE: a default route is required for %defaultroute to work...
leftnexthop=%defaultroute
rightnexthop=%defaultroute
left=172.16.203.154
leftid=172.16.203.154
auto=start
# NOTE:REQUIRED
# [subnet]
leftsubnet=10.0.1.0/24
# [updown]
# What "updown" script to run to adjust routing and/or firewalling when
# the status of the connection changes (default "ipsec _updown").
# "--route yes" allows to specify such routing options as mtu and metric.
leftupdown="ipsec _updown --route yes"
######################
# ipsec_site_connections
######################
# [peer_address]
right=172.16.201.79
# [peer_id]
rightid=172.16.201.79
# [peer_cidrs]
rightsubnets={ 10.0.2.0/24 }
# rightsubnet=networkA/netmaskA, networkB/netmaskB (IKEv2 only)
# [mtu]
mtu=1500
# [dpd_action]
dpdaction=hold
# [dpd_interval]
dpddelay=30
# [dpd_timeout]
dpdtimeout=120
# [auth_mode]
authby=secret
######################
# IKEPolicy params
######################
#ike version
ikev2=never
# [encryption_algorithm]-[auth_algorithm]-[pfs]
ike=aes128-sha1;modp1536
# [lifetime_value]
ikelifetime=3600s
# NOTE: it looks lifetime_units=kilobytes can't be enforced (could be seconds, hours, days...)
##########################
# IPsecPolicys params
##########################
# [transform_protocol]
phase2=esp
# [encryption_algorithm]-[auth_algorithm]-[pfs]
phase2alg=aes128-sha1;modp1536
# [encapsulation_mode]
type=tunnel
# [lifetime_value]
lifetime=3600s
# lifebytes=100000 if lifetime_units=kilobytes (IKEv2 only)
...
I don’t know if you could see something wrong…
but it is really very nice to help me with this problem.
There are no relevant differences in the VPNaaS plugin between Victoria and Wallaby.
Maybe there actually is a difference between Rocky and Stream: the pid directory expected by pluto. At least your error message (no "/run/pluto/pluto.ctl") suggests that.
The libreswan driver in the VPNaaS plugin works under the assumption that the run-files reside in /var/run.
ipsec commands are run via a wrapper to call the actual command inside a namespace, with /etc/ and /var/run/ bind-mounted.
The real paths are /var/lib/neutron/ipsec/{router-id}/... and in there .../etc and ..,/var/run.
In a working setup you should find /var/lib/neutron/ipsec/{router-id}/var/run/pluto/pluto.ctl
So it may be a problem if pluto wants to use /run (not bind-mounted), but the plugin only provides for /var/run (bind-mounted).
Bodo Petermann
SysEleven GmbH
On Mon, Jul 26, 2021 at 10:39 AM Franck VEDEL<franck.vedel@univ-grenoble-alpes.fr> wrote:
Hello.
unfortunately, despite the good functioning of Victoria, the VPNAAS service is not working.
Same error as for wallaby:
Command: ['ipsec', 'whack', '--status'] Exit code: 33 Stdout: Stderr: whack: Pluto is not running (no "/run/pluto/pluto.ctl")
; Stderr:
I think it's my fault. I didn't want to install CentOS Stream (not knowing what happened with this distribution), I put Rocky. This is a big mistake.
I will start all over again, put CentOS Stream (VPNaas worked with Victoria and CentOS Stream in my tests).
Thanks again.
I'm still disgusted with all this wasted time.
Hello Franck,Before you go wrecking your infra - I am pretty sure that Rocky vsStream does not make a difference here.I thought Victoria worked because you said so but it seems it hasalways broken in Kolla Ansible and we have a bug to fix:https://bugs.launchpad.net/kolla-ansible/+bug/1869491VPNaaS is not the most popular enabled option to be honest.Do you remember how you got it working back then?That could help here.-yoctozepto
Franck
Le 25 juil. 2021 à 21:25, Radosław Piliszek <radoslaw.piliszek@gmail.com> a écrit :
On Sun, Jul 25, 2021 at 9:18 PM Franck VEDEL
<franck.vedel@univ-grenoble-alpes.fr> wrote:
Oh !! Thanks a lot, really.
Indeed, I installed kolla-ansible 12.0, install wallaby (it works perfectly… expect Vpnaas), then I changed "wallaby" to « victoria » in globals.yml.
And in Wallaby's notes, there is the sentence:
The Karbor project is no longer maintained and retired since the Wallaby cycle. Its support and roles are also removed since Wallaby cycle.
So, it's not normal that it doesn't work. I understand…. There is a lot of things, it’s not easy to do the right thing the first time.
On the other hand ... and I hope not to abuse, I am not sure I understand "clone https://opendev.org/openstack/kolla-ansible".
Do you have to uninstall kolla-ansible 12 before putting 11?
How do you do "pip install that directory then"? Really sorry for these stupid questions, but I'm afraid to mess things up.
Sure thing.
I meant to use Git.
Try these commands:
git clone --branch stable/victoria \
https://opendev.org/openstack/kolla-ansible
pip install ./kolla-ansible
-yoctozepto
Franck
Le 25 juil. 2021 à 18:00, Radosław Piliszek <radoslaw.piliszek@gmail.com> a écrit :
On Sun, Jul 25, 2021 at 2:52 PM Franck VEDEL
<franck.vedel@univ-grenoble-alpes.fr> wrote:
Hello
Hello Franck,
Having had no help with my Vpnaas (centos wallaby) problem, I came back to Victoria because I know from having tested that Vpnaas works as it should under Victoria.
A few weeks ago, I had the opportunity to use 3 test servers, I had set up Victoria (with Centos and kolla-ansible). No problem, everything was working as I wanted it to.
I have since set up 3 new servers to set up an Openstack for my students.
if i install Wallaby, no Vpnaas, and I need VPNaaS…. So Victoria.
if I install Victoria, and this is the 1st time that this happens to me, horizon does not work. The horizon docker does not start.
The "docker logs horizon" command ends with the following 3 lines:
++ config_karbor_dashboard
++ for file in $ {SITE_PACKAGES} / karbor_dashboard / enabled / _ * [^ __]. py
/ usr / local / bin / kolla_extend_start: line 121: ENABLE_KARBOR: unbound variable
This error suggests you are using Kolla Ansible Wallaby or later to
deploy Victoria.
You probably just set "openstack_release" to "Victoria" without
downgrading Kolla Ansible to a supported version.
There is a reason why "openstack_release" is commented with "Do not
override this unless you know what you are doing.". ;-)
It is only really meant to be used for very specific tasks, not really
meant for regular users.
Please have a look at
https://docs.openstack.org/releasenotes/kolla-ansible/victoria.html
The latest release for Victoria is 11.0.0 but there are lots of
unreleased fixes so I advise you to just clone
https://opendev.org/openstack/kolla-ansible
checkout stable/victoria
and pip install that directory then.
It will fix your current issue.
-yoctozepto