May 2024 - openstack-discuss - lists.openstack.org

podman inspect : Error: unexpected end of JSON input | openstack wallaby | tripleo
by Swogat Pradhan 08 May '24

08 May '24

Hi, I was updating the ssl template and then ran openstack overcloud deploy and am getting error messages as below in most of the nodes: 2024-05-07 19:04:10.564330 | 48d539a1-1679-6e66-8636-00000000f0ab | FATAL | Pre-fetch all the containers | overcloud-cephstorage-0 | item=172.25.201.68:8787/tripleomaster/openstack-cron:current-tripleo | error={"ansible_loop_var": "prefetch_image", "changed": false, "msg": "Failed to run ['/bin/podman', 'inspect', '9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c', '--format', 'json'] ['inspect', '9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c', '--format', 'json']: time=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\ntime=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\ntime=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\nError: unexpected end of JSON input\n", "prefetch_image": "172.25.201.68:8787/tripleomaster/openstack-cron:current-tripleo"} 2024-05-07 19:04:10.573348 | 48d539a1-1679-6e66-8636-00000000f0ab | TIMING | Pre-fetch all the containers | overcloud-cephstorage-0 | 0:24:39.725509 | 1.78s 2024-05-07 19:04:10.578977 | 48d539a1-1679-6e66-8636-00000000f0ab | TIMING | Pre-fetch all the containers | overcloud-cephstorage-0 | 0:24:39.731144 | 1.78s 2024-05-07 19:04:10.582725 | 48d539a1-1679-6e66-8636-00000000f0ca | FATAL | Pre-fetch all the containers | overcloud-cephstorage-1 | item=172.25.201.68:8787/tripleomaster/openstack-cron:current-tripleo | error={"ansible_loop_var": "prefetch_image", "changed": false, "msg": "Failed to run ['/bin/podman', 'inspect', '9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c', '--format', 'json'] ['inspect', '9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c', '--format', 'json']: time=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\ntime=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\ntime=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\nError: unexpected end of JSON input\n", "prefetch_image": "172.25.201.68:8787/tripleomaster/openstack-cron:current-tripleo"} 2024-05-07 19:04:10.585487 | 48d539a1-1679-6e66-8636-00000000f0ca | TIMING | Pre-fetch all the containers | overcloud-cephstorage-1 | 0:24:39.737672 | 1.14s 2024-05-07 19:04:10.589799 | 48d539a1-1679-6e66-8636-00000000f0ca | TIMING | Pre-fetch all the containers | overcloud-cephstorage-1 | 0:24:39.741977 | 1.14s 2024-05-07 19:04:10.592984 | 48d539a1-1679-6e66-8636-00000000f23e | TIMING | Run lvmfilter role | overcloud-novacompute-2 | 0:24:39.745173 | 0.11s 2024-05-07 19:04:10.595783 | 48d539a1-1679-6e66-8636-00000000f0ca | FATAL | Pre-fetch all the containers | overcloud-cephstorage-3 | item=172.25.201.68:8787/tripleomaster/openstack-cron:current-tripleo | error={"ansible_loop_var": "prefetch_image", "changed": false, "msg": "Failed to run ['/bin/podman', 'inspect', '9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c', '--format', 'json'] ['inspect', '9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c', '--format', 'json']: time=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\ntime=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\ntime=\"2024-05-07T11:04:10Z\" level=warning msg=\"Failed to decode the keys [\\\"network.network_backend\\\"] from \\\"/usr/share/containers/containers.conf\\\".\"\nError: unexpected end of JSON input\n", "prefetch_image": "172.25.201.68:8787/tripleomaster/openstack-cron:current-tripleo"} 2024-05-07 19:04:10.598356 | 48d539a1-1679-6e66-8636-00000000f0ca | TIMING | Pre-fetch all the containers | overcloud-cephstorage-3 | 0:24:39.750539 | 1.06s 2024-05-07 19:04:10.602665 | 48d539a1-1679-6e66-8636-00000000f0ca | TIMING | Pre-fetch all the containers | overcloud-cephstorage-3 | 0:24:39.754836 | 1.06s Whereas in controller nodes I am not getting any errors when doing a podman inspect using the same id. [root@overcloud-cephstorage-0 ~]# podman inspect 9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c --format json Error: unexpected end of JSON input Where the same command in controller node gives: [root@overcloud-controller-0 ~]# podman inspect 9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c --format json [ { "Id": "9a0ed16c31a654d6130fa0d146d5f14e3529a0eeb1eab1d28c8594f2b2cf0f5c", "Digest": "sha256:de2f5e94491ebf5702cf7e1cefbce4643e0f42005cc210d817c0521c2b3effaf", "RepoTags": [ "172.25.201.68:8787/tripleomaster/openstack-cron:current-tripleo" ], "RepoDigests": [ "172.25.201.68:8787/tripleomaster/openstack-cron@sha256:9f5c53d6342200114593f63fe4e1321596bb9b45861abcdcb454fd3fbaae3a6f", "172.25.201.68:8787/tripleomaster/openstack-cron@sha256:de2f5e94491ebf5702cf7e1cefbce4643e0f42005cc210d817c0521c2b3effaf" ], "Parent": "", "Comment": "", "Created": "2022-02-09T05:23:45.932319128Z", "Config": { ...... I do not understand what is happening and why this issue is coming. With regards, Swogat Pradhan

2 4

[manila][cinder]Dell Isilon: Remove Isilon Manila driver from Dalmation release
by cuiye.liu＠dell.com 08 May '24

08 May '24

Hi, I will remove the isilon manila driver from Dalmation release. What should I do? and what's the flow of the action. Thanks in advance. Regards, Cherry

1 0

Re: [openstack][nova]about instance-state when resizing failed
by Nguyễn Hữu Khôi 07 May '24

07 May '24

Hello. I mean in this link https://docs.openstack.org/cinder/pike/install/cinder-storage-install-ubunt… It has " Warning" section to guide about setting filters on storage and compute nodes then I followed it. Nguyen Huu Khoi On Tue, May 7, 2024 at 10:37 PM Takashi Kajinami <kajinamit(a)oss.nttdata.com> wrote: > On 5/8/24 00:16, Nguyễn Hữu Khôi wrote: > > Hello. > > I do filter for both storage and compute nodes because it has warning > section in docs. > > > > Yes, i make like this > > > > > https://serverfault.com/questions/965942/centos-multipath-dev-mapper-dont-m… > < > https://serverfault.com/questions/965942/centos-multipath-dev-mapper-dont-m… > > > > > > So no partN-mpath-xxxx display when i use > > > > dmsetup info -C > > > > Honestly speaking I'm confused here because the link you posted does not > mention lvm filter. > It explains the multipath configuration. skip_kpartx may avoid partition > detection, but it does not > work for lvm detection if lvm is directly deployed on block devices (I > mean, without partitions). > > If you have the lvm filtering, then it should be configured in > /etc/lvm/lvm.conf , instead of > multipart.conf . > > > > On Tue, May 7, 2024, 9:58 PM Takashi Kajinami <kajinamit(a)oss.nttdata.com > <mailto:kajinamit@oss.nttdata.com>> wrote: > > > > > > > > On 5/7/24 22:51, Nguyễn Hữu Khôi wrote: > > > Hello. > > > I do filter as > > > > > > > https://docs.openstack.org/cinder/pike/install/cinder-storage-install-ubunt… > < > https://docs.openstack.org/cinder/pike/install/cinder-storage-install-ubunt…> > < > https://docs.openstack.org/cinder/pike/install/cinder-storage-install-ubunt… > < > https://docs.openstack.org/cinder/pike/install/cinder-storage-install-ubunt… > >> > > > > To make sure that we have no misunderstanding between us, did you > configured the filter > > not only in storage node (or probably controller node if you don't > have dedicated nodes > > for cinder-volume) BUT ALSO in all compute nodes ? > > > > I'm asking this because the current documentation is not specific > about this requirement > > but the filter should be configured in all nodes which may attach > instance volumes. > > > > > > > > I check and dont see any wrong mapping. I also set skip_kpartx > "yes". > > > > So you mean you don't see any lvms from guest instances when you do > lvs/vgs/pvs in your > > compute nodes ? > > > > > > > > I still find a solution for it or atleast when resizing/migrating > failed, my instance wont be set to ERROR state. > > > > > > This problem happens only with iscsi. > > > > > > On Tue, May 7, 2024, 8:12 PM Takashi Kajinami < > kajinamit(a)oss.nttdata.com <mailto:kajinamit@oss.nttdata.com> <mailto: > kajinamit(a)oss.nttdata.com <mailto:kajinamit@oss.nttdata.com>>> wrote: > > > > > > The failure reminds me of the "well-known" problem caused by > lvm. > > > I'd suggest that you check > > > > > > - If instances which fails to be resized/migrated have lvm > deployed in their volumes > > > > > > - If these lvms on instance volumes are detected by host > operating system > > > > > > - If you configure appropriate lvm filter in your compute > nodes to avoid host lvm system > > > from detecting lvms on devices for instance volumes. > > > > https://docs.openstack.org/cinder/latest/install/cinder-storage-install-rdo… > < > https://docs.openstack.org/cinder/latest/install/cinder-storage-install-rdo…> > < > https://docs.openstack.org/cinder/latest/install/cinder-storage-install-rdo… > < > https://docs.openstack.org/cinder/latest/install/cinder-storage-install-rdo… > >> > > > > > > > > > On 5/6/24 23:38, Nguyễn Hữu Khôi wrote: > > > > Hello. > > > > > > > > I am using Openstack Yoga with Dell Powerstore 5000T is > backend. > > > > > > > > This is my report: > > > > > > > > https://bugs.launchpad.net/os-brick/+bug/2063345 < > https://bugs.launchpad.net/os-brick/+bug/2063345> < > https://bugs.launchpad.net/os-brick/+bug/2063345 < > https://bugs.launchpad.net/os-brick/+bug/2063345>> < > https://bugs.launchpad.net/os-brick/+bug/2063345 < > https://bugs.launchpad.net/os-brick/+bug/2063345> < > https://bugs.launchpad.net/os-brick/+bug/2063345 < > https://bugs.launchpad.net/os-brick/+bug/2063345>>> > > > > > > > > This bug is also with Victoria and NetApp as backend: > > > > > > > > https://bugs.launchpad.net/os-brick/+bug/1992289 < > https://bugs.launchpad.net/os-brick/+bug/1992289> < > https://bugs.launchpad.net/os-brick/+bug/1992289 < > https://bugs.launchpad.net/os-brick/+bug/1992289>> < > https://bugs.launchpad.net/os-brick/+bug/1992289 < > https://bugs.launchpad.net/os-brick/+bug/1992289> < > https://bugs.launchpad.net/os-brick/+bug/1992289 < > https://bugs.launchpad.net/os-brick/+bug/1992289>>> > > > > > > > > It looks like with iscsi multipath only. I don't see any > reports with Ceph. > > > > > > > > I try to create problem but It is random when I do resize > instance(instance will move to another compute-cold migrate) > > > > > > > > My log tells that > > > > > > > > 2022-10-04 10:50:12.238 7 ERROR oslo_messaging.rpc.server > Command: multipath -f 3600a09803831486e695d536269665144 > > > > 2022-10-04 10:50:12.238 7 ERROR oslo_messaging.rpc.server > Exit code: 1 > > > > 2022-10-04 10:50:12.238 7 ERROR oslo_messaging.rpc.server > Stdout: '' > > > > 2022-10-04 10:50:12.238 7 ERROR oslo_messaging.rpc.server > Stderr: 'Oct 04 10:49:32 | 3600a09803831486e695d536269665144: map in use\n > > > > > > > > > > > > Nguyen Huu Khoi > > > > > > > > > > > > On Mon, May 6, 2024 at 9:22 PM Eugen Block <eblock(a)nde.ag > <mailto:eblock@nde.ag> <mailto:eblock@nde.ag <mailto:eblock@nde.ag>> > <mailto:eblock@nde.ag <mailto:eblock@nde.ag> <mailto:eblock@nde.ag > <mailto:eblock@nde.ag>>>> wrote: > > > > > > > > Hi, > > > > > > > > I think you'll need to be more specific here. Which > openstack version > > > > are you running? Just recently I have been shuffling > around instances > > > > with live and cold migration (in version Victoria, so > quite old), some > > > > of them did fail but I didn't have to reset any state > or > > > > shelve/unshelve instances. I can't recall if the > instances were in > > > > ERROR state, though. I didn't need to check because > after fixing > > > > whatever broke the migration I just issued another > (live-)migrate > > > > command and then it worked. > > > > We use only ceph as backend for glance, nova and > cinder, which > > > > backend(s) do you use? Maybe that has an impact as > well, I don't know. > > > > > > > > Regards, > > > > Eugen > > > > > > > > Zitat von Nguyễn Hữu Khôi <nguyenhuukhoinw(a)gmail.com > <mailto:nguyenhuukhoinw@gmail.com> <mailto:nguyenhuukhoinw@gmail.com > <mailto:nguyenhuukhoinw@gmail.com>> <mailto:nguyenhuukhoinw@gmail.com > <mailto:nguyenhuukhoinw@gmail.com> <mailto:nguyenhuukhoinw@gmail.com > <mailto:nguyenhuukhoinw@gmail.com>>>>: > > > > > > > > > Hello. > > > > > > > > > > I would like to know why we set instance-state to > ERROR when > > > > > resizing/migrating failed? I have a problem with > resizing/migrating > > > > > instances. I must reset state then shelve and > unshelve instances to make it > > > > > come back. > > > > > > > > > > Thank you for your time. > > > > > > > > > > Nguyen Huu Khoi > > > > > > > > > > > > > > > > > >

1 0

neutron's OVS based VXLAN seems to be unable to access the network
by 2292613444＠qq.com 06 May '24

06 May '24

I found that the cloud host I created cannot connect to the network. The network type is VXLAN, and it cannot even access cloud init I used this installation document (partially in Chinese) and it was normal around 2022, but it cannot be used after updating the software source recently. The use of VXLAN in cloud hosts has the following faults (1) Unable to access network resources other than ICMP (ping), and even curl http://169.254.169.254 (This also leads to the inability to use cloud init) (2) Sometimes unable to ping 169.254.169.254 for unknown reasons. (3)Any service other than ICMP service cannot be accessed, All firewalls and security groups are allowed or closed. My network situation is as follows Internal (vxlan) 10.0.0.0/24 Gateway 10.0.0.1 Router gateway 10.0.0.1 → 192.168.10.121 External (flat) 192.168.10.11.0/24 Gateway 192.168.10.1.1 If any other information is needed, I can provide it and hope to fix this error. system version:ubuntu 22.04 last version (online ) openstack Y version I used a command in the openstack routing host to check The specific details are as follows：：（10.0.0.129 is cloud host,vxlan is port qr-4528fcca-21） ###start root@controller:~# ip netns qdhcp-2aa854ba-7fa6-43e4-989e-e97dafd777ee (id: 1) qdhcp-c7d9811d-1fcc-4aee-8f01-6f556fb56044 (id: 2) qrouter-48610214-6030-4e7e-8fe8-c60241e7c89c (id: 0) root@controller:~# ip netns exec qrouter-48610214-6030-4e7e-8fe8-c60241e7c89c bash root@controller:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 13: qr-4528fcca-21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:41:c8:47 brd ff:ff:ff:ff:ff:ff inet 10.0.0.1/24 brd 10.0.0.255 scope global qr-4528fcca-21 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe41:c847/64 scope link valid_lft forever preferred_lft forever 14: qg-622550f9-80: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:31:eb:c2 brd ff:ff:ff:ff:ff:ff inet 192.168.101.121/24 brd 192.168.101.255 scope global qg-622550f9-80 valid_lft forever preferred_lft forever inet 192.168.101.177/32 brd 192.168.101.177 scope global qg-622550f9-80 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe31:ebc2/64 scope link valid_lft forever preferred_lft forever root@controller:~# tcpdump -i qr-4528fcca-21 -vvvvv tcpdump: listening on qr-4528fcca-21, link-type EN10MB (Ethernet), snapshot length 262144 bytes 17:36:02.295718 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:02.371746 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:0f:28:ea (oui Unknown), length 300, xid 0xe07c1d6a, Flags [none] (0x0000) Client-Ethernet-Address fa:16:3e:0f:28:ea (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message (53), length 1: Discover Parameter-Request (55), length 13: Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121) Domain-Name (15), Domain-Name-Server (6), Hostname (12), YD (40) YS (41), NTP (42), MTU (26), Unknown (119) Default-Gateway (3) END (255), length 0 PAD (0), length 0, occurs 41 17:36:02.373214 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:0f:28:ea (oui Unknown), length 300, xid 0xe07c1d6a, Flags [none] (0x0000) Client-Ethernet-Address fa:16:3e:0f:28:ea (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message (53), length 1: Request Server-ID (54), length 4: 10.0.0.2 Requested-IP (50), length 4: 10.0.0.129 Parameter-Request (55), length 13: Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121) Domain-Name (15), Domain-Name-Server (6), Hostname (12), YD (40) YS (41), NTP (42), MTU (26), Unknown (119) Default-Gateway (3) END (255), length 0 PAD (0), length 0, occurs 29 17:36:02.402630 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:36:02.402639 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:36:02.403415 IP (tos 0x0, ttl 64, id 47481, offset 0, flags [DF], proto UDP (17), length 72) 10.0.0.129.46568 > 223.5.5.5.domain: [udp sum ok] 57238+ A? does-not-exist.example.com. (44) 17:36:02.403453 IP (tos 0x0, ttl 64, id 47482, offset 0, flags [DF], proto UDP (17), length 72) 10.0.0.129.46568 > 223.5.5.5.domain: [udp sum ok] 33458+ AAAA? does-not-exist.example.com. (44) 17:36:02.404176 IP (tos 0x0, ttl 253, id 44451, offset 0, flags [DF], proto UDP (17), length 104) 223.5.5.5.domain > 10.0.0.129.46568: [udp sum ok] 33458 q: AAAA? does-not-exist.example.com. 0/1/0 ns: does-not-exist.example.com. [5m] SOA <BAD PTR> [|domain] 17:36:02.568387 IP (tos 0x0, ttl 62, id 34373, offset 0, flags [DF], proto UDP (17), length 148) 223.5.5.5.domain > 10.0.0.129.46568: [udp sum ok] 57238 NXDomain- q: A? does-not-exist.example.com. 0/1/0 ns: example.com. [1s] SOA ns.icann.org. noc.dns.icann.org. 2024041801 7200 3600 1209600 3600 (120) 17:36:02.721968 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:02.866526 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) :: > ff02::1:ff0f:28ea: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fe80::f816:3eff:fe0f:28ea unknown option (14), length 8 (1): 0x0000: b4d5 8b30 6d50 17:36:03.868276 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:03.868993 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:04.038227 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:07.590618 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:36:07.591603 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:36:07.876243 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:11.884088 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:26.791124 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe37:6938 > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:37:69:38 0x0000: fa16 3e37 6938 17:36:34.348772 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:36:34.348788 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:36:43.071294 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:43.139163 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328) 0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:0f:28:ea (oui Unknown), length 300, xid 0x29d5e37d, Flags [none] (0x0000) Client-Ethernet-Address fa:16:3e:0f:28:ea (oui Unknown) Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message (53), length 1: Request Requested-IP (50), length 4: 10.0.0.129 Hostname (12), length 15: "host-10-0-0-129" Parameter-Request (55), length 13: Subnet-Mask (1), BR (28), Time-Zone (2), Classless-Static-Route (121) Domain-Name (15), Domain-Name-Server (6), Hostname (12), YD (40) YS (41), NTP (42), MTU (26), Unknown (119) Default-Gateway (3) END (255), length 0 PAD (0), length 0, occurs 18 17:36:43.451136 IP6 (hlim 1, next-header Options (0) payload length: 36) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:43.788779 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) :: > ff02::1:ff0f:28ea: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has fe80::f816:3eff:fe0f:28ea unknown option (14), length 8 (1): 0x0000: eb9e aa08 ffc6 17:36:44.790128 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:44.791007 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:45.433011 IP6 (hlim 1, next-header Options (0) payload length: 36) fe80::f816:3eff:fe0f:28ea > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 1 group record(s) [gaddr ff02::1:ff0f:28ea to_ex { }] 17:36:45.876227 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:36:45.876239 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:36:48.796920 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:36:52.804263 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe0f:28ea > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:0f:28:ea 0x0000: fa16 3e0f 28ea 17:37:20.924762 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:37:20.924779 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:37:51.676660 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:37:51.676672 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:38:25.468636 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:38:25.468653 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:38:31.772908 IP (tos 0x0, ttl 64, id 19606, offset 0, flags [DF], proto ICMP (1), length 84) 10.0.0.129 > 8.8.8.8: ICMP echo request, id 1398, seq 1, length 64 17:38:31.783398 IP (tos 0x0, ttl 53, id 0, offset 0, flags [none], proto ICMP (1), length 84) 8.8.8.8 > 10.0.0.129: ICMP echo reply, id 1398, seq 1, length 64 17:38:32.774655 IP (tos 0x0, ttl 64, id 19790, offset 0, flags [DF], proto ICMP (1), length 84) 10.0.0.129 > 8.8.8.8: ICMP echo request, id 1398, seq 2, length 64 17:38:32.785097 IP (tos 0x0, ttl 53, id 0, offset 0, flags [none], proto ICMP (1), length 84) 8.8.8.8 > 10.0.0.129: ICMP echo reply, id 1398, seq 2, length 64 17:38:36.838640 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:38:36.839533 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:38:59.212868 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:38:59.212885 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:39:18.252924 IP (tos 0x0, ttl 64, id 29804, offset 0, flags [DF], proto TCP (6), length 60) 10.0.0.129.60668 > 169.254.169.254.http: Flags [S], cksum 0xd701 (correct), seq 3028122336, win 28200, options [mss 1410,sackOK,TS val 4294871008 ecr 0,nop,wscale 7], length 0 17:39:18.252998 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xfad3), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597215885 ecr 4294871008,nop,wscale 7], length 0 17:39:19.270668 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xf6d9), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597216903 ecr 4294871008,nop,wscale 7], length 0 17:39:21.286651 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xeef9), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597218919 ecr 4294871008,nop,wscale 7], length 0 17:39:23.430642 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:39:23.432189 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:39:25.478662 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xde99), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597223111 ecr 4294871008,nop,wscale 7], length 0 17:39:33.670667 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 169.254.169.254.http > 10.0.0.129.60668: Flags [S.], cksum 0x5eac (incorrect -> 0xbe99), seq 3154613265, ack 3028122337, win 64308, options [mss 1410,sackOK,TS val 3597231303 ecr 4294871008,nop,wscale 7], length 0 17:39:44.044984 IP (tos 0x0, ttl 64, id 29814, offset 0, flags [DF], proto TCP (6), length 131) 10.0.0.129.60668 > 169.254.169.254.http: Flags [P.], cksum 0x2428 (correct), seq 1:80, ack 1, win 221, options [nop,nop,TS val 4294896800 ecr 3597215885], length 79: HTTP, length: 79 GET / HTTP/1.1 User-Agent: curl/7.29.0 Host: 169.254.169.254 Accept: */* 17:39:44.086640 IP (tos 0x0, ttl 64, id 60194, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > 10.0.0.129.60668: Flags [.], cksum 0x5ea4 (incorrect -> 0x58b3), seq 1, ack 80, win 502, options [nop,nop,TS val 3597241719 ecr 4294896800], length 0 17:39:44.140919 IP (tos 0x0, ttl 64, id 60195, offset 0, flags [DF], proto TCP (6), length 267) 169.254.169.254.http > 10.0.0.129.60668: Flags [P.], cksum 0x5f7b (incorrect -> 0x1982), seq 1:216, ack 80, win 502, options [nop,nop,TS val 3597241773 ecr 4294896800], length 215: HTTP, length: 215 HTTP/1.1 200 OK content-type: text/plain; charset=UTF-8 content-length: 98 date: Mon, 29 Apr 2024 09:39:44 GMT 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 2009-04-04 latest [|http] 17:39:49.053068 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:39:49.053086 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:39:49.286648 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:39:49.287162 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:40:14.170722 IP (tos 0x0, ttl 64, id 60196, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > 10.0.0.129.60668: Flags [F.], cksum 0x5ea4 (incorrect -> 0xe256), seq 216, ack 80, win 502, options [nop,nop,TS val 3597271803 ecr 4294896800], length 0 17:40:14.172246 IP (tos 0x0, ttl 64, id 29815, offset 0, flags [DF], proto TCP (6), length 64) 10.0.0.129.60668 > 169.254.169.254.http: Flags [.], cksum 0x0c22 (correct), seq 80, ack 1, win 221, options [nop,nop,TS val 4294926927 ecr 3597241719,nop,nop,sack 1 {216:217}], length 0 17:40:14.172294 IP (tos 0x0, ttl 64, id 60197, offset 0, flags [DF], proto TCP (6), length 267) 169.254.169.254.http > 10.0.0.129.60668: Flags [P.], cksum 0x5f7b (incorrect -> 0x2e83), seq 1:216, ack 80, win 502, options [nop,nop,TS val 3597271804 ecr 4294926927], length 215: HTTP, length: 215 HTTP/1.1 200 OK content-type: text/plain; charset=UTF-8 content-length: 98 date: Mon, 29 Apr 2024 09:39:44 GMT 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 2009-04-04 latest [|http] 17:40:19.238647 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:40:19.240069 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:40:52.608165 IP (tos 0x0, ttl 64, id 24950, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.57838 > 223.5.5.5.domain: [udp sum ok] 18832+ A? 0.centos.pool.ntp.org. (39) 17:40:52.608222 IP (tos 0x0, ttl 64, id 24951, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.57838 > 223.5.5.5.domain: [udp sum ok] 12709+ AAAA? 0.centos.pool.ntp.org. (39) 17:40:52.608926 IP (tos 0x0, ttl 253, id 63426, offset 0, flags [DF], proto UDP (17), length 99) 223.5.5.5.domain > 10.0.0.129.57838: [udp sum ok] 12709 q: AAAA? 0.centos.pool.ntp.org. 0/1/0 ns: 0.centos.pool.ntp.org. [5m] SOA . ^@. [|domain] 17:40:52.621561 IP (tos 0x0, ttl 62, id 26683, offset 0, flags [DF], proto UDP (17), length 215) 223.5.5.5.domain > 10.0.0.129.57838: [udp sum ok] 18832- q: A? 0.centos.pool.ntp.org. 4/0/0 0.centos.pool.ntp.org. [5m] A 162.159.200.1, 0.centos.pool.ntp.org. [5m] A 193.182.111.14, 0.centos.pool.ntp.org. [5m] A 111.230.189.174, 0.centos.pool.ntp.org. [5m] A 193.182.111.12 (187) 17:40:57.612621 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129^C, length 28 17:40:57.612632 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:41:31.942692 IP (tos 0x0, ttl 64, id 60198, offset 0, flags [DF], proto TCP (6), length 267) 10.0.0.1.9697 > 10.0.0.129.60668: Flags [P.], cksum 0x157f (incorrect -> 0x2322), seq 3154613266:3154613481, ack 3028122416, win 502, options [nop,nop,TS val 3597349575 ecr 4294926927], length 215 17:41:31.944269 IP (tos 0x0, ttl 64, id 21103, offset 0, flags [DF], proto TCP (6), length 40) 10.0.0.129.60668 > 10.0.0.1.9697: Flags [R], cksum 0x58d3 (correct), seq 3028122416, win 0, length 0 17:41:32.668720 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:41:32.668750 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:41:37.062634 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:41:37.063306 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:42:43.622650 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 16) fe80::f816:3eff:fe41:c847 > ip6-allrouters: [icmp6 sum ok] ICMP6, router solicitation, length 16 source link-address option (1), length 8 (1): fa:16:3e:41:c8:47 0x0000: fa16 3e41 c847 17:42:54.604794 IP (tos 0x0, ttl 64, id 29817, offset 0, flags [DF], proto TCP (6), length 64) 10.0.0.129.60668 > 169.254.169.254.http: Flags [.], cksum 0x9970 (correct), seq 79, ack 1, win 221, options [nop,nop,TS val 120064 ecr 3597241719,nop,nop,sack 1 {216:217}], length 0 17:42:54.604842 IP (tos 0x0, ttl 64, id 60199, offset 0, flags [DF], proto TCP (6), length 52) 169.254.169.254.http > 10.0.0.129.60668: Flags [.], cksum 0x5ea4 (incorrect -> 0xf9f2), seq 217, ack 80, win 502, options [nop,nop,TS val 3597432237 ecr 4294926927], length 0 17:42:59.612900 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:42:59.612917 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:42:59.750644 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:42:59.751196 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:43:17.363162 IP (tos 0x0, ttl 64, id 49296, offset 0, flags [DF], proto UDP (17), length 76) 10.0.0.129.40786 > 162.159.200.1.ntp: [udp sum ok] NTPv4, Client, length 48 Leap indicator: (0), Stratum 0 (unspecified), poll 7 (128s), precision 32 Root Delay: 0.000000, Root dispersion: 0.000000, Reference-ID: (unspec) Reference Timestamp: 0.000000000 Originator Timestamp: 0.000000000 Receive Timestamp: 0.000000000 Transmit Timestamp: 3841207700.761775004 (2021-09-21T10:08:20Z) Originator - Receive Timestamp: 0.000000000 Originator - Transmit Timestamp: 3841207700.761775004 (2021-09-21T10:08:20Z) 17:43:32.380593 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:43:32.380610 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:44:07.420619 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:44:07.420635 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:44:54.925246 IP (tos 0x0, ttl 64, id 29819, offset 0, flags [DF], proto TCP (6), length 64) 10.0.0.129.60668 > 169.254.169.254.http: Flags [.], cksum 0xc36e (correct), seq 79, ack 1, win 221, options [nop,nop,TS val 240384 ecr 3597241719,nop,nop,sack 1 {216:217}], length 0 17:44:54.925303 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40) 169.254.169.254.http > 10.0.0.129.60668: Flags [R], cksum 0x13fc (correct), seq 3154613266, win 0, length 0 17:44:59.932983 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:44:59.933001 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:45:00.070625 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:45:00.070993 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 17:47:38.400269 IP (tos 0x0, ttl 64, id 29403, offset 0, flags [DF], proto UDP (17), length 76) 10.0.0.129.57173 > 162.159.200.1.ntp: [udp sum ok] NTPv4, Client, length 48 Leap indicator: (0), Stratum 0 (unspecified), poll 8 (256s), precision 32 Root Delay: 0.000000, Root dispersion: 0.000000, Reference-ID: (unspec) Reference Timestamp: 0.000000000 Originator Timestamp: 0.000000000 Receive Timestamp: 0.000000000 Transmit Timestamp: 3566054594.035896213 (2013-01-01T18:43:14Z) Originator - Receive Timestamp: 0.000000000 Originator - Transmit Timestamp: 3566054594.035896213 (2013-01-01T18:43:14Z) 17:47:43.405117 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:47:43.405133 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:48:21.491778 IP (tos 0x0, ttl 64, id 21240, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.60700 > 223.5.5.5.domain: [udp sum ok] 37019+ A? 1.centos.pool.ntp.org. (39) 17:48:21.491845 IP (tos 0x0, ttl 64, id 21241, offset 0, flags [DF], proto UDP (17), length 67) 10.0.0.129.60700 > 223.5.5.5.domain: [udp sum ok] 41914+ AAAA? 1.centos.pool.ntp.org. (39) 17:48:21.492638 IP (tos 0x0, ttl 253, id 58295, offset 0, flags [DF], proto UDP (17), length 99) 223.5.5.5.domain > 10.0.0.129.60700: [udp sum ok] 41914 q: AAAA? 1.centos.pool.ntp.org. 0/1/0 ns: 1.centos.pool.ntp.org. [5m] SOA . . [|domain] 17:48:21.504470 IP (tos 0x0, ttl 62, id 62111, offset 0, flags [DF], proto UDP (17), length 215) 223.5.5.5.domain > 10.0.0.129.60700: [udp sum ok] 37019- q: A? 1.centos.pool.ntp.org. 4/0/0 1.centos.pool.ntp.org. [5m] A 111.230.189.174, 1.centos.pool.ntp.org. [5m] A 193.182.111.14, 1.centos.pool.ntp.org. [5m] A 117.80.231.60, 1.centos.pool.ntp.org. [5m] A 110.42.98.138 (187) 17:48:26.508934 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.1 tell 10.0.0.129, length 28 17:48:26.508951 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.1 is-at fa:16:3e:41:c8:47 (oui Unknown), length 28 17:48:26.662640 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.0.0.129 tell 10.0.0.1, length 28 17:48:26.663269 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.0.0.129 is-at fa:16:3e:0f:28:ea (oui Unknown), length 28 107 packets captured 107 packets received by filter 0 packets dropped by kernel ###end Firstly, this is the situation of tcpdump from the beginning of creating the cloud host to the end of my following command execution The command is Ping 8.8.8.8 #Success Curl http://169.254.169.254 #Report the following errors Curl: (56) Recv failure: connection reset by peer That's all for these situations. If needed, I can provide the following content The situation and installation documents related to this environment Other documents that may help resolve this situation

2 6

[all][oslo][neutron][glance][ironic] Deprecating eventlet based WSGI server
by Takashi Kajinami 06 May '24

06 May '24

Hello, As you know there has been a long discussion about eventlet usage in OpenStack, and after seeing multiple problems caused by the library and no long term plan in the project, and we are moving toward doping the dependency on eventlet. https://review.opendev.org/c/openstack/governance/+/916546 One of the core implementation which heavily depends on eventlet is WSGI server. Multiple projects have been providing the -api console scripts (eg nova-api) based on the oslo.service's WSGI server implementation which depends on eventlet. On the other hand, nowadays we have multiple methods to launch api services, such as httpd+mod_wsgi, uwsgi and gunicorn. And a few projects in OpenStack don't provide the -api console script (eg. aodh, barbican and placement) but suggest using these external frameworks to run their api WSGI application. I know that multiple deployment tools (for example TripleO in the past) and some distros (Debian) already started using the external framework for consistency, better concurrency and so on. Given the situation, I wonder if we can consider deprecating the eventlet WSGI server in oslo.service in 2024.2/2025.1 and eventually remove all -api script in 2025.2. This was discussed during the previous nova PTG sessions and I think this makes complete sense as a global direction given the variety of alternative approaches currently available. I'm tagging neutron(*1) and glance(*2) because IIRC these had(or even have) some problems with running API in wsgi mode, and also ironic because I've seen some works coming from it, iiuc. Please let me know if you have any concern applying this approach to these (or the other) projects. Thank you, Takashi Kajinami (*1) For neutron, there are a few issues being fixed now https://bugs.launchpad.net/neutron/+bug/2062009 but ml2-ovn may need additional fixes. https://bugs.launchpad.net/neutron/+bug/1912359 (*2) As far as I know the issue with glance was already fixed some time ago[1]. https://bugs.launchpad.net/glance/+bug/1888713 -- Takashi Kajinami irc: tkajinam github: https://github.com/kajinamit launchpad: https://launchpad.net/~kajinamit

7 9

[OpenStack][VDI][Bumblebee] About CIDR, Security Group Help
by Matthew Swigart 06 May '24

06 May '24

Hello all, This is Matthew who just started working for the OpenStack VDI project last year and I decided to try with Bumblebee+DevStack for an experiment. [1] https://github.com/NeCTAR-RC/bumblebee [2] https://docs.openstack.org/devstack/latest/ So I created two VMs using libvirt on Ubuntu 22.04 - one for DevStack, the other one for the Bumblebee VDI project. Each VM is running on Ubuntu 22.04. I installed DevStack successfully and I can access the horizon dashboard from Bumblebee VM. So I created a new project and created a new user and assigned it to a new project. Also created a new application credential and downloaded openrc file. The problem is, when I tried to create a security group using command, it succeeded so I can see a new security group from Horizon *but when I run `openstack security group list` command, there is nothing.* Also I needed to add some security group rules to allow SSH access, but no idea how to get the correct *CIDR value*. [3] https://docs.openstack.org/ocata/user-guide/cli-nova-configure-access-secur… So I will appreciate it if I can get any support. Thank you, Matthew

3 3

[all][packaging][release][qa] pep-517 and pip 23
by smooney＠redhat.com 06 May '24

06 May '24

hi folks, TL;DR ----- this is a long email so the tl;dr is we should add pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` to all openstack projects to supprot pip 23.1+ Long version ------------ As some of you may know the ptg happend last week an done of hte topic raised there was a know limitation in devstack where GLOBAL_VENV=True does not work on centos/fedora derived distros. while i generally dont use centos/fedora derived distros for my daily work i have been using devstack for over 10 years at this point so when im told it broken i ocationlly will go fix it. In this case the issue is not with devstack (it often isnt) and is actully in our collective reliance on a deprecated functionality and how we package openstack today. Back story, for better or worse when openstack was created the landscape of python packaging was significantly different then it is today. To help corodinate our dependcies the openstack comunity went all in on requirements.txt and developed upper-constraits to manage our requirements and developed PBR (python build Reasonableness). both of those approches have served us well and likely will continue to do so for some time. Since then pyproject.toml has started to gain popularity and pep-517 (A build-system independent format for source trees) has been adotpted but so far openstack as a comunity has not embraced those changes. (although pbr already supprot pep-517 :) ). Until now this has not been a problem however with recent release of pip things are now breaking. Problem Context: - openstack does not provide pyporject.toml files - openstack project configure setuptools to use pbr via setup.py - setuptools support generating scripts form entry points(console_scripts) which we use for project binaries( nova- compute, neutron-server, ...) - openstack api services use a pbr extension know as wsgi_scripts which adds a similar functionality for generating wsgi application mains file - devstack due to coming changes in distro to block "sudo pip install" now installs openstack in a single by default on debian distros. on centos/fedora distros it currently almost functional (but not). - devstack installs in editable mode because that is how most developers use devstack for developemnt What changed: - pip 20.2 deprecated support for falling back to calling setup.py when pyproject.toml is not present - https://pip.pypa.io/en/stable/news/#v20-2 - https://github.com/pypa/pip/issues/8368 - at this point when pyproject.yaml is not called pip calls setup.py install [-e] <path to repo or package name> - setup.py invokes pbr to generate wsgi_scripts and other files - in pip 23.1 the fallback was removed https://pip.pypa.io/en/stable/news/#v23-1 - pip now builds a temp local wheel instead of calling setup.py install when pyproject.toml is not found - in editable mode (-e) the default way the wheel is built does not invoke pbr properly to generate the wsgi files - as a result the wsgi files are not installed to the venv bin dir and this causes devstack to fail to start keystone (or any other wsgi api) The fix, there are a few ways to approch this: - clamp pip (https://review.opendev.org/c/openstack/devstack/+/898845) this is a terible idea and it breaks debian - clamping pip below 23.1 works on centos for now however OVS on debian already requires pep 517 for installing - as such when that version of ovs (or other projects start enforcing that) hits centos it will break - add a minimal pyproject.toml to all openstack project - example for keystone https://review.opendev.org/c/openstack/keystone/+/899519 - this example jsut adds the minium pyproject.toml to enable pbr to be used as the build system - it does not remove supprot for setup.py but it will not be used by default going forward whit new version of pip - all config is still in setup.cfg - add pyproject.toml support but remove setup.py support https://review.opendev.org/c/opendev/bindep/+/816741 - this goes one step futrue and actuly nulls out setup.py - we could take this approach if we want to but its not required. The reason for this email is to get agreement on how to support pip 23.1+ My proposal would be to add pyproject.toml with the minimal config required to enable pip 23.2 but not drop the setup.py support in this cycle. i would suggest that we can deprecate installing via setup.py in caracal if that is desirable but not make that change until the next slurp to give time to projects and packages to catch up. if there are no objections to that i would like to ask what is the preferd way to create and submit the patches for the minimal pyproject.toml ``` [build-system] requires = ["pbr>=5.7.0", "setuptools>=36.6.0", "wheel"] build-backend = "pbr.build" ``` i can propose patches to the main project deployed by devstack by default. i.e. nova/neutron ectra but it might be better if someone form the core teams of each project could take on that task and chasing the reviews. if someone has a script to generate a patch for a set of repos that just add the above pyporject.toml then we can use that to keep a consistent topic/commit message across all patches. my proposal for testing this is two fold. in ci we can test the patches are functional by using a DNM patch to devstack to force enable the gloal venv on centos and depend on the set of patches, i will modify https://review.opendev.org/c/openstack/devstack/+/898845 to do that locally we can test that the patch is functional by doing the following substituting keystone for the relevant project. ``` git clone https://opendev.org/openstack/keystone cd keystone python3 -m venv .venv . .venv/bin/activate python3 -m pip install -U pip python3 -m pip install -e . ls .venv/bin/keystone* ``` the ls will not contain the wsgi script with the addition of the pyproject.toml it should. on debian sid i can reproduce the same issue the problem we are seeing on centos 9 now will start impacting other distos eventually so its good to fix this now. if i start proposing this to glance/nova/neutron/cinder/swift/horizon/placement (to get a default devstack working) can i get cores to review? can cores form each project nominate peopel to review or volenteer to do this work. i have created https://etherpad.opendev.org/p/pep-517-and-pip-23 to track this work i have some other work that i think we can do in parallel like addign pyporject.toml support to the pti requirements if we go this way but that i think we can discuss on a governacne patch to updated it. anyway this is a lot to digest so feedback is welcome. if someone else would like to drive this that is also welcome (i.e. tc/release team) regards sean.

10 27

[openstack][neutron] neutron api server deployment
by hai wu 06 May '24

06 May '24

Hi, Is it the case that there are basically 2 ways to deploy a neutron api server? One way is to deploy it via wsgi/apache, which requires deploying another neutron-rpc-server process along with it; Another way is to deploy it as standalone (without the help of wsgi/apache), with 'rpc_worker' parameter configured, so that rpc worker would run alongside with api worker process. If so, what are the pros and cons for them? Did not find much Openstack neutron documentation on this topic thus far. - Hai

1 0

[neutron] Bug deputy report for week of April 29th
by Brian Haley 06 May '24

06 May '24

Hi, I was Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * None High bugs --------- * https://bugs.launchpad.net/neutron/+bug/2064778 - The ML2/OVN OVSDB maintenance job fails to handle properly user defined flavor routers gateway ports - https://review.opendev.org/c/openstack/neutron/+/918151 Medium bugs ----------- * https://bugs.launchpad.net/python-neutronclient/+bug/2064286 - BGPVPN client is post to the wrong endpoint - https://review.opendev.org/c/openstack/python-neutronclient/+/917835 Low bugs -------- * None Misc bugs --------- * None Wishlist bugs ------------- * https://bugs.launchpad.net/neutron/+bug/2064120 - [RFE] BGP Dynamic Routing for IPv4 AND IPv6 routes not possible with just one dragent (really make use of MP-BGP or support multiple BGP speakers / sessions per dragent) * https://bugs.launchpad.net/neutron/+bug/2064708 - Neutron documentation using real IP addresses and AS numbers instead of example values - marked low-hanging fruit

1 0

[nova][cinder] Openstack with FC SAN storage with HA
by Satish Patel 06 May '24

06 May '24

Folks, I have a question, Does openstack support FC SAN storage? I have deployed openstack with ceph but first time got this requirement that a customer has SAN storage because they were using VMware but now move to openstack. How does openstack talk to FC SAN and what about VM migration etc? Do I need to mount the LUN disk on each computer and point nova to the LUN disk? Or I have to create some kind of shared file system on top of the LUN and expose that to nova/cinder. Can someone explain the entire workflow?

5 8