<div dir="ltr"><div>Hi,</div><div><br></div><div>Thank you for your input!<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Nov 25, 2020 at 3:09 AM Ian Wienand <<a href="mailto:iwienand@redhat.com">iwienand@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Tue, Nov 24, 2020 at 11:54:55AM +0100, Dmitry Tantsur wrote:<br>
> The problem is, I cannot make Glean work with any ramdisk I build. The crux<br>
> of the problem seems to be that NetworkManager (used by default in RHEL,<br>
> CentOS, Fedora and Debian at least) starts very early, creates the default<br>
> connection and ignores whatever files Glean happens to write afterwards. On<br>
> Debian running `systemctl restart networking` actually helped to pick the<br>
> new configuration, but I'm not sure we want to do that in Glean. I haven't<br>
> been able to make NetworkManager pick up the changes on RH systems so far.<br>
<br>
So we do use NetworkManager in the OpenDev images, and we do not see<br>
NetworkManager starting before glean.<br></blockquote><div><br></div><div>Okay, thanks for confirming. Maybe it's related to how IPA is built? It's not exactly a normal image after all, although it's pretty close to one.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
The way it should work is that simple-init in dib installs glean to<br>
the image. That runs the glean install script (use --use-nm argument<br>
if DIB_SIMPLE_INIT_NETWORKMANAGER, which is default on centos/fedora)<br>
which installs two things; udev rules and a systemd handler.<br></blockquote><div><br></div><div>I have checked that these are installed, but I don't know how to verify a udev rule.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
The udev is pretty simple [1] and should add a "Wants" for each net<br>
device; e.g. eth1 would match and create a Wants glean@eth1.service,<br>
which then matches [2] which should write out the ifcfg config file.<br>
After this, NetworkManager should start, notice the config file for<br>
the interface and bring it up.<br></blockquote><div><br></div><div>Yeah, I definitely see logging from NetworkManager DHCP before this service is run (i.e. before the output from Glean).<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> Do you maybe have any hints how to proceed? I'd be curious to know how<br>
> static IP assignment works in the infra setup. Do you have images with<br>
> NetworkManager there? Do you use the simple-init element?<br>
<br>
As noted yes we use this. Really only in two contexts; it's Rackspace<br>
that doesn't have DHCP so we have to setup the interface statically<br>
from the configdrive data. Other clouds all provide DHCP, which is<br>
used when there's no configdrive data.<br>
<br>
Here is a systemd-analyze from one of our Centos nodes if it helps: <br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
graphical.target @18.403s<br>
└─multi-user.target @18.403s<br>
└─unbound.service @5.467s +12.934s<br>
└─network.target @5.454s<br>
└─NetworkManager.service @5.339s +112ms<br>
└─network-pre.target @5.334s<br>
└─glean@ens3.service @4.227s +1.102s<br>
└─basic.target @4.167s<br>
└─sockets.target @4.166s<br>
└─iscsiuio.socket @4.165s<br>
└─sysinit.target @4.153s<br>
└─systemd-udev-settle.service @1.905s +2.245s<br>
└─systemd-udev-trigger.service @1.242s +659ms<br>
└─systemd-udevd-control.socket @1.239s<br>
└─system.slice<br></blockquote><div><br></div><div># systemd-analyze critical-chain</div><div>multi-user.target @2min 6.301s<br>└─tuned.service @1min 32.273s +34.024s<br> └─network.target @1min 31.590s<br> └─network-pre.target @1min 31.579s<br> └─glean@enp1s0.service @36.594s +54.952s<br> └─system-glean.slice @36.493s<br> └─system.slice @4.083s<br> └─-.slice @4.080s<br></div><div><br></div><div># systemd-analyze critical-chain NetworkManager.service<br>NetworkManager.service +9.287s<br>└─network-pre.target @1min 31.579s<br> └─glean@enp1s0.service @36.594s +54.952s<br> └─system-glean.slice @36.493s<br> └─system.slice @4.083s<br> └─-.slice @4.080s<br></div><div><br></div><div># cat /etc/sysconfig/network-scripts/ifcfg-enp1s0 <br># Automatically generated, do not edit<br>DEVICE=enp1s0<br>BOOTPROTO=static<br>HWADDR=52:54:00:1f:79:7e<br>IPADDR=192.168.122.42<br>NETMASK=255.255.255.0<br>ONBOOT=yes<br>NM_CONTROLLED=yes</div><div><br></div><div># ip addr<br>...<br>2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000<br> link/ether 52:54:00:1f:79:7e brd ff:ff:ff:ff:ff:ff<br> inet <a href="http://192.168.122.77/24">192.168.122.77/24</a> brd 192.168.122.255 scope global dynamic noprefixroute enp1s0<br> valid_lft 42957sec preferred_lft 42957sec<br> inet6 fe80::f182:7fb4:7a39:eb7b/64 scope link noprefixroute <br> valid_lft forever preferred_lft forever<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
At a guess; I feel like the udev bits are probably not happening<br>
correctly in your case? That's important to get the glean@<interface><br>
service in the chain to pre-create the config file<br></blockquote><div><br></div><div>It seems that the ordering is correct and the interface service is executed, but the IP address is nonetheless wrong.</div><div><br></div><div>Can it be related to how long glean takes to run in my case (54 seconds vs 1 second in your case)?</div><div><br></div><div>Dmitry<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
-i<br>
<br>
[1] <a href="https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-udev.rules" rel="noreferrer" target="_blank">https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-udev.rules</a><br>
[2] <a href="https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-nm@.service" rel="noreferrer" target="_blank">https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-nm@.service</a><br>
<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Red Hat GmbH, <a href="https://de.redhat.com/" target="_blank">https://de.redhat.com/</a> , Registered seat: Grasbrunn, <br>Commercial register: Amtsgericht Muenchen, HRB 153243,<br>Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill <br></div></div></div>