On Tue, Nov 24, 2020 at 11:54:55AM +0100, Dmitry Tantsur wrote:
> The problem is, I cannot make Glean work with any ramdisk I build. The crux
> of the problem seems to be that NetworkManager (used by default in RHEL,
> CentOS, Fedora and Debian at least) starts very early, creates the default
> connection and ignores whatever files Glean happens to write afterwards. On
> Debian running `systemctl restart networking` actually helped to pick the
> new configuration, but I'm not sure we want to do that in Glean. I haven't
> been able to make NetworkManager pick up the changes on RH systems so far.

So we do use NetworkManager in the OpenDev images, and we do not see
NetworkManager starting before glean.

The way it should work is that simple-init in dib installs glean to
the image.  That runs the glean install script (use --use-nm argument
if DIB_SIMPLE_INIT_NETWORKMANAGER, which is default on centos/fedora)
which installs two things; udev rules and a systemd handler.

The udev is pretty simple [1] and should add a "Wants" for each net
device; e.g. eth1 would match and create a Wants glean@eth1.service,
which then matches [2] which should write out the ifcfg config file.
After this, NetworkManager should start, notice the config file for
the interface and bring it up.

> Do you maybe have any hints how to proceed? I'd be curious to know how
> static IP assignment works in the infra setup. Do you have images with
> NetworkManager there? Do you use the simple-init element?

As noted yes we use this.  Really only in two contexts; it's Rackspace
that doesn't have DHCP so we have to setup the interface statically
from the configdrive data.  Other clouds all provide DHCP, which is
used when there's no configdrive data.

Here is a systemd-analyze from one of our Centos nodes if it helps:

graphical.target @18.403s
└─multi-user.target @18.403s
  └─unbound.service @5.467s +12.934s
    └─network.target @5.454s
      └─NetworkManager.service @5.339s +112ms
        └─network-pre.target @5.334s
          └─glean@ens3.service @4.227s +1.102s
            └─basic.target @4.167s
              └─sockets.target @4.166s
                └─iscsiuio.socket @4.165s
                  └─sysinit.target @4.153s
                    └─systemd-udev-settle.service @1.905s +2.245s
                      └─systemd-udev-trigger.service @1.242s +659ms
                        └─systemd-udevd-control.socket @1.239s
                          └─system.slice

At a guess; I feel like the udev bits are probably not happening
correctly in your case?  That's important to get the glean@<interface>
service in the chain to pre-create the config file

-i

[1] https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-udev.rules
[2] https://opendev.org/opendev/glean/src/branch/master/glean/init/glean-nm@.service