[openstack-dev] [ironic][neutron] SmartNics with Ironic

Isaku Yamahata isaku.yamahata at gmail.com
Mon Oct 1 19:28:48 UTC 2018

Hello. I'm willing to help it.

For detailed tech discussion, gerrit with updated spec would be better, though.
I'd like to supplement on Neutron port binding.

On Sun, Sep 30, 2018 at 06:25:58AM +0000,
Moshe Levi <moshele at mellanox.com> wrote:

> Hi Julia,
> I don't mind to update the ironic spec [1]. Unfortunately, I wasn't in the PTG but I had a sync meeting with Isuku.
> As I see it there is 2 use-cases:
>   1.  Running the neutron ovs agent in the smartnic
>   2.  Running the neutron super ovs agent which manage the ovs running on the smartnic.
> It seem that most of the discussion was around the second use-case.
> This is my understanding on the ironic neutron PTG meeting:
>   1.  Ironic cores don't want to change the deployment interface as proposed in [1].
>   2.  We should  a new network_interface for use case 2. But what about the first use case? Should it be a new network_interface as well?

  * Which component, Ironic or Neutron, should take care that
    SmartNIC is up/ready?
  * How is the up/readiness of smartnic defined?
    Common way or device-dependent way. For example,
    - able to connect via ovsdb/openflow
    - agent responsible for smartNIC has reported to Neutron agentdb.
    - able to ssh into smartnic
    - device specific way with driver for each device.
    - etc.

>   3.  We should delay the port binding until the baremetal is powered on the ovs is running.
>      *   For the first use case I was thinking to change the neutron server to just keep the port binding information in the neutron DB. Then when the neutron ovs agent is a live it will retrieve all the baremeal port , add them to the ovsdb and start the neutron ovs agent fullsync.
>      *   For the second use case the agent is alive so the agent itself can monitor the ovsdb of the baremetal and configure it when it up

Can you please elaborate why port binding for smartnic should be delayed?
I'm failing to see the necessity. Probably I'm missing something.
Since we can skip the check of agent liveness with the assumption that
hostid given by Ironic is correct, we don't have to delay the port binding
for both case, 1 and 2 as above.

>   4.  How to notify that neutron agent successfully/unsuccessfully bind the port ?
>      *   In both use-cases we should use neutron-ironic notification to make sure the port binding was done successfully.

I agree that neutron-ironic notification is necessary.

There seems the confusion of port-binding and ovs-programing. The
success/failure of port-binding is the result of neutron port-update.
The current code synchronously checks the ovs-agent liveness on the host and
parameter validity. port-binding doesn't directly/synchronously program ovs.
It only triggers to start ovs programming eventually.

On the other hand, the success/failure of ovs programming is
asynchronous regarding to neutron REST API because ovs-agent does it
asynchronously on behalf of neutron server.
So here neutron-ironic notification is necessary.
When the ovs programming is done, the agent sets port::status = UP from DOWN.
(and nova is notified that port is ready to use.)
In the case of the failure, port::status is set to ERROR.


> Is my understanding correct?
> [1] - https://review.openstack.org/#/c/582767/
> From: Miguel Lavalle <miguel at mlavalle.com>
> Sent: Sunday, September 30, 2018 3:20 AM
> To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic
> Hi,
> Yes, this also matches the recollection of the joint conversation in Denver. Please look at the "Ironic x-project discussion - Smartnics" section in http://lists.openstack.org/pipermail/openstack-dev/2018-September/135032.html<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openstack.org%2Fpipermail%2Fopenstack-dev%2F2018-September%2F135032.html&data=02%7C01%7Cmoshele%40mellanox.com%7Ce5607928a41c44bf065508d6266a9435%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636738636550233328&sdata=ZvAsZoTkk%2FFGlsig54J0d1EejcDiYdqsGlfRkuKBJTg%3D&reserved=0>
> Regards
> Miguel
> On Thu, Sep 27, 2018 at 1:31 PM Julia Kreger <juliaashleykreger at gmail.com<mailto:juliaashleykreger at gmail.com>> wrote:
> Greetings everyone,
> Now that the PTG is over, I would like to go ahead and get the
> specification that was proposed to ironic-specs updated to represent
> the discussions that took place at the PTG.
> A few highlights from my recollection:
> * Ironic being the source of truth for the hardware configuration for
> the neutron agent to determine where to push configuration to. This
> would include the address and credential information (certificates
> right?).
> * The information required is somehow sent to neutron (possibly as
> part of the binding profile, which we could send at each time port
> actions are requested by Ironic.)
> * The Neutron agent running on the control plane connects outbound to
> the smartnic, using information supplied to perform the appropriate
> network configuration.
> * In Ironic, this would likely be a new network_interface driver
> module, with some additional methods that help facilitate the
> work-flow logic changes needed in each deploy_interface driver module.
> * Ironic would then be informed or gain awareness that the
> configuration has been completed and that the deployment can proceed.
> (A different spec has been proposed regarding this.)
> I have submitted a forum session based upon this and the agreed upon
> goal at the PTG was to have the ironic spec written up to describe the
> required changes.
> I guess the next question is, who wants to update the specification?
> -Julia
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FOpenStack-dev-request%40lists.openstack.org%3Fsubject%3Aunsubscribe&data=02%7C01%7Cmoshele%40mellanox.com%7Ce5607928a41c44bf065508d6266a9435%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636738636550233328&sdata=PexKWkCH7u4kRWjzs2kOIZsHFmzSL%2BCl6bqzY2B5KWA%3D&reserved=0>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fopenstack-dev&data=02%7C01%7Cmoshele%40mellanox.com%7Ce5607928a41c44bf065508d6266a9435%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636738636550243338&sdata=dHTuFDlyKrA8ouPU7JV4GKokCKNBg%2Fzw9KlpIrZW5x0%3D&reserved=0>

> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Isaku Yamahata <isaku.yamahata at gmail.com>

More information about the OpenStack-dev mailing list