Open Stack

Hi Yujun,

A relationship is defined by the following trio: source id, target id and a label on the edge.
In the processor, I add only edges that doesn't exists.

Alexey

From: Yujun Zhang [mailto:zhangyujun+zte at gmail.com] 
Sent: Thursday, January 05, 2017 4:32 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [ALU] Re: [openstack-dev] [ALU] Re: [ALU] Re: [ALU] [vitrage] how touseplaceholder vertex

A follow up question on relationships.
On Thu, Jan 5, 2017 at 9:59 PM Weyl, Alexey (Nokia - IL) <alexey.weyl at nokia.com> wrote:
Hi Yujun,

Lets see.

1. There is no need for the transformer to handle this duplication. What will happen at the moment is that we will receive twice every neighbor, and it is fine by us, because it is a quite small datasource, and 99.999% of the time it won't be changed.

It's fine for neighbor because vertex can be identified by id and there won't be duplication. 

But what about relationship, how do we model redundant links between two entities? There seems to be no id for relationships.

2. It should be 2 events. We want to make it as simple as possible, and in the same time as flexible as possible. So you should create 2 events and each one will have the neighbor connection.

Hope it answers everything.

BR,
Alexey



From: Yujun Zhang [mailto:zhangyujun+zte at gmail.com]
Sent: Thursday, January 05, 2017 2:32 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [ALU] Re: [openstack-dev] [ALU] Re: [ALU] [vitrage] how to useplaceholder vertex

Alexey,

I have to dig this old thread to clarify some issues I met during static datasource implementation. Hope that you can still recall the context :-)

I'll try to simplify this question with an example. The following configuration are snippet from static datasource

1. suppose we have three switches linked in a ring. What would be the expected entity events emit by the driver?

In my proposed driver, there will be three entities. And each relationship will appear both in source entity and target entity, e.g. s1->s2 will be included in both s1 and s2. Should the transformer handle this duplication or the graph utils will?
entities:
  - config_id: s1
    type: switch
    name: switch-1
    id: 12345
    state: available
  - config_id: s2
    type: switch
    name: switch-2
    id: 23456
    state: available
  - config_id: s3
    type: switch
    name: switch-3
    id: 34567
    state: available
relationships:
  - source: s1
    target: s2
    relation_type: linked
  - source: s2
    target: s3
    relation_type: linked
  - source: s3
    target: s1
    relation_type: linked
2. suppose we created a link between switch and nova.host. What will be the expected entity events? Should it be one entity event of s1 with h1 embedded as neighbor? Or two entity events, s1 and h1?
entities:
  - config_id: s1
    type: switch
    name: switch-1
    id: 12345
    state: available
  - config_id: h1
    type: nova.host
    id: 1
relationships:
  - source: s1
    target: h1
    relation_type: attached

On Wed, Dec 14, 2016 at 11:54 PM Weyl, Alexey (Nokia - IL) <alexey.weyl at nokia.com> wrote:
1. That is correct.

2. That is not quite correct.
In static we only define the main properties of each entity, meaning type, id, category and thus it is ok that for each main entity we will create its neighbors and connect between them. There is no need for any distinguish due to that.


From: Yujun Zhang [mailto:zhangyujun+zte at gmail.com]
Sent: Wednesday, December 14, 2016 5:00 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [ALU] Re: [openstack-dev] [ALU] [vitrage] how to use placeholder vertex

Hi, Alexey,

Thanks for the detail example. It explains the existing mechanism of vertex creation well.

So it looks like each resource type will have a primary datasource, e.g. nova.host for nova.host, nova.intance for nova.instance, that holds full details. Is that correct?

Not sure that you remember the long discussion in static driver review[1] or not. At last, we agreed on a unified entity definition for both `nova.host` and `switch`, no extra key to indicate it is "external" (should create a placeholder).

If I understand it correctly, no placeholder will be created in this case. Because we can not distinguish them from the static configuration. And the properties of `nova.host` resource shall to be merged from `static` and nova.host` datasources. Is that so?

[1]: https://review.openstack.org/#/c/405354/  

On Wed, Dec 14, 2016 at 5:40 PM Weyl, Alexey (Nokia - IL) <alexey.weyl at nokia.com> wrote:
Hi Yujun,
 
This is a good question, and let me explain for you how it works.
Lets say we are supposed to get 2 entities from nova, nova.host called host1 and nova.instance called vm1 and vm1 is supposed to be connected to host1.
The nova.host driver and nova.instance driver are working simultaneously and thus we don’t know the order in which those events will arrive.
We have 2 use cases:
1.       Host1 event arrives before vm1.
In this case the processor will call the transformer of nova.host and will create a vertex for host1 in the graph with the full details of host1.
Then, vm1 event will arrive, the processor will create the vm1 vertex in the graph, it will update the host1 properties in the graph (because the host1 details that are created in nova.instance are only its basic details such as: category, type, id, is_deleted, is_placeholder they host1 properties won’t be changed in the graph because those details are basic and can’t be changed), and then it will create an edge between vm1 and host1 (only the nova.instance knows to which nova.host it is connected and not vice versa).
2.       Vm1 event arrives before host1.
In this case the processor will add vm1 to the graph, then it will add the host1 placeholder to the graph (so we will know to which host vm1 is connected) and then add the edge between them.
Then when the processor will handle with the host1 event, it will just add some properties of the host1 vertex, and of course will change the is_placeholder property of host1 to false.
We also has the consistency service that runs every 10 minutes (its configurable with the snapshot_interval) and checks if there are vertices that are is_placeholder=True and are in the graph more then 2*snapshot_interval time then it means that such a vertex of host1 for example doesn’t really exists and it deletes it from the graph).
In addition, when we understand that we need to delete some entity from the graph, upon delete notification for example, then we don’t delete it right away, we change the is_deleted property of that entity to True, and we will delete it on the next run of the consistency service. The reason we do that, is because we need it for the evaluator, because it runs a subgraph matching algorithm on the graph, and if the entity that is supposed to be there doesn’t appear then it won’t work correctly.
 
The best way to create a placeholder is to call the placeholder method of the correct transformer using the transformers array that each transformer class has.
 
I hope this helps.
 
Alexey
 
From: Yujun Zhang [mailto:zhangyujun+zte at gmail.com]
Sent: Wednesday, December 14, 2016 10:55 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [ALU] [openstack-dev] [vitrage] how to use placeholder vertex
 
Hi, root causers (assuming this nickname has been agreed)
 
It seems the placeholder is created for every neighbor of an entity[1]. I'm not sure what will happen if the neighbor entity has been created before. 
 
Will the graph utility handle it? Or we need to check the existence in transformer and deal with it?
 
Similar question for how to create a vertex when there is already a placeholder with same ID
 
[1]: https://github.com/openstack/vitrage/blob/master/vitrage/datasources/static_physical/transformer.py#L114 
--
Yujun
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev