Open Stack

Hi Duc,

many thanks for your feedback.

On 17/05/2021 18:08, Duc Truong wrote:
> H Giacomo,
>
> The patch set that you linked is 4 years old, so I don't remember what
> the issue was with implementing the LB poll in the Senlin health
> check.  I don't see a problem with implementing that feature so feel
> free to check the contribution guide and submit some code.  A few
> things to think about are:
>
> - Would the LB_STATUS_POLLING require the user to use the Senlin Load
> Balancer as well?

Yes, I am assuming that this feature could only work when using the LB 
policy as well.

> - How would Senlin know which health monitor to check for a given Senlin node?
>
> A common use case for Senlin is autoscaling, so nodes will be added or
> deleted on the fly.  Most likely that requires that the
> LB_STATUS_POLLING needs to be tied to the Senlin LB policy that is
> responsible for adding/removing LB pools and LB health monitors.  If
> we go down that route, it is easy to retrieve the LB health monitor ID
> because that is stored in the Senlin node properties.

It looks like the LB health monitor ID is not actually stored in the 
node properties. At least, it doesn't show up when looking at node 
details via CLI:

```
$ openstack cluster node show node-uHgcqJz9
+---------------+---------------------------------------------------------------
| Field         | Value
+---------------+---------------------------------------------------------------
| cluster_id    | 6cd9973c-0387-4fdf-a20c-4471ec905c67
| created_at    | 2021-05-13T11:28:46Z
| data          | {
|               |   "internal_ports": [
|               | {
|               |       "fixed_ips": [
|               | {
|               |           "ip_address": "10.0.0.175",
|               |           "subnet_id": 
"2867ceb3-240d-4694-b177-57d521deec4d"
|               | }
|               | ],
|               |       "id": "1ab76390-3301-4caa-b4da-7a0f2947a40a",
|               |       "network_id": 
"cf177807-7da1-4312-9f41-3ba7de0d5cc7",
|               |       "remove": true,
|               |       "security_group_ids": [
|               | "699b230a-2ddd-4d28-9cb2-95c83ca3e2a8"
|               | ]
|               | }
|               | ],
|               |   "lb_member": "a11c5118-4d85-4c5f-b4ee-47bf32482f19",
|               |   "placement": {
|               |     "zone": "nova"
|               | }
|               | }
[...]
```

However, it should be feasible to get nodes' health status, as detected 
by the LB, in this way:

1) Fetch the LB policy attached to the cluster:

```
$ openstack cluster policy binding list 6cd9973c --filter 
"policy_type=senlin.policy.loadbalance-1.3"
+-----------+--------------------------------------+-------------------------------+------------+
| policy_id | policy_name                          | 
policy_type                   | is_enabled |
+-----------+--------------------------------------+-------------------------------+------------+
| 70b95b84  | senlin-lb_policy-bifia6bjfn4h        | 
senlin.policy.loadbalance-1.3 | True       |
+-----------+--------------------------------------+-------------------------------+------------+
```

2) Get the LB ID from the policy spec:

```
$ openstack cluster policy show 70b95b84
+------------+-------------------------------------------------------------
| Field      | Value
+------------+-------------------------------------------------------------
[...]
| spec       | {
|            |   "properties": {
|            |     "health_monitor": {
|            |       "id": "db7c9243-1bd7-4ba1-b7ff-0be1d410c0de",
|            |       "type": "TCP"
|            | },
|            |     "lb_status_timeout": 300,
|            |     "loadbalancer": "d228e578-a413-44d7-96f5-09663402e40a",
|            |     "pool": {
|            |       "id": "a2d3023d-956f-4ee8-af5e-ab852769675f",
|            |       "lb_method": "ROUND_ROBIN",
|            |       "protocol": "TCP",
|            |       "protocol_port": 5001,
|            |       "subnet": "self-serv-subnet-1"
|            | },
|            |     "vip": {
|            |       "protocol": "TCP",
|            |       "protocol_port": 5001,
|            |       "subnet": "provider-subnet"
|            | }
|            | },
|            |   "type": "senlin.policy.loadbalance",
|            |   "version": "1.3"
|            | }
[...]
```

3) For each LB member, get the value of "operating_status" property:

```
$ openstack loadbalancer status show d228e578-a413-44d7-96f5-09663402e40a
{
     "loadbalancer": {
         [...]
         "listeners": [
             {
                 [...]
                 "pools": [
                     {
                         [...]
                         "health_monitor": {
                             [...]
                         },
                         "members": [
                             {
                                 "id": 
"a11c5118-4d85-4c5f-b4ee-47bf32482f19",
                                 "name": "",
                                 "operating_status": "ONLINE",
                                 "provisioning_status": "ACTIVE",
                                 "address": "10.0.0.175",
                                 "protocol_port": 5001
                             },
                             {
                                 "id": 
"0da3d20b-9fb1-42cd-814d-29812c6c2783",
                                 "name": "",
                                 "operating_status": "ONLINE",
                                 "provisioning_status": "ACTIVE",
                                 "address": "10.0.0.222",
                                 "protocol_port": 5001
                             }
                         ]
                     }
                 ]
             }
         ]
     }
}
```

4) Associate each LB member status to the corresponding node in the 
cluster, by leveraging on the "lb_member" property included in node 
properties.


Such an approach seems sound to you?

Kind regards.

Giacomo


>
> Anyways, just some things to think about.
>
> Duc
>
> On Fri, May 14, 2021 at 2:51 AM Giacomo Lanciano
> <giacomo.lanciano at sns.it> wrote:
>> Hi folks,
>>
>> I'd like to know what is the status of making the LB_STATUS_POLLING
>> detection mode available for the Senlin Health Policy [1]. According to
>> the docs and this patch [2], the implementation of this feature is
>> blocked by some issue on LBaaS/Octavia side, but I could not find any
>> details on what this issue really is.
>>
>> As the docs state, it would be really useful to have this detection mode
>> available, as it is much more reliable than the others at evaluating the
>> status of an application. If I can be of any help, I would be willing to
>> contribute.
>>
>> Thanks in advance.
>>
>> Kind regards.
>>
>> Giacomo
>>
>> [1]
>> https://docs.openstack.org/senlin/latest/contributor/policies/health_v1.html#failure-detection
>> [2] https://review.opendev.org/c/openstack/senlin/+/423012

-- 
Giacomo Lanciano
Ph.D. Student in Data Science
Scuola Normale Superiore, Pisa, Italy
https://www.linkedin.com/in/giacomolanciano