[Vitrage] Cannot delete old entities on graph
Hi, I’ve deployed Kolla-Ansible (Victoria) and after using it for some time I have errors from Vitrage service: 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init [-] Got Exception for event {'vitrage_entity_type': 'consistency', 'vitrage_datasource_action': 'update', 'vitrage_sample_date': '2021-03-18T13:30:19Z', 'vitrage_event_type': 'delete_entity', 'vitrage_id': 'a19330f1-49c1-4a97-9787-c80b645f51a6', 'id': '93c1df77-dbb5-4ba7-9b7d-16fad97169f1', 'vitrage_type': 'cinder.volume', 'vitrage_category': 'RESOURCE', 'is_real_vitrage_id': True}: KeyError: 'vitrage_is_placeholder' 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init Traceback (most recent call last): 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/graph_init.py", line 146, in do_work 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init return do_work_func(event) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/graph_init.py", line 111, in process_event 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init self.processor.process_event(event) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor.py", line 63, in process_event 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init self.actions[entity.action](entity.vertex, entity.neighbors) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor.py", line 149, in delete_entity 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init PUtils.delete_placeholder_vertex(self.entity_graph, vertex) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor_utils.py", line 68, in delete_placeholder_vertex 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init if not vertex[VProps.VITRAGE_IS_PLACEHOLDER]: 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/graph/driver/elements.py", line 24, in __getitem__ 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init return self.properties[key] 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init KeyError: 'vitrage_is_placeholder' 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init As I can see in Horizon, there are some old volumes , which in Cinder database have proper (DELETED/DETACHED) status, but they’re still visible on Entity Graph, and as in error message above - cannot be deleted by vitrage-graph. How to delete such entities? Best regards, Adam Tomas
Hi As a workaround stop the vitrage graph process run vitrage-purge-data and start vitrage graph again Eyal On Thu, Mar 18, 2021, 17:10 Adam Tomas <bkslash@poczta.onet.pl> wrote:
Hi, I’ve deployed Kolla-Ansible (Victoria) and after using it for some time I have errors from Vitrage service:
2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init [-] Got Exception for event {'vitrage_entity_type': 'consistency', 'vitrage_datasource_action': 'update', 'vitrage_sample_date': '2021-03-18T13:30:19Z', 'vitrage_event_type': 'delete_entity', 'vitrage_id': 'a19330f1-49c1-4a97-9787-c80b645f51a6', 'id': '93c1df77-dbb5-4ba7-9b7d-16fad97169f1', 'vitrage_type': 'cinder.volume', 'vitrage_category': 'RESOURCE', 'is_real_vitrage_id': True}: KeyError: 'vitrage_is_placeholder' 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init Traceback (most recent call last): 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/graph_init.py", line 146, in do_work 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init return do_work_func(event) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/graph_init.py", line 111, in process_event 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init self.processor.process_event(event) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor.py", line 63, in process_event 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init self.actions[entity.action](entity.vertex, entity.neighbors) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor.py", line 149, in delete_entity 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init PUtils.delete_placeholder_vertex(self.entity_graph, vertex) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor_utils.py", line 68, in delete_placeholder_vertex 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init if not vertex[VProps.VITRAGE_IS_PLACEHOLDER]: 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/graph/driver/elements.py", line 24, in __getitem__ 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init return self.properties[key] 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init KeyError: 'vitrage_is_placeholder' 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init
As I can see in Horizon, there are some old volumes , which in Cinder database have proper (DELETED/DETACHED) status, but they’re still visible on Entity Graph, and as in error message above - cannot be deleted by vitrage-graph. How to delete such entities?
Best regards, Adam Tomas
Hi - thanks, that worked :) Actually I was using vitrage-purge-data, but I didn’t stop/start Vitrage service (I use Kolla, so Vitrage is inside docker container and stopping the process would of course stop the container. Instead now I did vitrage-purge-data&& kill $VITRAGE_MASTER_PID). Thanks again! Best regards, Adam
Wiadomość napisana przez Eyal B <eyalb1@gmail.com> w dniu 18.03.2021, o godz. 21:56:
Hi
As a workaround stop the vitrage graph process run vitrage-purge-data and start vitrage graph again
Eyal
On Thu, Mar 18, 2021, 17:10 Adam Tomas <bkslash@poczta.onet.pl <mailto:bkslash@poczta.onet.pl>> wrote: Hi, I’ve deployed Kolla-Ansible (Victoria) and after using it for some time I have errors from Vitrage service:
2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init [-] Got Exception for event {'vitrage_entity_type': 'consistency', 'vitrage_datasource_action': 'update', 'vitrage_sample_date': '2021-03-18T13:30:19Z', 'vitrage_event_type': 'delete_entity', 'vitrage_id': 'a19330f1-49c1-4a97-9787-c80b645f51a6', 'id': '93c1df77-dbb5-4ba7-9b7d-16fad97169f1', 'vitrage_type': 'cinder.volume', 'vitrage_category': 'RESOURCE', 'is_real_vitrage_id': True}: KeyError: 'vitrage_is_placeholder' 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init Traceback (most recent call last): 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/graph_init.py", line 146, in do_work 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init return do_work_func(event) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/graph_init.py", line 111, in process_event 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init self.processor.process_event(event) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor.py", line 63, in process_event 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init self.actions[entity.action](entity.vertex, entity.neighbors) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor.py", line 149, in delete_entity 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init PUtils.delete_placeholder_vertex(self.entity_graph, vertex) 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/entity_graph/processor/processor_utils.py", line 68, in delete_placeholder_vertex 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init if not vertex[VProps.VITRAGE_IS_PLACEHOLDER]: 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init File "/var/lib/kolla/venv/lib/python3.8/site-packages/vitrage/graph/driver/elements.py", line 24, in __getitem__ 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init return self.properties[key] 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init KeyError: 'vitrage_is_placeholder' 2021-03-18 13:30:19.873 7 ERROR vitrage.entity_graph.graph_init
As I can see in Horizon, there are some old volumes , which in Cinder database have proper (DELETED/DETACHED) status, but they’re still visible on Entity Graph, and as in error message above - cannot be deleted by vitrage-graph. How to delete such entities?
Best regards, Adam Tomas
Hi, after deploying Vitrage in multi region environment Horizon always uses first public Vitrage endpoint (I have one public endpoint in each region) regardless if I’m logged in first or second region. So in both regions I see exactly the same entity map, etc. (from the first region). When I disable this endpoint, Horizon uses next one - and again I see the same things in both regions but this time from the second region. Horizon should check in which region I’m logged in and display Vitrage data for that region - right? So what’s wrong? Best regards Adam Tomas
Hi, Currently Vitrage supports only one region to get the data from. The region is configured in vitrage.conf under section [service_credentials] and is used by its clients to get the data Eyal On Mon, Apr 26, 2021 at 5:50 PM Adam Tomas <bkslash@poczta.onet.pl> wrote:
Hi, after deploying Vitrage in multi region environment Horizon always uses first public Vitrage endpoint (I have one public endpoint in each region) regardless if I’m logged in first or second region. So in both regions I see exactly the same entity map, etc. (from the first region). When I disable this endpoint, Horizon uses next one - and again I see the same things in both regions but this time from the second region. Horizon should check in which region I’m logged in and display Vitrage data for that region - right? So what’s wrong?
Best regards Adam Tomas
Hi Eyal, thank you for your answer. So it’s pointless to install Vitrage in a region other than „parent” region :( Is there any way to prevent users (other than admin, let’s say with member role) from seeing „Vitrage” menu in Horizon? If I put role:admin in horizon’s vitrage_policy.json file (for all options) would it make the Vitrage menu disappear for users other than admin? Best regards, Adam
Wiadomość napisana przez Eyal B <eyalb1@gmail.com> w dniu 26.04.2021, o godz. 17:35:
Hi,
Currently Vitrage supports only one region to get the data from. The region is configured in vitrage.conf under section [service_credentials] and is used by its clients to get the data
Eyal
On Mon, Apr 26, 2021 at 5:50 PM Adam Tomas <bkslash@poczta.onet.pl <mailto:bkslash@poczta.onet.pl>> wrote:
Hi, after deploying Vitrage in multi region environment Horizon always uses first public Vitrage endpoint (I have one public endpoint in each region) regardless if I’m logged in first or second region. So in both regions I see exactly the same entity map, etc. (from the first region). When I disable this endpoint, Horizon uses next one - and again I see the same things in both regions but this time from the second region. Horizon should check in which region I’m logged in and display Vitrage data for that region - right? So what’s wrong?
Best regards Adam Tomas
HI, Vitrage is supposed to support multi-tenancy so the tenant in horizon should see only its own resources. Regarding horizon vitrage menu, I don't know, maybe you can write some code to horizon that can disable the vitrage menu for certain users. You need to consult with the horizon guys for that if there is an option for that. Eyal On Tue, Apr 27, 2021 at 9:57 AM Adam Tomas <bkslash@poczta.onet.pl> wrote:
Hi Eyal, thank you for your answer. So it’s pointless to install Vitrage in a region other than „parent” region :( Is there any way to prevent users (other than admin, let’s say with member role) from seeing „Vitrage” menu in Horizon? If I put role:admin in horizon’s vitrage_policy.json file (for all options) would it make the Vitrage menu disappear for users other than admin?
Best regards, Adam
Wiadomość napisana przez Eyal B <eyalb1@gmail.com> w dniu 26.04.2021, o godz. 17:35:
Hi,
Currently Vitrage supports only one region to get the data from. The region is configured in vitrage.conf under section [service_credentials] and is used by its clients to get the data
Eyal
On Mon, Apr 26, 2021 at 5:50 PM Adam Tomas <bkslash@poczta.onet.pl> wrote:
Hi, after deploying Vitrage in multi region environment Horizon always uses first public Vitrage endpoint (I have one public endpoint in each region) regardless if I’m logged in first or second region. So in both regions I see exactly the same entity map, etc. (from the first region). When I disable this endpoint, Horizon uses next one - and again I see the same things in both regions but this time from the second region. Horizon should check in which region I’m logged in and display Vitrage data for that region - right? So what’s wrong?
Best regards Adam Tomas
Thanks again. In kolla there is an option to completely disable Horizon’s Vitrage menu (in globals.yaml: enable_horizon_vitrage, which sets env variable disabling or enabling menu globally for all users), maybe policy will do the trick, but it requires a lot of testing, that’s why I was searching for quicker solution ;) Best regards Adam
Wiadomość napisana przez Eyal B <eyalb1@gmail.com> w dniu 27.04.2021, o godz. 09:26:
HI,
Vitrage is supposed to support multi-tenancy so the tenant in horizon should see only its own resources. Regarding horizon vitrage menu, I don't know, maybe you can write some code to horizon that can disable the vitrage menu for certain users. You need to consult with the horizon guys for that if there is an option for that.
Eyal
On Tue, Apr 27, 2021 at 9:57 AM Adam Tomas <bkslash@poczta.onet.pl <mailto:bkslash@poczta.onet.pl>> wrote: Hi Eyal, thank you for your answer. So it’s pointless to install Vitrage in a region other than „parent” region :( Is there any way to prevent users (other than admin, let’s say with member role) from seeing „Vitrage” menu in Horizon? If I put role:admin in horizon’s vitrage_policy.json file (for all options) would it make the Vitrage menu disappear for users other than admin?
Best regards, Adam
Wiadomość napisana przez Eyal B <eyalb1@gmail.com <mailto:eyalb1@gmail.com>> w dniu 26.04.2021, o godz. 17:35:
Hi,
Currently Vitrage supports only one region to get the data from. The region is configured in vitrage.conf under section [service_credentials] and is used by its clients to get the data
Eyal
On Mon, Apr 26, 2021 at 5:50 PM Adam Tomas <bkslash@poczta.onet.pl <mailto:bkslash@poczta.onet.pl>> wrote:
Hi, after deploying Vitrage in multi region environment Horizon always uses first public Vitrage endpoint (I have one public endpoint in each region) regardless if I’m logged in first or second region. So in both regions I see exactly the same entity map, etc. (from the first region). When I disable this endpoint, Horizon uses next one - and again I see the same things in both regions but this time from the second region. Horizon should check in which region I’m logged in and display Vitrage data for that region - right? So what’s wrong?
Best regards Adam Tomas
participants (2)
-
Adam Tomas
-
Eyal B