Open Stack

Sorry to reply you so late, venus could already be installed through devstack, you can try it. 

In addition, we will also consider it in kolla-ansible.

 

发件人: Laurent Dumont <laurentfdumont at gmail.com> 
发送时间: 2021年1月12日 12:57
收件人: Liye Pang(逄立业) <pangliye at inspur.com>
抄送: openstack-discuss at lists.openstack.org
主题: Re: [all]Introduction to venus which is the project of log management and has been contributed to the OpenStack community

 

This seems really interesting. Tracing events with request-ids is something that is quite useful.

 

What is the current state? Can it be deployed by a third party?

 

On Sun, Jan 10, 2021 at 4:01 PM Liye Pang(逄立业) <pangliye at inspur.com <mailto:pangliye at inspur.com> > wrote:

Hello everyone, after feedback from a large number of operations and maintenance personnel in InCloud OpenStack, we developed the log management project “Venus” for the OpenStack projects and that has contributed to the OpenStack community. The following is an introduction to “Venus”. If there is interest in the community, we are interested in proposing it to become an official OpenStack project in the future.


Background


In the day-to-day operation and maintenance of large-scale cloud platform, the following problems are encountered：

l  Time-consuming for log querying while the server increasing to thousands.

l  Difficult to retrieve logs, since there are many modules in the platform, e.g. systems service, compute, storage, network and other platform services.

l  The large amount and dispersion of log make faults are difficult to be discovered.

l  Because of distributed and interaction between components of the cloud platform, and scattered logs between components, it will take more time to locate problems.


About Venus


According to the key requirements of OpenStack in log storage, retrieval, analysis and so on, we introduced Venus project, a unified log management module. This module can provide a one-stop solution to log collection, cleaning, indexing, analysis, alarm, visualization, report generation and other needs, which involves helping operator or maintainer to quickly solve retrieve problems, grasp the operational health of the platform, and improve the management capabilities of the cloud platform. 

Additionally, this module plans to use machine learning algorithms to quickly locate IT failures and root causes, and improve operation and maintenance efficiency.


Application scenario


Venus played a key role in the following scenarios：

l  Retrieval: Provide a simple and easy-to-use way to retrieve all log and the context.

l  Analysis: Realize log association, field value statistics, and provide multi-scene and multi-dimensional visual analysis reports.

l  Alerts：Convert retrieval into active alerts to realize the error finding in massive logs.

l  Issue location: Establish a chain relationship and knowledge graphs to quickly locate problems.


Overall structure


The architecture of log management system based on Venus and elastic search is as follows:



Diagram 0: Architecture of Venus

venus_api: API module，provide API、rest-api service.

venus_manager: Internal timing task module to realize the core functions of the log system.


Current progress


The current progress of the Venus project is as follows:

l  Collection：Develop fluentd collection tasks based on collectd to read, filter, format and send plug-ins for OpenStack, operating systems, and platform services, etc.

l  Index：Dealing with multi-dimensional index data in elasticsearch, and provide more concise and comprehensive authentication interface to return query results.

l  Analysis：Analyzing and display the related module errors, Mariadb connection errors, and Rabbitmq connection errors.

l  Alerts：Develop alarm task code to set threshold for the number of error logs of different modules at different times, and provides alarm services and notification services.

l  Location：Develop the call chain analysis function based on global_requested series, which can show the execution sequence, time and error information, etc., and provide the export operation.

l  Management：Develop configuration management functions in the log system, such as alarm threshold setting, timing task management, and log saving time setting, etc.


Application examples


Two examples of Venus application scenarios are as follows.

1.       The virtual machine creation operation was performed on the cloud platform and it was found that the virtual machine was not created successfully. 

First, we can find the request id of the operation and jump to the virtual machine creation call chain page.





Then, we can query the calling process, view and download the details of the log of the call.





2.       In the cloud platform, the error log of each module can be converted into alarms to remind the users.



Further, we can retrieve the details of the error log and error log statistics.






Next step


The next step of the Venus project is as follows:

*  Collection：In addition to fluent, other collection plugins such as logstash will be integrated.

*  Analysis: Explore more operation and maintenance scenarios, and conduct statistical analysis and alarm on key data.

*  display: The configuration, analysis  and alarm of Venus will be integrated into horizon in the form of plugin.

*  location: Form clustering log and construct knowledge map, and integrate algorithm class library to locate the root cause of the fault. 


Venus Project Registry


Venus library: https://opendev.org/inspur/venus 

You can grab the source code using the following git command:

git clone https://opendev.org/inspur/venus.git

 


Venus Demo 


Youtu.be:  <https://youtu.be/mE2MoEx3awM> https://youtu.be/mE2MoEx3awM

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 24507 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0008.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 3184 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0009.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 8136 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0010.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 15944 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0011.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 8405 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0012.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 3046 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0013.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.jpg
Type: image/jpeg
Size: 15175 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0014.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image008.jpg
Type: image/jpeg
Size: 8496 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0015.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3786 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210317/c889c14e/attachment-0001.bin>