Open Stack

Hi again,
I did some experiment:

in /var/lib/kolla/venv/lib/python3.8/site-packages/monasca_api/api/core/log/validation.py

I changed the default value from 255 to 300:

DIMENSION_VALUE_CONSTRAINTS = {
    'MAX_LENGTH': 300
}

and restarted the container. And now there are no value validation errors, but how to check if this max value do not cause problems in monasca_log_transformer or any other service in the stack (kafka/monasca_persister/elasticsearch)?

Thanks in advance for any help with this issue…

Best regards
Adam Tomas

> Wiadomość napisana przez Adam Tomas <bkslash at poczta.onet.pl> w dniu 02.11.2021, o godz. 15:01:
> 
> Hi,
> 
> in my test deployment I get following error messages for each metric at each metric collection (every 60s in my case). Of course whole URL (i.e. /v2.0/metrics/statistics?name=transfer.region2.size&merge_metrics=True&dimensions=project_id%3Afcfaxxxxx&start_time=2021-10-31+02%3A55%3A00%2B02%3A00&end_time=2021-10-31+02%3A00%3A00%2B01%3A00&period=-3300&statistics=avg&group_by=project_id&group_by=resource_id)  is longer than 255 characters, but shouldn’t this URL be processed in smaller chunks? As I see the validation is done by monasca_api, so in case it fails no data is processed by monasca_log_transformer, right? Limiting metric name lenght won’t change this situation (in this example metric name is 22 characters long, while the whole URL is 285 characters long, so 285-22 is still > 255).
> 
> How can I avoid this error?
> Best regards
> 
> Adam Tomas
> 
> 
> 2021-11-02 14:19:44.935 693 ERROR monasca_api.v2.common.bulk_processor [req-a251xxxx 60bexxxx 27bexxxxx - default default] Log transformation failed, rejecting log: monasca_api.api.core.log.exceptions.HTTPUnprocessableEntity: Dimension value /v2.0/metrics/statistics?name=transfer.region2.size&merge_metrics=True&dimensions=project_id%3Afcfaxxxxx&start_time=2021-10-31+02%3A55%3A00%2B02%3A00&end_time=2021-10-31+02%3A00%3A00%2B01%3A00&period=-3300&statistics=avg&group_by=project_id&group_by=resource_id must be 255 characters or less
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor [req-a251xxxx 60bexxxx 27bexxxx - default default] Dimension value /v2.0/metrics/statistics?name=transfer.region2.size&merge_metrics=True&dimensions=project_id%3Afcfaxxx&start_time=2021-10-31+02%3A55%3A00%2B02%3A00&end_time=2021-10-31+02%3A00%3A00%2B01%3A00&period=-3300&statistics=avg&group_by=project_id&group_by=resource_id must be 255 characters or less: monasca_api.api.core.log.exceptions.HTTPUnprocessableEntity: Dimension value /v2.0/metrics/statistics?name=transfer.region2.size&merge_metrics=True&dimensions=project_id%3Afcfaxxxx&start_time=2021-10-31+02%3A55%3A00%2B02%3A00&end_time=2021-10-31+02%3A00%3A00%2B01%3A00&period=-3300&statistics=avg&group_by=project_id&group_by=resource_id must be 255 characters or less
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor Traceback (most recent call last):
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor   File "/var/lib/kolla/venv/lib/python3.8/site-packages/monasca_api/v2/common/bulk_processor.py", line 81, in _transform_message
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor     dimensions=self._get_dimensions(log_element,
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor   File "/var/lib/kolla/venv/lib/python3.8/site-packages/monasca_api/v2/common/bulk_processor.py", line 129, in _get_dimensions
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor     validation.validate_dimensions(local_dims)
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor   File "/var/lib/kolla/venv/lib/python3.8/site-packages/monasca_api/api/core/log/validation.py", line 138, in validate_dimensions
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor     _validate_dimension_value(dim_value)
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor   File "/var/lib/kolla/venv/lib/python3.8/site-packages/monasca_api/api/core/log/validation.py", line 114, in _validate_dimension_value
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor     raise exceptions.HTTPUnprocessableEntity(
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor monasca_api.api.core.log.exceptions.HTTPUnprocessableEntity: Dimension value /v2.0/metrics/statistics?name=transfer.region2.size&merge_metrics=True&dimensions=project_id%3Afcfaxxxx&start_time=2021-10-31+02%3A55%3A00%2B02%3A00&end_time=2021-10-31+02%3A00%3A00%2B01%3A00&period=-3300&statistics=avg&group_by=project_id&group_by=resource_id must be 255 characters or less
> 2021-11-02 14:19:44.936 693 ERROR monasca_api.v2.common.bulk_processor