[kolla-ansible][kolla] merge_configs module replacing colon to equal
Hi,
In kolla-ansible, I am using the merge_configs module to merge service configs files. I want to merge api-paste.ini config with my custom changes, however while merging if a line consists of one colon, it is getting replaced with equal sign. merge_configs splits the line by considering colon as delimiter and treats two parts as key and value. Later which will be substituted as key = value.
Original */v2.1: oscomputeversion_v2* Merged */v2.1= oscomputeversion_v2 *
Below, colon will be 1 and equal will be -1,
*def _split_key_value(self, line): colon = line.find(':') equal = line.find('=') if colon < 0 and equal < 0: return self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and equal < colon): key, value = line[:equal], line[equal + 1:] else: key, value = line[:colon], line[colon + 1:] value = value.strip() if value and value[0] == value[-1] and value.startswith((""", "'")): value = value[1:-1] return key.strip(), [value]*
I want to keep the config as it is and merge only my custom changes. Please let me know how to achieve this or colon can be escaped somehow.
Regards, Sharath Regards, Sharath
Hi Sharath,
I had not realised this. Wikipedia suggests colons are sometimes used instead of equals. Perhaps the function should return which delimiter was found, and use it to generate the merged file?
Mark
On Tue, 7 Jun 2022 at 06:19, Sharath Ck sharath.madhava@gmail.com wrote:
Hi,
In kolla-ansible, I am using the merge_configs module to merge service configs files. I want to merge api-paste.ini config with my custom changes, however while merging if a line consists of one colon, it is getting replaced with equal sign. merge_configs splits the line by considering colon as delimiter and treats two parts as key and value. Later which will be substituted as key = value.
Original */v2.1: oscomputeversion_v2* Merged */v2.1= oscomputeversion_v2 *
Below, colon will be 1 and equal will be -1,
*def _split_key_value(self, line): colon = line.find(':') equal = line.find('=') if colon < 0 and equal < 0: return self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and equal < colon): key, value = line[:equal], line[equal + 1:] else: key, value = line[:colon], line[colon + 1:] value = value.strip() if value and value[0] == value[-1] and value.startswith((""", "'")): value = value[1:-1] return key.strip(), [value]*
I want to keep the config as it is and merge only my custom changes. Please let me know how to achieve this or colon can be escaped somehow.
Regards, Sharath Regards, Sharath
Hi Mark,
Yes, that would be correct. If the function returns the delimiter and uses the same while mapping key value, it would keep the config content intact.
If colon can be replaced by equal and vice versa in openstack configs, especially api-paste.ini. I can continue with the merge now and update the merge_configs module when a fix is available.
Regards, Sharath
On Thu, Jun 9, 2022 at 2:20 PM Mark Goddard mark@stackhpc.com wrote:
Hi Sharath,
I had not realised this. Wikipedia suggests colons are sometimes used instead of equals. Perhaps the function should return which delimiter was found, and use it to generate the merged file?
Mark
On Tue, 7 Jun 2022 at 06:19, Sharath Ck sharath.madhava@gmail.com wrote:
Hi,
In kolla-ansible, I am using the merge_configs module to merge service configs files. I want to merge api-paste.ini config with my custom changes, however while merging if a line consists of one colon, it is getting replaced with equal sign. merge_configs splits the line by considering colon as delimiter and treats two parts as key and value. Later which will be substituted as key = value.
Original */v2.1: oscomputeversion_v2* Merged */v2.1= oscomputeversion_v2 *
Below, colon will be 1 and equal will be -1,
*def _split_key_value(self, line): colon = line.find(':') equal = line.find('=') if colon < 0 and equal < 0: return self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and equal < colon): key, value = line[:equal], line[equal + 1:] else: key, value = line[:colon], line[colon + 1:] value = value.strip() if value and value[0] == value[-1] and value.startswith((""", "'")): value = value[1:-1] return key.strip(), [value]*
I want to keep the config as it is and merge only my custom changes. Please let me know how to achieve this or colon can be escaped somehow.
Regards, Sharath Regards, Sharath
On Thu, 2022-06-09 at 14:27 +0530, Sharath Ck wrote:
Hi Mark,
Yes, that would be correct. If the function returns the delimiter and uses the same while mapping key value, it would keep the config content intact.
If colon can be replaced by equal and vice versa in openstack configs, especially api-paste.ini. I can continue with the merge now and update the merge_configs module when a fix is available.
just not that openstack for most file does not use ini format. it juse conf format or strictly speaking the oslo config format but the openstack service dont actully use ini unless we are using lib that have config that we need to generate.
so not evenything that is vaild in conf format is vlaid in ini format and visversa
for example ini formant defient that if repeate a key the last value is used with oslo config if the key is defiend as a multiopt type repeated values are formed into a list.
so key=val1 key=val2
in ini format result in key=val2 when parsed but in conforma that can be intpertated as k=[val1,val2]
the merge_configs module is based on pythons config parser which is inteded for ini files but we use them for conf files. api-paste.ini is not form openstack it s a config for a lib we use so in principal it might support : instead of = but that replacemnt is not intended to be support in nova.conf for example which is not an ini file its a .conf or oslo conf file.
kolla can extend merge_config to alow : for ini file if they want too but its not a bug if you try to use : in nova.conf and it does not work. we dont support that for openstack's config files intentinally. it may work because of the the code oslo.config is built on but that woudl not be intentinal as far as im aware and it might break.
in this case you are modifying nova's api-paste.ini https://github.com/openstack/nova/blob/master/etc/nova/api-paste.ini
specificaly https://github.com/openstack/nova/blob/master/etc/nova/api-paste.ini#L21-L22...
im not entirely sure why we use :
[composite:osapi_compute] use = call:nova.api.openstack.urlmap:urlmap_factory /: oscomputeversions /v2: oscomputeversion_legacy_v2 /v2.1: oscomputeversion_v2 # v21 is an exactly feature match for v2, except it has more stringent # input validation on the wsgi surface (prevents fuzzing early on the # API). It also provides new features via API microversions which are # opt into for clients. Unaware clients will receive the same frozen # v2 API feature set, but with some relaxed validation /v2/+: openstack_compute_api_v21_legacy_v2_compatible /v2.1/+: openstack_compute_api_v21
looking a the pastedeploy docs breifly https://docs.pylonsproject.org/projects/pastedeploy/en/latest/index.html#pas...
it seams to be using = and none of the test code used : so i dont think they inted to suport that either.
it might be better for nova and other poject to not use : in the api-paste.ini as we seem to be in undefiend behavior land on this one. https://github.com/openstack/nova/blob/master/nova/api/openstack/urlmap.py implementes the factory in nova which delegets the path parsign to paste https://github.com/cdent/paste/blob/225c7ebf34a9c90435932b690967538493a943ce...
[composite:osapi_compute] use = call:nova.api.openstack.urlmap:urlmap_factory / = oscomputeversions /v2 = oscomputeversion_legacy_v2 /v2.1 = oscomputeversion_v2 # v21 is an exactly feature match for v2, except it has more stringent # input validation on the wsgi surface (prevents fuzzing early on the # API). It also provides new features via API microversions which are # opt into for clients. Unaware clients will receive the same frozen # v2 API feature set, but with some relaxed validation /v2/+ = openstack_compute_api_v21_legacy_v2_compatible /v2.1/+ = openstack_compute_api_v21
i assume ^ does not work today? i have not tried it.
by all means add supprot to kolla for this use case if you would like too but longter we proably shoudl remove the use of paste since it undeploy or at least consider a syntax that is tested by them. if this syntax is somethign we added we do not appear to have any testing for it lookign at nova quickly.
Regards, Sharath
On Thu, Jun 9, 2022 at 2:20 PM Mark Goddard mark@stackhpc.com wrote:
Hi Sharath,
I had not realised this. Wikipedia suggests colons are sometimes used instead of equals. Perhaps the function should return which delimiter was found, and use it to generate the merged file?
Mark
On Tue, 7 Jun 2022 at 06:19, Sharath Ck sharath.madhava@gmail.com wrote:
Hi,
In kolla-ansible, I am using the merge_configs module to merge service configs files. I want to merge api-paste.ini config with my custom changes, however while merging if a line consists of one colon, it is getting replaced with equal sign. merge_configs splits the line by considering colon as delimiter and treats two parts as key and value. Later which will be substituted as key = value.
Original */v2.1: oscomputeversion_v2* Merged */v2.1= oscomputeversion_v2 *
Below, colon will be 1 and equal will be -1,
*def _split_key_value(self, line): colon = line.find(':') equal = line.find('=') if colon < 0 and equal < 0: return self.error_invalid_assignment(line) if colon < 0 or (equal >= 0 and equal < colon): key, value = line[:equal], line[equal + 1:] else: key, value = line[:colon], line[colon + 1:] value = value.strip() if value and value[0] == value[-1] and value.startswith((""", "'")): value = value[1:-1] return key.strip(), [value]*
I want to keep the config as it is and merge only my custom changes. Please let me know how to achieve this or colon can be escaped somehow.
Regards, Sharath Regards, Sharath
participants (3)
-
Mark Goddard
-
Sean Mooney
-
Sharath Ck