<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"><base href="x-msg://1023/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">There are two possible solutions depending on what the goal of usage metering is.<div><br></div><div>Goal A: log usage of all created volumes regardless of attach status.</div><div><br></div><div>This should be done in cinder (or nova-volume/manager.py) side where the list of volumes exists.</div><div><br></div><div>Goal B: log usage of all attached volumes.</div><div><br></div><div>Compute keeps a record of volumes that are attached via the block_device_mapping table. The method for doing this is to get all instances belonging to the host, iterate through each instance and get a list of attached volues from block_device_mapping, and log usage for each one. You could do this inside the existing _instance_usage_audit task (which already grabs all active instances on the host) using self._get_instance_volume_bdms(instance_uuid).</div><div><br></div><div>Vish</div><div><br></div><div><div><div>On Aug 2, 2012, at 4:43 AM, "O'Driscoll, Cian" <<a href="mailto:cian.o-driscoll@hp.com">cian.o-driscoll@hp.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div lang="EN-US" link="blue" vlink="purple" style="font-family: Menlo; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div class="WordSection1" style="page: WordSection1; "><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">Just wanted to start a discussion on the issue I hit yesterday when implementing volume usage metering (<a href="https://blueprints.launchpad.net/nova/+spec/volume-usage-metering" style="color: purple; text-decoration: underline; ">https://blueprints.launchpad.net/nova/+spec/volume-usage-metering</a>)<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">Here is what I found:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">I create a periodic task in nova/compute/manager.py, the function looks like this<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> @manager.periodic_task<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> def _poll_volume_usage(self, context, start_time=None, stop_time=None):<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> if not start_time:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> start_time = utils.last_completed_audit_period()[1]<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> curr_time = time.time()<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> if curr_time - self._last_vol_usage_poll > FLAGS.volume_poll_interval:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> self._last_vol_usage_poll = curr_time<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> LOG.info(_("Updating volume usage cache"))<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> <span class="Apple-converted-space"> </span><b>volumes = self.volume_api.get_all(context)<o:p></o:p></b></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> try:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> vol_usage = self.driver.get_all_volume_usage(context, volumes,<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> start_time, stop_time)<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> except NotImplementedError:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> return<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> for usage in vol_usage:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> self.db.vol_usage_update(context, usage['volume'], start_time,<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> usage['rd_req'], usage['rd_bytes'],<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> usage['wr_req'], usage['wr_bytes'])<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">The issue is with the call “self.volume_api.get_all(context)”, in my blueprint I was going to use self.db.volume_get_all_by_host(ctxt, self.host) but because nova db doesn’t contain volume info anymore, we need to talk to cinder to get the info.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">_poll_volume_usage is passed and admin context when ran. The admin context doesn’t contain a service_catalog or any authentication info from Keystone, bascically just a context that says is_admin=True.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">So when “self.volume_api.get_all(context)” is called passing in the admin context, the instantiation of a python cinder client fails as the service catalog is empty.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">Even if the service catalog was populated, I still think we’d fail as we wouldn’t have a Auth token to talk to cinder.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 ERROR nova.manager [-] Error during ComputeManager._poll_volume_usage: 'NoneType' object is not iterable<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager Traceback (most recent call last):<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager File "/opt/stack/nova/nova/manager.py", line 173, in periodic_tasks<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager task(self, context)<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager File "/opt/stack/nova/nova/compute/manager.py", line 2639, in _poll_volume_usage<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager volumes = self.volume_api.get_all(context)<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager File "/opt/stack/nova/nova/volume/cinder.py", line 125, in get_all<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager items = cinderclient(context).volumes.list(detailed=True)<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager File "/opt/stack/nova/nova/volume/cinder.py", line 45, in cinderclient<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager url = sc.url_for(service_type='volume', service_name='cinder')<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager File "/opt/stack/python-cinderclient/cinderclient/service_catalog.py", line 53, in url_for<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager for service in catalog:<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager TypeError: 'NoneType' object is not iterable<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">2012-08-01 14:05:01 TRACE nova.manager<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">One possible solution to this is to have a cinder admin user with read only(read all data from cinder db) access created in Keystone (Glance does something very similar when talking to swift)? It keeps the user data in a conf file on the glance nodes.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">So before we call “self.volume_api.get_all(context)”, we can generate a cinder admin context using the info in a conf file.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">I think this could be done in another blueprint as I feel there are other use cases where a cinder admin user is required (Any cinder auditing in nova would require the admin user).<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">For now I propose, to make progress in my implementation of just adding stats collection on detach and not implementing the periodic task for now.<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE">This would be volume usage metering part 1 say, as a proof of concept and when a general consensus/implementation is reached around the cinder admin user, I can implement the periodic task as part ?<o:p></o:p></span></div><div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-IE"> </span></div></div>_______________________________________________<br>OpenStack-dev mailing list<br><a href="mailto:OpenStack-dev@lists.openstack.org" style="color: purple; text-decoration: underline; ">OpenStack-dev@lists.openstack.org</a><br><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" style="color: purple; text-decoration: underline; ">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br></div></blockquote></div><br></div></body></html>