Open Stack

> > A follow-up for your discussion on IRC last night, I had a quick
> > check of the loopingcall implementation, and seems there is really
> > no time gurantee. Even if we adjust the interval for
> > greenthread.sleep() dynamically, we also must make the metering
> > periodic task always at the head.
> > 
> > I think we can enhance the periodic task to meet partially our
> > requirement, like create a separated task type as high-priority,
> > which should make sure no long-delay.
> > 
> > The only concern is, can we make sure the LoopingCall itself will
> > be invoked on time after the greenthread.sleep(interval),
> > considering the attribute of greenthread, or even python thread.
> > 
> > openstack/common/loopingcall.py-> LoopingCall(object):
> > 
> >                 while self._running:
> >                     self.f(*self.args, **self.kw)
> >                     if not self._running:
> >                         break
> >                     greenthread.sleep(interval)

Thanks for following this up!

So can I confirm that I've understood the basic issues here are
that:

 (a) The time spent executing tasks is not accounted for when
     determining how much time to sleep between task runs. So
     for example if periodic_interval is set to N seconds, the
     actual time between tasks is of the order of:

       N + \Sigma duration(task_i)/(1 + ticks for task_i)

     The more tasks with ticks=0, and the longer the task
     duration, the more we skew away from tasks executing on 
     wall-clock boundaries. 

 (b) There is no guarantee (beyond convention) that a task won't
     take longer than periodic_interval/|tasks| to execute.

 (c) There is an indeterminate lag after the expiry of the sleep 
     interval before the LoopingCall thread is re-scheduled.

So could we at least address issue (a) by simply subtracting
the duration of the last tasks run from the next sleep interval?

e.g. change LoopingCall.start()._inner() as follows:

    while self._running:
+       start = datetime.datetime.now()
        self.f(*self.args, **self.kw)
+       end = datetime.datetime.now()
+       delta = end - start
+       elapsed = delta.seconds + delta.microseconds/(10 ** 6)
+       delay = interval - elapsed
        if not self._running:
            break
-       greenthread.sleep(interval)
+       greenthread.sleep(delay if delay > 0 else 0)

I guess that's what you meant by adjusting the interval dynamically?

But I'm not sure that we can always address (b) or (c) even with a
special thread for high-priority/time-sensitive tasks.

Cheers,
Eoghan