Open Stack

I’d need to go through the logs to see exactly the occurrence maybe 10 minutes was a bit much. We run the cron job every 10 mins.

The cron job stopped a few weeks back and we had at least half a dozen projects with out of sync quotas within a few hours.

Cheers,
Sam


> On 21 Mar 2015, at 4:01 am, Mike Dorman <mdorman at godaddy.com> wrote:
> 
> Hey Sam,
> 
> When you say it occurs every 10 minutes, what exactly do you mean?  The 
> quota refresh?  Or a quota getting out of sync?
> 
> I am surprised you have max_age set so low.  I would think that would 
> basically trigger a quota refresh on every single reservation for most 
> users, right?
> 
> Mike
> 
> 
> 
> 
> 
> On 3/20/15, 4:18 AM, "Sam Morrison" <sorrison at gmail.com> wrote:
> 
>> We’ve had the following for a year or so but doesn’t help much, we still 
>> see it occurring every 10 mins or so.
>> 
>> max_age = 10
>> until_refresh = 5
>> reservation_expire=600
>> 
>> We have a cron job that runs every 10 mins that figures out what projects 
>> are out of sync and corrects them.
>> We’ve always been scared of setting these to zero but we probably should.
>> 
>> Sam
>> 
>> 
>>> On 15 Mar 2015, at 2:53 pm, Mike Dorman <mdorman at godaddy.com> wrote:
>>> 
>>> Yeah the default is just ‘0’ for both, which disables the refresh.
>>> 
>>> 
>>> 
>>> The one downside is that it may not be 100% transparent to the user.  
>>> If 
>>> the quota is already (incorrectly) too high, and exceeding the quota 
>>> limit, the reservation that triggers the refresh will still fail.  I.e. 
>>> the reservation is attempted based on the quota usage values _before_ 
>>> the 
>>> refresh.  But then after that the quota should be fixed and it will 
>>> work 
>>> again on the next reservation.
>>> 
>>> But my thinking is that most quota issues happen slowly over time.  If 
>>> we 
>>> are correcting them often and automatically, they hopefully never get 
>>> to 
>>> the point where they’re bad enough to manifest reservation errors to 
>>> the 
>>> user.
>>> 
>>> I don’t have any information re: db load.  I assume it regenerates 
>>> based 
>>> on what’s in the instances or reservations table.  I imagine the load 
>>> for 
>>> doing a single refresh is probably comparable to doing a ‘nova list’.
>>> 
>>> Mike
>>> 
>>> 
>>> 
>>> On 3/14/15, 2:27 PM, "Tim Bell" <Tim.Bell at cern.ch> wrote:
>>> 
>>>> Interesting... what are the defaults ?
>>>> 
>>>> Assuming no massive DB load, getting synced within a day would seem 
>>>> reasonable. Is the default no max age ?
>>>> 
>>>> Tim
>>>> 
>>>>> -----Original Message-----
>>>>> From: Jesse Keating [mailto:jlk at bluebox.net]
>>>>> Sent: 14 March 2015 16:59
>>>>> To: openstack-operators at lists.openstack.org
>>>>> Subject: Re: [Openstack-operators] max_age and until_refresh for 
>>>>> fixing 
>>>>> Nova
>>>>> quotas
>>>>> 
>>>>> On 3/14/15 8:11 AM, Mike Dorman wrote:
>>>>>> I did short write-up here http://t.co/Q5X1hTgJG1 if you are 
>>>>>> interested
>>>>>> in the details.
>>>>>> 
>>>>> 
>>>>> Thanks for sharing Matt! That's an excellent write up.
>>>>> 
>>>>> --
>>>>> -jlk
>>>>> 
>>>>> _______________________________________________
>>>>> OpenStack-operators mailing list
>>>>> OpenStack-operators at lists.openstack.org
>>>>> 
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>> 
>>>> _______________________________________________
>>>> OpenStack-operators mailing list
>>>> OpenStack-operators at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>> _______________________________________________
>>> OpenStack-operators mailing list
>>> OpenStack-operators at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators