Open Stack

> -----Original Message-----
> From: Lisa [mailto:lisa.zangrando at pd.infn.it]
> Sent: 01 July 2014 10:45
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
> 
> Hi Tim,
> 
> for sure this is one of the main issues we are facing and the approach you
> suggested is the same we are investigating on.
> Could you provide some details about the Heat proxy renew mechanism?
> Thank you very much for your feedback.
> Cheers,
> Lisa
> 
> 

I was thinking about how the Keystone Trusts mechanism could be used ... https://wiki.openstack.org/wiki/Keystone/Trusts. Heat was looking to use something like this (https://wiki.openstack.org/wiki/Heat/Blueprints/InstanceUsers#1._Use_credentials_associated_with_a_trust)

Maybe one of the Keystone experts could advise how tokens could be renewed in such a scenario.

Tim

On 01/07/2014 08:46, Tim Bell wrote:
> Eric,
>
> Thanks for sharing your work, it looks like an interesting development.
>
> I was wondering how the Keystone token expiry is handled since the tokens generally have a 1 day validity. If the request is scheduling for more than one day, it would no longer have a valid token. We have similar scenarios with Kerberos/AFS credentials in the CERN batch system. There are some interesting proxy renew approaches used by Heat to get tokens at a later date which may be useful for this problem.
>
> $ nova credentials
> +-----------+-----------------------------------------------------------------+
> | Token     | Value                                                           |
> +-----------+-----------------------------------------------------------------+
> | expires   | 2014-07-02T06:39:59Z                                            |
> | id        | 1a819279121f4235a8d85c694dea5e9e                                |
> | issued_at | 2014-07-01T06:39:59.385417                                      |
> | tenant    | {"id": "841615a3-ece9-4622-9fa0-fdc178ed34f8", "enabled": true, |
> |           | "description": "Personal Project for user timbell", "name":     |
> |           | "Personal timbell"}                                             |
> +-----------+-----------------------------------------------------------------+
>
> Tim
>> -----Original Message-----
>> From: Eric Frizziero [mailto:eric.frizziero at pd.infn.it]
>> Sent: 30 June 2014 16:05
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: [openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
>>
>> Hi All,
>>
>> we have analyzed the nova-scheduler component (FilterScheduler) in our
>> Openstack installation used by some scientific teams.
>>
>> In our scenario, the cloud resources need to be distributed among the teams by
>> considering the predefined share (e.g. quota) assigned to each team, the portion
>> of the resources currently used and the resources they have already consumed.
>>
>> We have observed that:
>> 1) User requests are sequentially processed (FIFO scheduling), i.e.
>> FilterScheduler doesn't provide any dynamic priority algorithm;
>> 2) User requests that cannot be satisfied (e.g. if resources are not
>> available) fail and will be lost, i.e. on that scenario nova-scheduler doesn't
>> provide any queuing of the requests;
>> 3) OpenStack simply provides a static partitioning of resources among various
>> projects / teams (use of quotas). If project/team 1 in a period is systematically
>> underutilizing its quota and the project/team 2 instead is systematically
>> saturating its quota, the only solution to give more resource to project/team 2 is
>> a manual change (to be done by the admin) to the related quotas.
>>
>> The need to find a better approach to enable a more effective scheduling in
>> Openstack becomes more and more evident when the number of the user
>> requests to be handled increases significantly. This is a well known problem
>> which has already been solved in the past for the Batch Systems.
>>
>> In order to solve those issues in our usage scenario of Openstack, we have
>> developed a prototype of a pluggable scheduler, named FairShareScheduler,
>> with the objective to extend the existing OpenStack scheduler (FilterScheduler)
>> by integrating a (batch like) dynamic priority algorithm.
>>
>> The architecture of the FairShareScheduler is explicitly designed to provide a
>> high scalability level. To all user requests will be assigned a priority value
>> calculated by considering the share allocated to the user by the administrator
>> and the evaluation of the effective resource usage consumed in the recent past.
>> All requests will be inserted in a priority queue, and processed in parallel by a
>> configurable pool of workers without interfering with the priority order.
>> Moreover all significant information (e.g. priority queue) will be stored in a
>> persistence layer in order to provide a fault tolerance mechanism while a proper
>> logging system will annotate all relevant events, useful for auditing processing.
>>
>> In more detail, some features of the FairshareScheduler are:
>> a) It assigns dynamically the proper priority to every new user requests;
>> b) The priority of the queued requests will be recalculated periodically using the
>> fairshare algorithm. This feature guarantees the usage of the cloud resources is
>> distributed among users and groups by considering the portion of the cloud
>> resources allocated to them (i.e. share) and the resources already consumed;
>> c) all user requests will be inserted in a (persistent) priority queue and then
>> processed asynchronously by the dedicated process (filtering + weighting phase)
>> when compute resources are available;
>> d) From the client point of view the queued requests remain in "Scheduling"
>> state till the compute resources are available. No new states added: this
>> prevents any possible interaction issue with the Openstack clients;
>> e) User requests are dequeued by a pool of WorkerThreads (configurable), i.e.
>> no sequential processing of the requests;
>> f) The failed requests at filtering + weighting phase may be inserted again in the
>> queue for n-times (configurable).
>>
>> We have integrated the FairShareScheduler in our Openstack installation
>> (release "HAVANA"). We're now working to adapt the FairShareScheduler to the
>> new release "IceHouse".
>>
>> Does anyone have experiences in those issues found in our cloud scenario?
>>
>> Could the FairShareScheduler be useful for the Openstack community?
>> In that case, we'll be happy to share our work.
>>
>> Any feedback/comment is welcome!
>>
>> Cheers,
>> Eric.
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev