Open Stack

Le 01/07/2014 10:45, Lisa a écrit :
> Hi Tim,
>
> for sure this is one of the main issues we are facing and the approach
> you suggested is the same we are investigating on.
> Could you provide some details about the Heat proxy renew mechanism?
> Thank you very much for your feedback.
> Cheers,
> Lisa
>


Keystone already provides support for that problem, that's called trusts.
https://wiki.openstack.org/wiki/Keystone/Trusts

I would be also interested in reviewing your implementation, because
Blazar [1] is also providing some reservation mechanism based on trusts.

One long-term objective of splitting the Scheduler into a separate
project (called Gantt) would be to interoperate Blazar and Gantt for
scheduling non-immediate requests, so your project is going into the
right direction.

As Don said, feel free to join us today at 3pm UTC to discuss about your
proposal.


[1] https://wiki.openstack.org/wiki/Blazar


>
> On 01/07/2014 08:46, Tim Bell wrote:
>> Eric,
>>
>> Thanks for sharing your work, it looks like an interesting development.
>>
>> I was wondering how the Keystone token expiry is handled since the
>> tokens generally have a 1 day validity. If the request is scheduling
>> for more than one day, it would no longer have a valid token. We have
>> similar scenarios with Kerberos/AFS credentials in the CERN batch
>> system. There are some interesting proxy renew approaches used by
>> Heat to get tokens at a later date which may be useful for this problem.
>>
>> $ nova credentials
>> +-----------+-----------------------------------------------------------------+
>>
>> | Token     |
>> Value                                                           |
>> +-----------+-----------------------------------------------------------------+
>>
>> | expires   |
>> 2014-07-02T06:39:59Z                                            |
>> | id        |
>> 1a819279121f4235a8d85c694dea5e9e                                |
>> | issued_at |
>> 2014-07-01T06:39:59.385417                                      |
>> | tenant    | {"id": "841615a3-ece9-4622-9fa0-fdc178ed34f8",
>> "enabled": true, |
>> |           | "description": "Personal Project for user timbell",
>> "name":     |
>> |           | "Personal
>> timbell"}                                             |
>> +-----------+-----------------------------------------------------------------+
>>
>>
>> Tim
>>> -----Original Message-----
>>> From: Eric Frizziero [mailto:eric.frizziero at pd.infn.it]
>>> Sent: 30 June 2014 16:05
>>> To: OpenStack Development Mailing List (not for usage questions)
>>> Subject: [openstack-dev] [nova][scheduler] Proposal:
>>> FairShareScheduler.
>>>
>>> Hi All,
>>>
>>> we have analyzed the nova-scheduler component (FilterScheduler) in our
>>> Openstack installation used by some scientific teams.
>>>
>>> In our scenario, the cloud resources need to be distributed among
>>> the teams by
>>> considering the predefined share (e.g. quota) assigned to each team,
>>> the portion
>>> of the resources currently used and the resources they have already
>>> consumed.
>>>
>>> We have observed that:
>>> 1) User requests are sequentially processed (FIFO scheduling), i.e.
>>> FilterScheduler doesn't provide any dynamic priority algorithm;
>>> 2) User requests that cannot be satisfied (e.g. if resources are not
>>> available) fail and will be lost, i.e. on that scenario
>>> nova-scheduler doesn't
>>> provide any queuing of the requests;
>>> 3) OpenStack simply provides a static partitioning of resources
>>> among various
>>> projects / teams (use of quotas). If project/team 1 in a period is
>>> systematically
>>> underutilizing its quota and the project/team 2 instead is
>>> systematically
>>> saturating its quota, the only solution to give more resource to
>>> project/team 2 is
>>> a manual change (to be done by the admin) to the related quotas.
>>>
>>> The need to find a better approach to enable a more effective
>>> scheduling in
>>> Openstack becomes more and more evident when the number of the user
>>> requests to be handled increases significantly. This is a well known
>>> problem
>>> which has already been solved in the past for the Batch Systems.
>>>
>>> In order to solve those issues in our usage scenario of Openstack,
>>> we have
>>> developed a prototype of a pluggable scheduler, named
>>> FairShareScheduler,
>>> with the objective to extend the existing OpenStack scheduler
>>> (FilterScheduler)
>>> by integrating a (batch like) dynamic priority algorithm.
>>>
>>> The architecture of the FairShareScheduler is explicitly designed to
>>> provide a
>>> high scalability level. To all user requests will be assigned a
>>> priority value
>>> calculated by considering the share allocated to the user by the
>>> administrator
>>> and the evaluation of the effective resource usage consumed in the
>>> recent past.
>>> All requests will be inserted in a priority queue, and processed in
>>> parallel by a
>>> configurable pool of workers without interfering with the priority
>>> order.
>>> Moreover all significant information (e.g. priority queue) will be
>>> stored in a
>>> persistence layer in order to provide a fault tolerance mechanism
>>> while a proper
>>> logging system will annotate all relevant events, useful for
>>> auditing processing.
>>>
>>> In more detail, some features of the FairshareScheduler are:
>>> a) It assigns dynamically the proper priority to every new user
>>> requests;
>>> b) The priority of the queued requests will be recalculated
>>> periodically using the
>>> fairshare algorithm. This feature guarantees the usage of the cloud
>>> resources is
>>> distributed among users and groups by considering the portion of the
>>> cloud
>>> resources allocated to them (i.e. share) and the resources already
>>> consumed;
>>> c) all user requests will be inserted in a (persistent) priority
>>> queue and then
>>> processed asynchronously by the dedicated process (filtering +
>>> weighting phase)
>>> when compute resources are available;
>>> d) From the client point of view the queued requests remain in
>>> "Scheduling"
>>> state till the compute resources are available. No new states added:
>>> this
>>> prevents any possible interaction issue with the Openstack clients;
>>> e) User requests are dequeued by a pool of WorkerThreads
>>> (configurable), i.e.
>>> no sequential processing of the requests;
>>> f) The failed requests at filtering + weighting phase may be
>>> inserted again in the
>>> queue for n-times (configurable).
>>>
>>> We have integrated the FairShareScheduler in our Openstack installation
>>> (release "HAVANA"). We're now working to adapt the
>>> FairShareScheduler to the
>>> new release "IceHouse".
>>>
>>> Does anyone have experiences in those issues found in our cloud
>>> scenario?
>>>
>>> Could the FairShareScheduler be useful for the Openstack community?
>>> In that case, we'll be happy to share our work.
>>>
>>> Any feedback/comment is welcome!
>>>
>>> Cheers,
>>> Eric.
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev