Open Stack

On Mon, Mar 3, 2014 at 8:00 AM, Dina Belova <dbelova at mirantis.com> wrote:
>> Capacity planning not falling under Compute's umbrella is news to me,
>
>> are you referring to Gantt and scheduling in general? Perhaps I don't
>
>> fully understand the full extent of what 'capacity planning' actually
>
>> is.
>
>
> We intend that 'capacity' means not only 'compute capacity', but any kind of
> resources capacity - we may plan not only compute resources usage, but also
> storage, etc. Gantt is about 'where to schedule' potentially any kind of
> resources, Climate is about 'when'. Term 'scheduling' in Climate sphere has
> no reference to common OS scheduling process, here we mean something like
> 'timetable', that's much closer. Capacity planning here is ability to
> predict the fact of resource usage in future (for cloud providers), and
> provide these resources to users.
>
>
>> I don't see how Climate reserves resources is relevant to the user.
>
>
>> From the user's point of view what is the benefit from making a
>
>> reservation in the future? Versus what Nova supports today, asking for
>
>> an instance in the present.
>
>
>> Same thing from the operator's perspective,  what is the benefit of
>
>> taking reservations for the future?
>
>
> Joe, I'll describe two original use cases, that were the reason for Climate
> to be created. I suppose it might give idea of what we were thinking about
> while its implementation.
>
>
> ====================
>
> Host reservation use case
>
> ====================
>
>
> This use case was born to fit XLcloud (http://www.xlcloud.org) project.
> Needs of this project require usage of whole hosts compute capacities
> without any other VMs running on them - and to create some kind of "hosts
> usage timetable". The last thing is needed to implement energy efficiency
> for OS cloud, where these High Performance Cloud Computing jobs will be
> running on. That will allow to keep unused hosts in 'shut down' state and to
> wake them up where they'll be really needed.
>
>
> Steps for this use case may be described in the following way:
>
>
> 1) admin user marks some hosts from common pool as 'reservable' - these
> hosts then will be given to user who wants to use their whole capacity and
> will be 'isolated' to the solely use of Climate until they are freed
>
> 2) if user wants to use some of these hosts, he/she asks Climate to provide
> some amount of these hosts with any wanted specs - like amount of RAM, CPU,
> etc.
>
> 3) Climate remembers that these hosts will be used in some time for some
> amount of time
>
> 4) when lease (contract between user and Climate) will start, user may use
> these hosts to run VMs on them
>
>
> In future we'll implement energy efficiency, that will completely fit this
> first use case.

This sounds like something that belongs in nova, Phil Day has an
elegant solution for this:
https://blueprints.launchpad.net/nova/+spec/whole-host-allocation

>
>
> ===================
>
> VM reservation use case
>
> ===================
>
>
> This use case was created as a result of automated virtual resource
> allocation/reclaiming idea for dev clouds.
>
>
> If company has dev cloud for its developers, it'll be great if there won't
> be created and forgotten VMs running on - for this, process of VMs
> booting/shutting them down should be automated and processed by some
> independent service.
>

Heat?

I spin up dev instances all the time and have never had this problem
in part because if I forget my quota will remind me.

>
> Here was born idea of virtual reservations.
>
>
> Some usual workflow for this use case looks like:
>
>
> 1) user knows that in future he'll need lab with some VMs in it and he asks
> Climate to reserve them (now we've implemented one VM reservation)
>

Why does he need to reserve them in the future? When he wants an
instance can't he just get one? As Sean said, what happens if someone
has no free quota when the reservation kicks in?

> 2) Climate remembers that this VM will be used in some time for some amount
> of time (now we've implemented plugins that keeps these VMs in shelved state
> while they are not used)
>
> 3) when lease starts, Climate wakes this VM and it's completely ready to be
> used

How is this different from 'nova boot?' When nova boot finishes the VM
is completely ready to be used

>
>
> As we've discussed with Cafe project folks
> (https://wiki.openstack.org/wiki/Cafe) - this ability will be helpful for
> academic usage, where students also need to have automatically
> allocated/reclaimed virtual resources.

Cafe sounds a bit like more sophisticated quotas, quotas that work on
overall usage not just current usage.

>
>
> =====================
>
>
> As said, these two use cases were original ones, and now we're thinking
> about more general view - any type of physical/virtual resource to be
> managed in time. For example, we're planning to implement complex virtual
> resource reservations (like Heat stacks), that will allow
> developers/students to have working small OS clouds by the time they'll need
> them.

I can do that today, I spin a up a devstack VM in the cloud whenever I need one.

>
>
> Also Climate will be helpful in some way, close to mentioned by Anne - usage
> of Climate+billing_service will allow to sell cloud resources to end users
> in the following manner:
>
>
> - if you're reserving resources far before you'll need them, it'll be
> cheaper

Why? How does this save a provider money?

>
> - if you're reserving resources just about small amount of time before
> you'll need them, it'll be more expensive
>
>
> That's not really AWS use case, where reserved resources are not guarantied
> to user, but it might be un-guarantied type of reservation, that will be
> much cheaper like:
>

"Reserved Instances provide a capacity reservation so that you can
have confidence in your ability to launch the number of instances you
have reserved when you need them."
https://aws.amazon.com/ec2/purchasing-options/reserved-instances/

Amazon does guarantee the resource will be available.  Amazon can
guarantee the resource because they can terminate spot instances at
will, but OpenStack doesn't have this concept today.

>
> - if you're reserving cloud resources without any guarantees you'll have
> them, they will be much cheaper than guaranteed ones.
>
>
> Thanks
>
> Dina
>
>
> On Mon, Mar 3, 2014 at 6:27 PM, Anne Gentle <anne at openstack.org> wrote:
>>
>>
>>
>> On Mon, Mar 3, 2014 at 8:20 AM, Joe Gordon <joe.gordon0 at gmail.com> wrote:
>>>
>>> On Mon, Mar 3, 2014 at 4:42 AM, Sylvain Bauza <sylvain.bauza at bull.net>
>>> wrote:
>>> > Hi Joe,
>>> >
>>> > Thanks for your reply, I'll try to further explain.
>>> >
>>> >
>>> > Le 03/03/2014 05:33, Joe Gordon a écrit :
>>> >
>>> >> On Sun, Mar 2, 2014 at 11:32 AM, Dina Belova <dbelova at mirantis.com>
>>> >> wrote:
>>> >>>
>>> >>> Hello, folks!
>>> >>>
>>> >>> I'd like to request Climate project review for incubation. Here is
>>> >>> official
>>> >>> incubation application:
>>> >>>
>>> >>> https://wiki.openstack.org/wiki/Climate/Incubation
>>> >>
>>> >> I'm unclear on what Climate is trying to solve. I read the 'Detailed
>>> >> Description' from the link above, and it states Climate is trying to
>>> >> solve two uses cases (and the more generalized cases of those).
>>> >>
>>> >> 1) Compute host reservation (when user with admin privileges can
>>> >> reserve hardware resources that are dedicated to the sole use of a
>>> >> tenant)
>>> >> 2) Virtual machine (instance) reservation (when user may ask
>>> >> reservation service to provide him working VM not necessary now, but
>>> >> also in the future)
>>> >
>>> > Climate is born from the idea of dedicating compute resources to a
>>> > single
>>> > tenant or user for a certain amount of time, which was not yet
>>> > implemented
>>> > in Nova: how as an user, can I ask Nova for one compute host with
>>> > certain
>>> > specs to be exclusively allocated to my needs, starting in 2 days and
>>> > being
>>> > freed in 5 days ?
>>> >
>>> > Albeit the exclusive resource lock can be managed on the Nova side,
>>> > there is
>>> > currently no possibilities to ensure resource planner.
>>> >
>>> > Of course, and that's why we think Climate can also stand by its own
>>> > Program, resource reservation can be seen on a more general way : what
>>> > about
>>> > reserving an Heat stack with its volume and network nested resources ?
>>> >
>>> >
>>> >> You want to support being able to reserve an instance in the future.
>>> >> As a cloud operator how do I take advantage of that information? As a
>>> >> cloud consumer, what is the benefit? Today OpenStack supports both
>>> >> uses cases, except it can't request an Instance for the future.
>>> >
>>> >
>>> > Again, that's not only reserving an instance, but rather a complex mix
>>> > of
>>> > resources. At the moment, we do provide way to reserve virtual
>>> > instances by
>>> > shelving/unshelving them at the lease start, but we also give
>>> > possibility to
>>> > provide dedicated compute hosts. Considering it, the logic of resource
>>> > allocation and scheduling (take the word as resource planner, in order
>>> > not
>>> > to confuse with Nova's scheduler concerns) and capacity planning is too
>>> > big
>>> > to fail under the Compute's umbrella, as it has been agreed within the
>>> > Summit talks and periodical threads.
>>>
>>> Capacity planning not falling under Compute's umbrella is news to me,
>>> are you referring to Gantt and scheduling in general? Perhaps I don't
>>> fully understand the full extent of what 'capacity planning' actually
>>> is.
>>>
>>> >
>>> > From the user standpoint, there are multiple ways to integrate with
>>> > Climate
>>> > in order to get Capacity Planning capabilities. As you perhaps noticed,
>>> > the
>>> > workflow for reserving resources is different from one plugin to
>>> > another.
>>> > Either we say the user has to explicitly request for dedicated
>>> > resources
>>> > (using Climate CLI, see dedicate compute hosts allocation), or we
>>> > implicitly
>>> > integrate resource allocation from the Nova API (see virtual instance
>>> > API
>>> > hook).
>>>
>>> I don't see how Climate reserves resources is relevant to the user.
>>>
>>> >
>>> > We truly accept our current implementation as a first prototype, where
>>> > scheduling decisions can be improved (possibly thanks to some tight
>>> > integration with a future external Scheduler aaS, hello Gantt), where
>>> > also
>>> > resource isolation and preemption must also be integrated with
>>> > subprojects
>>> > (we're currently seeing how to provision Cinder volumes and Neutron
>>> > routers
>>> > and nets), but anyway we still think there is a (IMHO big) room for
>>> > resource
>>> > and capacity management on its own project.
>>> >
>>> > Hoping it's clearer now,
>>>
>>> Unfortunately that doesn't clarify things for me.
>>>
>>> From the user's point of view what is the benefit from making a
>>> reservation in the future? Versus what Nova supports today, asking for
>>> an instance in the present.
>>>
>>> Same thing from the operator's perspective,  what is the benefit of
>>> taking reservations for the future?
>>>
>>> This whole model is unclear to me because as far as I can tell no
>>> other clouds out there support this model, so I have nothing to
>>> compare it to.
>>>
>>
>> Hi Joe,
>> I think it's meant to save consumers money by pricing instances based on
>> today's prices.
>>
>> https://aws.amazon.com/ec2/purchasing-options/reserved-instances/
>>
>> Anne
>>
>>>
>>> > -Sylvain
>>> >
>>> >
>>> > _______________________________________________
>>> > OpenStack-dev mailing list
>>> > OpenStack-dev at lists.openstack.org
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
>
> Best regards,
>
> Dina Belova
>
> Software Engineer
>
> Mirantis Inc.
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>