[openstack-dev] [nova] increasing the number of allowed volumes attached per instance > 26
Eric Fried
openstack at fried.cc
Mon Jun 11 15:14:33 UTC 2018
I thought we were leaning toward the option where nova itself doesn't
impose a limit, but lets the virt driver decide.
I would really like NOT to see logic like this in any nova code:
> if kvm|qemu:
> return 256
> elif POWER:
> return 4000
> elif:
> ...
On 06/11/2018 10:06 AM, Kashyap Chamarthy wrote:
> On Mon, Jun 11, 2018 at 11:55:29AM +0200, Sahid Orentino Ferdjaoui wrote:
>> On Fri, Jun 08, 2018 at 11:35:45AM +0200, Kashyap Chamarthy wrote:
>>> On Thu, Jun 07, 2018 at 01:07:48PM -0500, Matt Riedemann wrote:
>
> [...]
>
>>>> The 26 volumes thing is a libvirt driver restriction.
>>>
>>> The original limitation of 26 disks was because at that time there was
>>> no 'virtio-scsi'.
>>>
>>> (With 'virtio-scsi', each of its controller allows upto 256 targets, and
>>> each target can use any LUN (Logical Unit Number) from 0 to 16383
>>> (inclusive). Therefore, the maxium allowable disks on a single
>>> 'virtio-scsi' controller is 256 * 16384 == 4194304.) Source[1].
>>
>> Not totally true for Nova. Nova handles one virtio-scsi controller per
>> guest and plug all the volumes in one target so in theory that would
>> be 16384 LUN (only).
>
> Yeah, I could've been clearer that I was only talking the maximum
> allowable disks regardless of how Nova handles it.
>
>> But you made a good point the 26 volumes thing is not a libvirt driver
>> restriction. For example the QEMU SCSI native implementation handles
>> 256 disks.
>>
>> About the virtio-blk limitation I made the same finding but Tsuyoshi
>> Nagata shared an interesting point saying that virtio-blk is not longer
>> limited by the number of PCI slot available. That in recent kernel and
>> QEMU version [0].
>>
>> I could join what you are suggesting at the bottom and fix the limit
>> to 256 disks.
>
> Right, that's for KVM-based hypervisors.
>
> Eric Fried on IRC said the other day that for IBM POWER hypervisor they
> have tested (not with OpenStack) upto 4000 disks. But I am yet to see
> any more concrete details from POWER hypervisor users on this thread.
>
> If people can't seem to reach an agreement on the limits, we may have to
> settle with conditionals:
>
> if kvm|qemu:
> return 256
> elif POWER:
> return 4000
> elif:
> ...
>
> Before that we need concrete data that it is a _reasonble_ limit for
> POWER hypervisor (and possibly others).
>
>> [0] https://review.openstack.org/#/c/567472/16/nova/virt/libvirt/blockinfo.py@162
>
> [...]
>
More information about the OpenStack-dev
mailing list