Open Stack

Hello William,

We are booting our instances from Cinder volumes over ISCSI with
multipath and can perform live migration just fine.

The things we have changed:

nova.conf on compute nodes

[libvirt]
virt_type=kvm
inject_password=true
inject_key=true
inject_partition=-1
live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE
iscsi_use_multipath = True

The live_migration_flag is deprecated and will be removed in next
release I think, but we are running Liberty stable and haven't upgraded yet.

Also we had issues with live migration and multipath volumes turning
into a failed state. Changing the failback and retry settings for
multipathd solved it (see https://bugs.launchpad.net/nova/+bug/1525802)

multipath.conf on compute nodes

defaults {
        failback immediate
        no_path_retry 0
}

Best regards

Tobias


On 08/15/2016 05:15 PM, William Josefsson wrote:
> yes thx tobias, did you try live-migration after this change? I will
> try on my end too and share whether it worked with the DiskFilter
> disabled or not. thx will
>
> On Mon, Aug 15, 2016 at 4:22 PM, Tobias Urdin <tobias.urdin at crystone.com> wrote:
>> Hello,
>>
>> We are having this issue in production and I have since pushed for a fix
>> for this.
>>
>> We solved it by removing the DiskFilter filter, ignore any RT data from
>> nova and seen in horizon and keep close monitoring on our block storage
>> usage.
>>
>> nova.conf:
>>
>> #scheduler_default_filters = RetryFilter, AvailabilityZoneFilter,
>> RamFilter, DiskFilter, ComputeFilter, ComputeCapabilitiesFilter,
>> ImagePropertiesFilter, ServerGroupAntiAffinityFilter,
>> ServerGroupAffinityFilter
>> scheduler_default_filters = RetryFilter, AvailabilityZoneFilter,
>> RamFilter, ComputeFilter, ComputeCapabilitiesFilter,
>> ImagePropertiesFilter, ServerGroupAntiAffinityFilter,
>> ServerGroupAffinityFilter
>>
>> Default options is the one commented out, the other one has DiskFilter
>> removed.
>>
>> We haven't had any issues after removing the DiskFilter.
>>
>>
>> Best regards
>>
>>
>> On 08/15/2016 10:17 AM, Kostiantyn.Volenbovskyi at swisscom.com wrote:
>>> Hi,
>>>
>>> Just to add that this bug was recently brought up in openstack-dev [1]
>>>
>>> it sounds that you are waiting for fix via [2] or, probably more likely [3]
>>> Disabling DiskFilter sounds like OK workaround in case your VMs are never booted from image in your OpenStack Region(s).
>>> But if I understand the fault - the migrate/resize/live migrate/rebuild/evacuate will not work because
>>> . assuming that on target Compute Host you don't have local disk - and your flavor has more than 0 GB for root and/or ephemeral disk
>>> -it would fail.
>>>
>>> So I would say that setting your root disk space to '0' in the flavor might be a better workaround
>>>
>>> BR,
>>> Konstantin
>>> [1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101566.html
>>> [2] https://review.openstack.org/#/c/200870/
>>> [3] https://review.openstack.org/#/c/355091/
>>>
>>>
>>>> -----Original Message-----
>>>> From: William Josefsson [mailto:william.josefson at gmail.com]
>>>> Sent: Sunday, August 14, 2016 5:47 PM
>>>> To: openstack-operators at lists.openstack.org
>>>> Subject: [Openstack-operators] DiskFilter Disabled - No valid host was found
>>>>
>>>> Hello, I got Error, no valid host was found while trying to provision and I could in
>>>> nova-scheduler.log see that the DiskFilter failed to find a compute host. My OS is
>>>> CentOS7/ running Liberty.
>>>>
>>>> If I'm not mistaken, it is a bug that DiskFilter checks space available on compute-
>>>> host / partition even when I'm using iSCSI and RBD backends. The / is no way
>>>> near full.
>>>> https://bugs.launchpad.net/nova/+bug/1358566
>>>>
>>>> Can anyone confirm whether it is fine to disable DiskFilter if I'm using iSCSI and
>>>> RBD backend or would anything else brake? thx will
>>>>
>>>> _______________________________________________
>>>> OpenStack-operators mailing list
>>>> OpenStack-operators at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>> _______________________________________________
>>> OpenStack-operators mailing list
>>> OpenStack-operators at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>