Open Stack

On 10/01, Jay S. Bryant wrote:
> Grant,
>
> So, the copy is failing because it can't find the volume to copy the image
> into.
>
> I would check the host and container for any iSCSI errors as well as the
> backend.  It appears that something is going wrong when attempting to
> temporarily attach the volume to write the image into it.
>
> Jay

Hi,

I've also seen this error when the initiator name in
/etc/iscsi/initiatorname.iscsi inside the container does not match the
one in use by the iscsid initiator daemon.

This can happen because the initiator name was changed after the daemon
started or because it is not shared between the container and the host.

I've also seen this happen (thought this is not the case) on VM
migrations when the driver has a bug and doesn't return the right
connection information (returns the first one).

I would recommend setting the log level to debug to see additional info
from OS-Brick.

I've debugged these type of issues many times, and if it's not a
production env I usually go with:

- Setting a breakpoint in the OS-Brick code: I stop at the right place
  and check the state of the system and how the volume has been exported
  and mapped in the backend.

- Installing cinderlib on a virtualenv (with --system-site-packages) in
  the cinder node, then using cinderlib to create a volume and debug an
  attach operation same as in previous step, like this:

  * Prepare the env:

    $ virtualenv --system-site-packages venv
    $ source venv/bin/activate
    (venv) $ pip install cinderlib

  * Run python interpreter
    (venv) $ python

  * Initialize cinderlib to store volumes in ./cl3.sqlite

    >>> import cinderlib as cl
    >>> db_connection = 'sqlite:///cl3.sqlite'
    >>> persistence_config = {'storage': 'db', 'connection': db_connection}
    >>> cl.setup(persistence_config=persistence_config, disable_logs=False,
                 debug=True)

  * Setup the backend.  You'll have to use your own configuration here:

    >>> sf = cl.Backend(
    ... volume_backend_name='solidfire',
    ... volume_driver='cinder.volume.drivers.solidfire.SolidFireDriver',
    ... san_ip='192.168.1.4',
    ... san_login='admin',
    ... san_password='admin_password',
    ... sf_allow_template_caching=False)

  * Create a 1GB empty volume:

    >>> vol = sf.create_volume(1)

  * Debug the attachment:

    >>> import pdb
    >>> pdb.run('att = vol.attach()')

- If it's a container I usually execute a bash terminal interactively
  and pip install cinderlib and do the debugging like in the step above.

Cheers,
Gorka.

>
> On 1/10/2019 7:16 AM, Grant Morley wrote:
> >
> > Hi all,
> >
> > We are in the process of trying to add a SolidFire storage solution to
> > our existing OpenStack setup and seem to have hit a snag with cinder /
> > iscsi.
> >
> > We are trying to create a bootable volume to allow us to launch an
> > instance from it, but we are getting some errors in our cinder-volumes
> > containers that seem to suggest they can't connect to iscsi although the
> > volume seems to create fine on the SolidFire node.
> >
> > The command we are running is:
> >
> > openstack volume create --image $image-id --size 20 --bootable --type
> > solidfire sf-volume-v12
> >
> > The volume seems to create on SolidFire but I then see these errors in
> > the "cinder-volume.log"
> >
> > https://pastebin.com/LyjLUhfk
> >
> > The volume containers can talk to the iscsi VIP on the SolidFire so I am
> > a bit stuck and wondered if anyone had come across any issues before?
> >
> > Kind Regards,
> >
> >
> > --
> > Grant Morley
> > Cloud Lead
> > Absolute DevOps Ltd
> > Units H, J & K, Gateway 1000, Whittle Way, Stevenage, Herts, SG1 2FP
> > www.absolutedevops.io <http://www.absolutedevops.io/>
> > grant at absolutedevops.io <mailto:grant at absolutedevops.io> 0845 874 0580