Open Stack

On 09/22/2015 06:43 PM, Robert Collins wrote:
> On 23 September 2015 at 09:52, Chris Friesen 
> <chris.friesen at windriver.com> wrote:
>> Hi,
>>
>> I recently had an issue with one file out of a dozen or so in 
>> "/opt/cgcs/cinder/data/volumes/" being present but of size zero.  I'm 
>> running stable/kilo if it makes a difference.
>>
>> Looking at the code in 
>> volume.targets.tgt.TgtAdm.create_iscsi_target(), I'm wondering if we 
>> should do a fsync() before the close().  The way it stands now, it 
>> seems like it might be possible to write the file, start making use 
>> of it, and then take a power outage before it actually gets written 
>> to persistent storage.  When we come back up we could have an 
>> instance expecting to make use of it, but no target information in the on-disk copy of the file.

I think even if there is no target information in configuration file dir, c-vol started successfully
and iSCSI targets were created automatically and volumes were exported, right?

There is an problem in this case that the iSCSI target was created without authentication because
we can't get previous authentication from the configuration file.

I'm curious what kind of problem did you met?
  
> If its being kept in sync with DB records, and won't self-heal from 
> this situation, then yes. e.g. if the overall workflow is something 
> like

In my understanding, the provider_auth in database has user name and password for iSCSI target. 
Therefore if we get authentication from DB, I think we can self-heal from this situation
correctly after c-vol service is restarted.

The lio target obtains authentication from provider_auth in database, but tgtd, iet, cxt obtain
authentication from file to recreate iSCSI target when c-vol is restarted.
If the file is missing, these volumes are exported without authentication and configuration
file is recreated as I mentioned above.

tgtd: Get target chap auth from file
iet:  Get target chap auth from file
cxt:  Get target chap auth from file
lio:  Get target chap auth from Database(in provider_auth)
scst: Get target chap auth by using original command

If we get authentication from DB for tgtd, iet and cxt same as lio, we can recreate iSCSI target
with proper authentication when c-vol is restarted.
I think this is a solution for this situation.

Any thought?

Thanks,
Mitsuhiro Tanino

> -----Original Message-----
> From: Chris Friesen [mailto:chris.friesen at windriver.com]
> Sent: Friday, September 25, 2015 12:48 PM
> To: openstack-dev at lists.openstack.org
> Subject: Re: [openstack-dev] [cinder] should we use fsync when writing iscsi
> config file?
> 
> On 09/24/2015 04:21 PM, Chris Friesen wrote:
> > On 09/24/2015 12:18 PM, Chris Friesen wrote:
> >
> >>
> >> I think what happened is that we took the SIGTERM after the open()
> >> call in create_iscsi_target(), but before writing anything to the file.
> >>
> >>          f = open(volume_path, 'w+')
> >>          f.write(volume_conf)
> >>          f.close()
> >>
> >> The 'w+' causes the file to be immediately truncated on opening,
> >> leading to an empty file.
> >>
> >> To work around this, I think we need to do the classic "write to a
> >> temporary file and then rename it to the desired filename" trick.
> >> The atomicity of the rename ensures that either the old contents or the new
> contents are present.
> >
> > I'm pretty sure that upstream code is still susceptible to zeroing out
> > the file in the above scenario.  However, it doesn't take an
> > exception--that's due to a local change on our part that attempted to fix the
> below issue.
> >
> > The stable/kilo code *does* have a problem in that when it regenerates
> > the file it's missing the CHAP authentication line (beginning with
> "incominguser").
> 
> I've proposed a change at https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__review.openstack.org_-23_c_227943_&d=BQICAg&c=DZ-
> EF4pZfxGSU6MfABwx0g&r=klD1krzABGW034E9oBtY1xmIn3g7xZAIxV0XxaZpkJE&m=SVlOqKiqO04_
> NttKUIoDiaOR7cePB0SOA1bpjakqAss&s=q2_8XBAVH9lQ2mdT72nW7dN2EafIqJEpHGLBuf4K970&e=
> 
> If anyone has suggestions on how to do this more robustly or more cleanly,
> please let me know.
> 
> Chris
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-
> 2Dbin_mailman_listinfo_openstack-2Ddev&d=BQICAg&c=DZ-
> EF4pZfxGSU6MfABwx0g&r=klD1krzABGW034E9oBtY1xmIn3g7xZAIxV0XxaZpkJE&m=SVlOqKiqO04_
> NttKUIoDiaOR7cePB0SOA1bpjakqAss&s=0DBbmeXSIK2c5QlBnwURY1iwNN1AXuqOLaUYnxjBl0w&e=