Open Stack

Guys,

Please, ignore my last message... I'm thinking that I'm facing a different
problem, out off this topic.

Now, instances can not ping its gateway (Neutron L3 is unreachable). I'll
make a different post if necessary.

Sorry about the buzz.

Tks,
Thiago


On 8 March 2014 16:53, Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote:

> Well, I think I just destroyed the entire Cloud Computing environment...
> Security Groups isn't working, it is impossible to reach the tenant's
> subnet, they are just closed (i.e. not working), all sites hosted here, are
> down. Damn... What a mess...   :-/
>
> Is there any way to try to fix this? I have the dump of keystone db,
> before dropping the INDEXes...
>
>
> On 8 March 2014 16:12, Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote:
>
>> Stackers,
>>
>> I'll mark this as fixed, look:
>>
>> ---
>> root at controller-1:~# apt-get -f install
>> Reading package lists... Done
>> Building dependency tree
>> Reading state information... Done
>> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>> 1 not fully installed or removed.
>> After this operation, 0 B of additional disk space will be used.
>> *Setting up keystone (1:2013.2.2-0ubuntu1~cloud0) ...*
>> ---
>>
>> Keystone seems to be upgraded now...
>>
>> Thanks!
>> Thiago
>>
>>
>> On 8 March 2014 16:06, Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote:
>>
>>> Well,
>>>
>>> I just did a shot in the dark:
>>>
>>>
>>> > ALTER TABLE `keystone`.`token` DROP INDEX `ix_token_expires_valid`;
>>>
>>> And "keystone-manage db_sync" seems to be running again... But, I have
>>> no idea if this DROP I did will break my OpenStack, or not...
>>>
>>> I don't want to waste more time from you guys with this but, maybe it is
>>> not a big deal... What do you guys think?!
>>>
>>> Thanks!
>>> Thiago
>>>
>>>
>>> On 8 March 2014 15:52, Martinx - ジェームズ <thiagocmartinsc at gmail.com>wrote:
>>>
>>>> Hi!
>>>>
>>>> I'm seeing some progress here, look:
>>>>
>>>> ---
>>>> root at controller-1:~# pt-duplicate-key-checker --host localhost
>>>> --databases=keystone --ask-pass
>>>> Enter password:
>>>> #
>>>> ########################################################################
>>>> # keystone.token
>>>>
>>>> #
>>>> ########################################################################
>>>>
>>>> # ix_token_expires is a left-prefix of ix_token_expires_valid
>>>> # Key definitions:
>>>> #   KEY `ix_token_expires` (`expires`),
>>>> #   KEY `ix_token_expires_valid` (`expires`,`valid`)
>>>> # Column types:
>>>> #         `expires` datetime default null
>>>> #         `valid` tinyint(1) not null
>>>> # To remove this duplicate index, execute:
>>>> ALTER TABLE `keystone`.`token` DROP INDEX `ix_token_expires`;
>>>>
>>>> #
>>>> ########################################################################
>>>> # Summary of indexes
>>>>
>>>> #
>>>> ########################################################################
>>>>
>>>> # Size Duplicate Indexes   1321470
>>>> # Total Duplicate Indexes  1
>>>> # Total Indexes            42
>>>> ----
>>>>
>>>> ---
>>>> mysql> ALTER TABLE `keystone`.`token` DROP INDEX `ix_token_expires`;
>>>> Query OK, 0 rows affected (1.03 sec)
>>>> Records: 0  Duplicates: 0  Warnings: 0
>>>> ---
>>>>
>>>> ---
>>>> No more duplicate key found by "pt-duplicate-key-checker" but, same
>>>> problem with "keystone-manage "db_sync", look:
>>>>
>>>> root at controller-1:~# keystone-manage db_sync
>>>> 2014-03-08 15:46:59.186 8043 CRITICAL keystone [-] (OperationalError)
>>>> (1061, "Duplicate key name 'ix_token_expires_valid'") 'CREATE INDEX
>>>> ix_token_expires_valid ON token (expires, valid)' ()
>>>> ---
>>>>
>>>> Guys, what can I do!? I really need to keep this OpenStack installation
>>>> intact, I can not afford losing it...
>>>>
>>>> Tks!
>>>> Thiago
>>>>
>>>>
>>>> On 8 March 2014 04:39, Martinx - ジェームズ <thiagocmartinsc at gmail.com>wrote:
>>>>
>>>>> Lance,
>>>>>
>>>>> I think I know what had happened... The Keystone upgrade process died
>>>>> in the middle of the upgrade.
>>>>>
>>>>> Now, the Keystone SQL database is stucked at some point "in the
>>>>> middle" of both versions...
>>>>>
>>>>> My "keystone" MySQL db dump is huge, more than >1G...
>>>>>
>>>>> Any tips about how to fix this?!
>>>>>
>>>>> I really appreciate any help!
>>>>>
>>>>> Tks!
>>>>> Thiago
>>>>>
>>>>>
>>>>> On 8 March 2014 02:55, Martinx - ジェームズ <thiagocmartinsc at gmail.com>wrote:
>>>>>
>>>>>> Lance,
>>>>>>
>>>>>> I can provide everything, no problem... I'm doing this to reproduce
>>>>>> it:
>>>>>>
>>>>>>
>>>>>> root at controller-1:~# keystone-manage db_sync
>>>>>> 2014-03-08 02:44:46.723 5407 CRITICAL keystone [-] (OperationalError)
>>>>>> (1061, "Duplicate key name 'ix_token_expires_valid'") 'CREATE INDEX
>>>>>> ix_token_expires_valid ON token (expires, valid)' ()
>>>>>>
>>>>>>
>>>>>> Log shows:
>>>>>>
>>>>>> ==> /var/log/keystone/keystone.log <==
>>>>>> 2014-03-08 02:44:46.723 5407 CRITICAL keystone [-] (OperationalError)
>>>>>> (1061, "Duplicate key name 'ix_token_expires_valid'") 'CREATE INDEX
>>>>>> ix_token_expires_valid ON token (expires, valid)' ()
>>>>>>
>>>>>>
>>>>>> root at controller-1:~# dpkg -l | grep keystone
>>>>>> iF  keystone                         1:2013.2.2-0ubuntu1~cloud0
>>>>>>                    OpenStack identity service - Daemons
>>>>>> ii  python-keystone                  1:2013.2.2-0ubuntu1~cloud0
>>>>>>                    OpenStack identity service - Python library
>>>>>> ii  python-keystoneclient            1:0.3.2-0ubuntu1~cloud0
>>>>>>                     Client library for OpenStack Identity API
>>>>>>
>>>>>> ...
>>>>>>
>>>>>> If I run "apt-get -f install", to try to fix it ("iF" dpkg status
>>>>>> above), same problem, look:
>>>>>>
>>>>>> ---
>>>>>> root at controller-1:~# apt-get -f install
>>>>>> Reading package lists... Done
>>>>>> Building dependency tree
>>>>>> Reading state information... Done
>>>>>> 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
>>>>>> 1 not fully installed or removed.
>>>>>> After this operation, 0 B of additional disk space will be used.
>>>>>> Setting up keystone (1:2013.2.2-0ubuntu1~cloud0) ...
>>>>>> 2014-03-08 02:49:11.050 5469 CRITICAL keystone [-] (OperationalError)
>>>>>> (1061, "Duplicate key name 'ix_token_expires_valid'") 'CREATE INDEX
>>>>>> ix_token_expires_valid ON token (expires, valid)' ()
>>>>>>  dpkg: error processing keystone (--configure):
>>>>>>  subprocess installed post-installation script returned error exit
>>>>>> status 1
>>>>>> Errors were encountered while processing:
>>>>>>  keystone
>>>>>> E: Sub-process /usr/bin/dpkg returned an error code (1)
>>>>>> ---
>>>>>>
>>>>>> I'm lost...    :-/
>>>>>>
>>>>>> Maybe the upgrade procedure got stucked, then the second time I
>>>>>> tried, it triggered this error... I'm not sure...
>>>>>>
>>>>>> Tks!
>>>>>> Thiago
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 8 March 2014 01:55, Lance D Bragstad <ldbragst at us.ibm.com> wrote:
>>>>>>
>>>>>>> Thiago,
>>>>>>>
>>>>>>> Are you able to provide a little more information? Log files maybe?
>>>>>>> What are the steps you're taking to recreate the issue?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Best Regards,
>>>>>>>
>>>>>>> Lance Bragstad
>>>>>>> ldbragst at us.ibm.com
>>>>>>>
>>>>>>> Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote on 03/07/2014
>>>>>>> 08:34:11 PM:
>>>>>>>
>>>>>>>
>>>>>>> > From: Martinx - ジェームズ <thiagocmartinsc at gmail.com>
>>>>>>>
>>>>>>> > To: Lance D Bragstad/Rochester/IBM at IBMUS,
>>>>>>> > Cc: "openstack at lists.openstack.org" <openstack at lists.openstack.org
>>>>>>> >
>>>>>>> > Date: 03/07/2014 08:34 PM
>>>>>>> > Subject: Re: [Openstack] Error on upgrading to latest Keystone -
>>>>>>> > Ubuntu 12.04.4 + UCA
>>>>>>> >
>>>>>>> > Thanks Lance!
>>>>>>>
>>>>>>> >
>>>>>>> > I just did: "keystone-manage db_sync" and the output is the same:
>>>>>>> >
>>>>>>> > ---
>>>>>>> > 2014-03-07 23:23:48.564 2258 CRITICAL keystone [-]
>>>>>>> > (OperationalError) (1061, "Duplicate key name
>>>>>>> > 'ix_token_expires_valid'") 'CREATE INDEX ix_token_expires_valid ON
>>>>>>> > token (expires, valid)' ()
>>>>>>> > ---
>>>>>>> >
>>>>>>> > I just read [1] and [2] but, not a clue about how to fix it.
>>>>>>> >
>>>>>>> > My Keystone is still running and I have no idea if I can restart
>>>>>>> it
>>>>>>> > or not... What if it doesn't come back?! My entire Cloud will
>>>>>>> become
>>>>>>> > useless... This is a production environment, it was working
>>>>>>> smoothly
>>>>>>> > for months...    :'(
>>>>>>> >
>>>>>>> > Do you know if there is something that I can do directly on MySQL
>>>>>>> to
>>>>>>> > try to fix it?!
>>>>>>> >
>>>>>>> > Best,
>>>>>>> > Thiago
>>>>>>> >
>>>>>>>
>>>>>>> > On 7 March 2014 23:18, Lance D Bragstad <ldbragst at us.ibm.com>
>>>>>>> wrote:
>>>>>>> > Thiago,
>>>>>>> >
>>>>>>> > Your database might be out of sync with the latest Keystone code
>>>>>>> if
>>>>>>> > you just performed an update. Try migrating your database [1]. The
>>>>>>> > link gives some information on how to do that, but you'll
>>>>>>> > essentially want to do a `keystone-manage db_sync`, also check the
>>>>>>> > man pages [2]. This should ensure your Keystone database has the
>>>>>>> > proper schema for the Keystone code you're running.
>>>>>>> >
>>>>>>> > [1] http://docs.openstack.org/developer/keystone/
>>>>>>> > configuration.html#step-2-sync-your-new-empty-database
>>>>>>> > [2]
>>>>>>> http://docs.openstack.org/developer/keystone/man/keystone-manage.html
>>>>>>> >
>>>>>>> > Best Regards,
>>>>>>> >
>>>>>>> > Lance Bragstad
>>>>>>> > ldbragst at us.ibm.com
>>>>>>> >
>>>>>>> > Martinx - ジェームズ <thiagocmartinsc at gmail.com> wrote on 03/07/2014
>>>>>>> > 06:50:01 PM:
>>>>>>> >
>>>>>>> > > From: Martinx - ジェームズ <thiagocmartinsc at gmail.com>
>>>>>>> > > To: "openstack at lists.openstack.org" <
>>>>>>> openstack at lists.openstack.org>,
>>>>>>> > > Date: 03/07/2014 07:03 PM
>>>>>>> > > Subject: [Openstack] Error on upgrading to latest Keystone -
>>>>>>> Ubuntu
>>>>>>> > > 12.04.4 + UCA
>>>>>>> >
>>>>>>> > >
>>>>>>> > > Guys,
>>>>>>> > >
>>>>>>> > > I just did:
>>>>>>> > >
>>>>>>> > > apt-get update
>>>>>>> > > apt-get dist-upgrade
>>>>>>> > >
>>>>>>> > > And I'm seeing the following error:
>>>>>>> > >
>>>>>>> > > ---
>>>>>>> > > Setting up keystone (1:2013.2.2-0ubuntu1~cloud0) ...
>>>>>>> > > 2014-03-07 21:45:39.473 15540 CRITICAL keystone [-]
>>>>>>> > > (OperationalError) (1061, "Duplicate key name
>>>>>>> > > 'ix_token_expires_valid'") 'CREATE INDEX ix_token_expires_valid
>>>>>>> ON
>>>>>>> > > token (expires, valid)' ()
>>>>>>> > > dpkg: error processing keystone (--configure):
>>>>>>> > >  subprocess installed post-installation script returned error
>>>>>>> exit status 1
>>>>>>> > > Errors were encountered while processing:
>>>>>>> > >  keystone
>>>>>>> > > E: Sub-process /usr/bin/dpkg returned an error code (1)
>>>>>>> > > ---
>>>>>>> > >
>>>>>>> > > What can I do?!
>>>>>>> > >
>>>>>>> > > Tks!
>>>>>>> > > Thiago_______________________________________________
>>>>>>> > > Mailing list:
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>>> > > Post to     : openstack at lists.openstack.org
>>>>>>> > > Unsubscribe :
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140308/9b18ab20/attachment.html>