Open Stack

Tue Mar 3 06:37:33 UTC 2015

Sorry again ..... its been a trying week at work and I've only just had
time to pick this up

So, progress. I can now get the mons up and running. Took some fiddling
with, the $cluster param breaks just about everything so had to stick with
the default cluster name. I can live with that! I havent had to patch
anything this time round.

Next problem seems to be registering the osd_bootstrap key, the error is

Notice:
/Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns:
+ ceph --name mon. --keyring /var/lib/ceph/mon/ceph-ceph-001/keyring auth
add client.bootstrap-osd --in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring

Notice:
/Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns:
libust[28614/28614]: Warning: HOME environment variable not set. Disabling
LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:305)

Notice:
/Stage[main]/Cloud::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns:
Error EINVAL: entity client.bootstrap-osd exists but key does not match

Error: /bin/true # comment to satisfy puppet syntax requirements

set -ex

ceph   --name 'mon.'   --keyring '/var/lib/ceph/mon/ceph-ceph-001/keyring'
auth add client.bootstrap-osd
--in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring returned 22 instead of
one of [0]

Error:
/Stage[main]/Openfish::Roles::Ofceph/Ceph::Key[client.bootstrap-osd]/Exec[ceph-injectkey-client.bootstrap-osd]/returns:
change from notrun to 0 failed: /bin/true # comment to satisfy puppet
syntax requirements

set -ex

ceph   --name 'mon.'   --keyring '/var/lib/ceph/mon/ceph-ceph-001/keyring'
auth add client.bootstrap-osd
--in-file=/var/lib/ceph/bootstrap-osd/ceph.keyring returned 22 instead of
one of [0]

I havent gotten as far as the osd's or gateways yet

Help greatly appreciated!

On Tue, Feb 24, 2015 at 10:31 AM, David Gurtner <aldavud at crimson.ch> wrote:

> Hi again
>
> Just to test if the modules generally work on Trusty I set up a three
> node cluster with shared mon/osd using the configuration below and
> that worked. I was using the master branch of the module and had
> firewall disabled.
>
> Cheers,
> David
>
>
> $admin_key = 'AQCTg71RsNIHORAAW+O6FCMZWBjmVfMIPk3MhQ=='
> $mon_key = 'AQDesGZSsC7KJBAAw+W/Z4eGSQGAIbxWjxjvfw=='
> $bootstrap_osd_key = 'AQABsWZSgEDmJhAAkAGSOOAJwrMHrM5Pz5On1A=='
> $fsid = '066F558C-6789-4A93-AAF1-5AF1BA01A3AD'
>
> node /first|second|third/ {
>   class { 'ceph::repo': }
>   class { 'ceph':
>     fsid                      => $fsid,
>     mon_initial_members       => 'first,second,third',
>     mon_host                  => '10.11.12.2,10.11.12.3,10.11.12.4',
>   }
>   ceph::mon { $::hostname:
>     key => $mon_key,
>   }
>   Ceph::Key {
>     inject         => true,
>     inject_as_id   => 'mon.',
>     inject_keyring => "/var/lib/ceph/mon/ceph-${::hostname}/keyring",
>   }
>   ceph::key { 'client.admin':
>     secret  => $admin_key,
>     cap_mon => 'allow *',
>     cap_osd => 'allow *',
>     cap_mds => 'allow',
>   }
>   ceph::key { 'client.bootstrap-osd':
>     secret       => $bootstrap_osd_key,
>     keyring_path => '/var/lib/ceph/bootstrap-osd/ceph.keyring',
>     cap_mon      => 'allow profile bootstrap-osd',
>   }
>   ceph::osd { '/dev/sdb': }
> }
>
>
> On Mon, Feb 23, 2015 at 11:39 PM, David Gurtner <aldavud at crimson.ch>
> wrote:
> > Hi Stuart
> >
> > osds can only come up once at least the initial mon services are
> > running and formed a quorum, so lets concentrate at getting the mons
> > up first.
> > If using cephx authentication all mon services need to be started at
> > the same time (a similar time), translated to puppet this means all
> > the puppet runs on the mons need to be started within a certain time
> > frame (before the timeout of the mons trying to connect each other).
> > The reason being that the mons first need to form a quorum before most
> > of the further actions can happen.
> >
> > Can you provide some additional information about your setup?
> Specifically:
> > - what's your exact puppet modules configuration
> > - what version are you running?
> > - please be aware that running on Ubuntu 14.04 is currently not
> > getting integration tested (currently only Ubuntu 12.04 and CentOS 6
> > are tested, support for CentOS 7 is in the works).
> > - how many mons are you running
> > - are the osds running on the same hosts as the mons
> > - do you use a single network, or do you use separate cluster/public
> networks
> > - do you have a firewall running, if yes, can you share your firewall
> > configuration (iptables -vnL)
> > - what exactly did you need to patch?
> >
> > Cheers,
> > David
> >
> > On Mon, Feb 23, 2015 at 8:16 PM, Stuart Fox <stuart at demonware.net>
> wrote:
> >> Sorry to ask a question and run, I had a busy weekend. I'll be back on
> the
> >> ceph POC later today.
> >>
> >> I'll paste some outputs from the puppet runs shortly but essentially,
> no mon
> >> or osd comes up.
> >> I had to patch several manifests to explicitly reference
> /etc/ceph/ceph.conf
> >> and given that the puppet module calls a reasonable amount of shell
> code,
> >> getting debug info is not easy. The puppet runs exit correctly but no
> >> service comes up.
> >> I'm using an almost cut'n'paste from USECASES.md, even down to keeping
> the
> >> keys the same.
> >>
> >> BR,
> >> Stuart
> >>
> >> On Sun, Feb 22, 2015 at 6:57 AM, David Moreau Simard <dmsimard at iweb.com
> >
> >> wrote:
> >>>
> >>> Hey Stuart,
> >>>
> >>> You might want to look at (and use, if you want) the built-in roles and
> >>> profiles layer [1] which gives you a good idea of how the module is
> used.
> >>>
> >>> It leverages ceph::profile::params to pass values [2] to other classes
> >>> such as ceph::profile::osd [3].
> >>>
> >>> Another good place to look at how to use the module would be the
> >>> integration tests [4].
> >>> Since the module is integration tested, this means we actually use the
> >>> module to deploy a virtual Ceph cluster and we test that the cluster
> works -
> >>> see for example the tests for ceph::profile::osd [4].
> >>>
> >>> I'm dmsimard on #puppet-openstack and #openstack-operators if you need
> a
> >>> hand to get this to work.
> >>> This is still a relatively new module in comparison to the likes of
> >>> puppet-nova and such. Feedback and contributions are appreciated!
> >>>
> >>> [1]
> >>>
> https://github.com/stackforge/puppet-ceph/tree/master/manifests/profile
> >>> [2]
> >>>
> https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/params.pp
> >>> [3]
> >>>
> https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/osd.pp
> >>> [4]
> >>>
> https://github.com/stackforge/puppet-ceph/blob/master/spec/system/ceph_profile_osd_spec.rb
> >>> --
> >>> David Moreau Simard
> >>>
> >>> From: Andrew Woodward <xarses at gmail.com>
> >>> Date: Saturday, February 21, 2015 at 4:39 PM
> >>> To: Stuart Fox <stuart at demonware.net>
> >>> Cc: "openstack-operators at lists.openstack.org"
> >>> <openstack-operators at lists.openstack.org>, "
> puppet-openstack at puppetlabs.com"
> >>> <puppet-openstack at puppetlabs.com>
> >>> Subject: Re: [Openstack-operators] Ceph puppet module
> >>>
> >>> [+puppet-openstack ml]
> >>>
> >>> Stuart,
> >>>
> >>> Please review the USECASES.md such as [1]. You should find usable
> examples
> >>> there. If you are still having problems please reach out with a more
> detail
> >>> about what you configuration you are attempting to deploy. The module
> >>> authors can be found in #puppet-openstack and are usually lurking on
> the
> >>> puppet-openstack mailing list (CC'd).
> >>>
> >>>
> >>> [1]
> https://github.com/stackforge/puppet-ceph/blob/master/USECASES.md#i-want-to-operate-a-production-cluster
> >>>
> >>>
> >>>
> >>> On Sat, Feb 21, 2015 at 12:04 AM, Stuart Fox <stuart at demonware.net>
> wrote:
> >>>>
> >>>> Hey all
> >>>>
> >>>> Im having a complete nightmare trying to get ceph deployed using the
> >>>> https://github.com/stackforge/puppet-ceph module. Scant
> documentation isn't
> >>>> helping my cause!
> >>>>
> >>>> Is anybody else using this module? Im attempting to deploy ceph as a
> >>>> cinder backend in Juno on Ubuntu 14.04 although I haven't gotten as
> far as
> >>>> integration yet.
> >>>>
> >>>> Does anyone have working example's I could look at?
> >>>>
> >>>> --
> >>>> BR,
> >>>> Stuart
> >>>>
> >>>> _______________________________________________
> >>>> OpenStack-operators mailing list
> >>>> OpenStack-operators at lists.openstack.org
> >>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Andrew
> >>> Mirantis
> >>> Fuel community ambassador
> >>> Ceph community
> >>
> >>
> >>
> >>
> >> --
> >> BR,
> >> Stuart
> >>
> >> _______________________________________________
> >> OpenStack-operators mailing list
> >> OpenStack-operators at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >>
>

-- 
BR,
Stuart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150302/5f302d70/attachment.html>

Open Stack

[Openstack-operators] Ceph puppet module

OpenStack

Community

Documentation

Branding & Legal