Open Stack

Hi again

Just to test if the modules generally work on Trusty I set up a three
node cluster with shared mon/osd using the configuration below and
that worked. I was using the master branch of the module and had
firewall disabled.

Cheers,
David


$admin_key = 'AQCTg71RsNIHORAAW+O6FCMZWBjmVfMIPk3MhQ=='
$mon_key = 'AQDesGZSsC7KJBAAw+W/Z4eGSQGAIbxWjxjvfw=='
$bootstrap_osd_key = 'AQABsWZSgEDmJhAAkAGSOOAJwrMHrM5Pz5On1A=='
$fsid = '066F558C-6789-4A93-AAF1-5AF1BA01A3AD'

node /first|second|third/ {
  class { 'ceph::repo': }
  class { 'ceph':
    fsid                      => $fsid,
    mon_initial_members       => 'first,second,third',
    mon_host                  => '10.11.12.2,10.11.12.3,10.11.12.4',
  }
  ceph::mon { $::hostname:
    key => $mon_key,
  }
  Ceph::Key {
    inject         => true,
    inject_as_id   => 'mon.',
    inject_keyring => "/var/lib/ceph/mon/ceph-${::hostname}/keyring",
  }
  ceph::key { 'client.admin':
    secret  => $admin_key,
    cap_mon => 'allow *',
    cap_osd => 'allow *',
    cap_mds => 'allow',
  }
  ceph::key { 'client.bootstrap-osd':
    secret       => $bootstrap_osd_key,
    keyring_path => '/var/lib/ceph/bootstrap-osd/ceph.keyring',
    cap_mon      => 'allow profile bootstrap-osd',
  }
  ceph::osd { '/dev/sdb': }
}


On Mon, Feb 23, 2015 at 11:39 PM, David Gurtner <aldavud at crimson.ch> wrote:
> Hi Stuart
>
> osds can only come up once at least the initial mon services are
> running and formed a quorum, so lets concentrate at getting the mons
> up first.
> If using cephx authentication all mon services need to be started at
> the same time (a similar time), translated to puppet this means all
> the puppet runs on the mons need to be started within a certain time
> frame (before the timeout of the mons trying to connect each other).
> The reason being that the mons first need to form a quorum before most
> of the further actions can happen.
>
> Can you provide some additional information about your setup? Specifically:
> - what's your exact puppet modules configuration
> - what version are you running?
> - please be aware that running on Ubuntu 14.04 is currently not
> getting integration tested (currently only Ubuntu 12.04 and CentOS 6
> are tested, support for CentOS 7 is in the works).
> - how many mons are you running
> - are the osds running on the same hosts as the mons
> - do you use a single network, or do you use separate cluster/public networks
> - do you have a firewall running, if yes, can you share your firewall
> configuration (iptables -vnL)
> - what exactly did you need to patch?
>
> Cheers,
> David
>
> On Mon, Feb 23, 2015 at 8:16 PM, Stuart Fox <stuart at demonware.net> wrote:
>> Sorry to ask a question and run, I had a busy weekend. I'll be back on the
>> ceph POC later today.
>>
>> I'll paste some outputs from the puppet runs shortly but essentially, no mon
>> or osd comes up.
>> I had to patch several manifests to explicitly reference /etc/ceph/ceph.conf
>> and given that the puppet module calls a reasonable amount of shell code,
>> getting debug info is not easy. The puppet runs exit correctly but no
>> service comes up.
>> I'm using an almost cut'n'paste from USECASES.md, even down to keeping the
>> keys the same.
>>
>> BR,
>> Stuart
>>
>> On Sun, Feb 22, 2015 at 6:57 AM, David Moreau Simard <dmsimard at iweb.com>
>> wrote:
>>>
>>> Hey Stuart,
>>>
>>> You might want to look at (and use, if you want) the built-in roles and
>>> profiles layer [1] which gives you a good idea of how the module is used.
>>>
>>> It leverages ceph::profile::params to pass values [2] to other classes
>>> such as ceph::profile::osd [3].
>>>
>>> Another good place to look at how to use the module would be the
>>> integration tests [4].
>>> Since the module is integration tested, this means we actually use the
>>> module to deploy a virtual Ceph cluster and we test that the cluster works -
>>> see for example the tests for ceph::profile::osd [4].
>>>
>>> I'm dmsimard on #puppet-openstack and #openstack-operators if you need a
>>> hand to get this to work.
>>> This is still a relatively new module in comparison to the likes of
>>> puppet-nova and such. Feedback and contributions are appreciated!
>>>
>>> [1]
>>> https://github.com/stackforge/puppet-ceph/tree/master/manifests/profile
>>> [2]
>>> https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/params.pp
>>> [3]
>>> https://github.com/stackforge/puppet-ceph/blob/master/manifests/profile/osd.pp
>>> [4]
>>> https://github.com/stackforge/puppet-ceph/blob/master/spec/system/ceph_profile_osd_spec.rb
>>> --
>>> David Moreau Simard
>>>
>>> From: Andrew Woodward <xarses at gmail.com>
>>> Date: Saturday, February 21, 2015 at 4:39 PM
>>> To: Stuart Fox <stuart at demonware.net>
>>> Cc: "openstack-operators at lists.openstack.org"
>>> <openstack-operators at lists.openstack.org>, "puppet-openstack at puppetlabs.com"
>>> <puppet-openstack at puppetlabs.com>
>>> Subject: Re: [Openstack-operators] Ceph puppet module
>>>
>>> [+puppet-openstack ml]
>>>
>>> Stuart,
>>>
>>> Please review the USECASES.md such as [1]. You should find usable examples
>>> there. If you are still having problems please reach out with a more detail
>>> about what you configuration you are attempting to deploy. The module
>>> authors can be found in #puppet-openstack and are usually lurking on the
>>> puppet-openstack mailing list (CC'd).
>>>
>>>
>>> [1]https://github.com/stackforge/puppet-ceph/blob/master/USECASES.md#i-want-to-operate-a-production-cluster
>>>
>>>
>>>
>>> On Sat, Feb 21, 2015 at 12:04 AM, Stuart Fox <stuart at demonware.net> wrote:
>>>>
>>>> Hey all
>>>>
>>>> Im having a complete nightmare trying to get ceph deployed using the
>>>> https://github.com/stackforge/puppet-ceph module. Scant documentation isn't
>>>> helping my cause!
>>>>
>>>> Is anybody else using this module? Im attempting to deploy ceph as a
>>>> cinder backend in Juno on Ubuntu 14.04 although I haven't gotten as far as
>>>> integration yet.
>>>>
>>>> Does anyone have working example's I could look at?
>>>>
>>>> --
>>>> BR,
>>>> Stuart
>>>>
>>>> _______________________________________________
>>>> OpenStack-operators mailing list
>>>> OpenStack-operators at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>>
>>>
>>>
>>>
>>> --
>>> Andrew
>>> Mirantis
>>> Fuel community ambassador
>>> Ceph community
>>
>>
>>
>>
>> --
>> BR,
>> Stuart
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>