[manila][cinder][glance][nova] Pop-up team for design and development of a Cephadm DevStack plugin
Hi all, I'm reaching out to you to let you know that we will start the design and development of a Cephadm DevStack plugin. Some of the reasons on why we want to take this approach: - devstack-plugin-ceph worked for us for a lot of years, but the development of it relies on several hacks to adapt to the different Ceph versions we use and the different distros we support. This led to a monolithic script that sometimes is hard to debug and break our development environments and our CI - cephadm is the deployment tool developed and maintained by the Ceph community, it allows their users to get specific Ceph versions very easily and enforces good practices for Ceph clusters. From their docs, "Cephadm manages the full lifecycle of a Ceph cluster. It starts by bootstrapping a tiny Ceph cluster on a single node (one monitor and one manager) and then uses the orchestration interface (“day 2” commands) to expand the cluster to include all hosts and to provision all Ceph daemons and services. [0]" - OpenStack deployment tools are starting to use cephadm as their way to deploy Ceph, so it would be nice to include cephadm in our development process to be closer with what is being done in the field I started the development of this in [1], but it might be better to change devstack-plugin-ceph to do this instead of having a new plugin. This is something I would love to discuss in a first meeting. Having said this, I propose using the channel #openstack-cephadm in the OFTC network to talk about this and set up a first meeting with people interested in contributing to this effort. Thanks, Victoria [0] https://docs.ceph.com/en/pacific/cephadm/ [1] https://github.com/vkmc/devstack-plugin-cephadm
On Wed, 2022-01-19 at 12:04 +0100, Victoria Martínez de la Cruz wrote:
Hi all,
I'm reaching out to you to let you know that we will start the design and development of a Cephadm DevStack plugin.
Some of the reasons on why we want to take this approach:
- devstack-plugin-ceph worked for us for a lot of years, but the development of it relies on several hacks to adapt to the different Ceph versions we use and the different distros we support. This led to a monolithic script that sometimes is hard to debug and break our development environments and our CI - cephadm is the deployment tool developed and maintained by the Ceph community, it allows their users to get specific Ceph versions very easily and enforces good practices for Ceph clusters. From their docs, "Cephadm manages the full lifecycle of a Ceph cluster. It starts by bootstrapping a tiny Ceph cluster on a single node (one monitor and one manager) and then uses the orchestration interface (“day 2” commands) to expand the cluster to include all hosts and to provision all Ceph daemons and services. [0]" - OpenStack deployment tools are starting to use cephadm as their way to deploy Ceph, so it would be nice to include cephadm in our development process to be closer with what is being done in the field
I started the development of this in [1], but it might be better to change devstack-plugin-ceph to do this instead of having a new plugin. This is something I would love to discuss in a first meeting. i would advocate for pivoting devstack-plugin-ceph. i dont think we have the capsity as a comunity to devleop, maintaine and debug/support 2 differnt ways of deploying ceph in our ci system in the long term.
to me the way devstack-plugin-ceph install cpeh is jsut an implementaion detail. its contract is that it will install and configure ceph for use with openstack. if you make it use cephadm for that its just and internal detail that should not affect the consomes of the plugin provide you maintain the interface to the devstack pluging mostly the same. i would suggest addign a devstack macro initally to choose the backend but then eventually once the cephadm appoch is stable just swap the default.
Having said this, I propose using the channel #openstack-cephadm in the OFTC network to talk about this and set up a first meeting with people interested in contributing to this effort.
ack im not sure i will get involed with this but the other option woudl be to just use #openstack-qa since that is the chanlle for devstack development.
Thanks,
Victoria
[0] https://docs.ceph.com/en/pacific/cephadm/ [1] https://github.com/vkmc/devstack-plugin-cephadm
Hi Victoria, thanks for starting this thread. On Wed, Jan 19, 2022 at 2:03 PM Sean Mooney <smooney@redhat.com> wrote:
On Wed, 2022-01-19 at 12:04 +0100, Victoria Martínez de la Cruz wrote:
Hi all,
I'm reaching out to you to let you know that we will start the design and development of a Cephadm DevStack plugin.
Some of the reasons on why we want to take this approach:
- devstack-plugin-ceph worked for us for a lot of years, but the development of it relies on several hacks to adapt to the different Ceph versions we use and the different distros we support. This led to a monolithic script that sometimes is hard to debug and break our development environments and our CI - cephadm is the deployment tool developed and maintained by the Ceph community, it allows their users to get specific Ceph versions very easily and enforces good practices for Ceph clusters. From their docs, "Cephadm manages the full lifecycle of a Ceph cluster. It starts by bootstrapping a tiny Ceph cluster on a single node (one monitor and one manager) and then uses the orchestration interface (“day 2” commands) to expand the cluster to include all hosts and to provision all Ceph daemons and services. [0]" - OpenStack deployment tools are starting to use cephadm as their way to deploy Ceph, so it would be nice to include cephadm in our development process to be closer with what is being done in the field
I started the development of this in [1], but it might be better to change devstack-plugin-ceph to do this instead of having a new plugin. This is something I would love to discuss in a first meeting. i would advocate for pivoting devstack-plugin-ceph. i dont think we have the capsity as a comunity to devleop, maintaine and debug/support 2 differnt ways of deploying ceph in our ci system in the long term.
to me the way devstack-plugin-ceph install cpeh is jsut an implementaion detail. its contract is that it will install and configure ceph for use with openstack. if you make it use cephadm for that its just and internal detail that should not affect the consomes of the plugin provide you maintain the interface to the devstack pluging mostly the same.
Starting with pacific the deployment of Ceph is moved from ceph-ansible to cephadm: the implication of this change it's not just on the deployment side but this new component (which interacts with the ceph orchestrator module) is able to maintain the lifecycle of the deployed containers, so I'd say the new approach it's not just an implementation detail but also changes the way some components interact with Ceph. Manila using ganesha, for instance, it's the first component that should start using the orchestrator interface, so I guess it's worth aligning (and extending) the most popular dev installer to support the new way (like other projects already did).
i would suggest addign a devstack macro initally to choose the backend but then eventually once the cephadm appoch is stable just swap the default.
+1 on choosing the backend and plan the switch when the cephadm approach is ready and works for all the openstack components
Having said this, I propose using the channel #openstack-cephadm in the OFTC network to talk about this and set up a first meeting with people interested in contributing to this effort.
ack im not sure i will get involed with this but the other option woudl be to just use #openstack-qa since that is the chanlle for devstack development.
Either #openstack-qa or a dedicated one works well , maybe #openstack-qa is useful to reach more people who can help / review the relevant changes .. wdyt
Thanks,
Victoria
[0] https://docs.ceph.com/en/pacific/cephadm/ [1] https://github.com/vkmc/devstack-plugin-cephadm
-- Francesco Pantano GPG KEY: F41BD75C
Sounds good. I wanna join . I haven't tried cephadm yet but It would help us to make ceph new features more transparent to Cinder in the future. Thanks On Thu, Jan 20, 2022 at 4:45 AM Francesco Pantano <fpantano@redhat.com> wrote:
Hi Victoria, thanks for starting this thread.
On Wed, Jan 19, 2022 at 2:03 PM Sean Mooney <smooney@redhat.com> wrote:
Hi all,
I'm reaching out to you to let you know that we will start the design and development of a Cephadm DevStack plugin.
Some of the reasons on why we want to take this approach:
- devstack-plugin-ceph worked for us for a lot of years, but the development of it relies on several hacks to adapt to the different Ceph versions we use and the different distros we support. This led to a monolithic script that sometimes is hard to debug and break our development environments and our CI - cephadm is the deployment tool developed and maintained by the Ceph community, it allows their users to get specific Ceph versions very easily and enforces good practices for Ceph clusters. From their docs, "Cephadm manages the full lifecycle of a Ceph cluster. It starts by bootstrapping a tiny Ceph cluster on a single node (one monitor and one manager) and
On Wed, 2022-01-19 at 12:04 +0100, Victoria Martínez de la Cruz wrote: then
uses the orchestration interface (“day 2” commands) to expand the cluster to include all hosts and to provision all Ceph daemons and services. [0]" - OpenStack deployment tools are starting to use cephadm as their way to deploy Ceph, so it would be nice to include cephadm in our development process to be closer with what is being done in the field
I started the development of this in [1], but it might be better to change devstack-plugin-ceph to do this instead of having a new plugin. This is something I would love to discuss in a first meeting. i would advocate for pivoting devstack-plugin-ceph. i dont think we have the capsity as a comunity to devleop, maintaine and debug/support 2 differnt ways of deploying ceph in our ci system in the long term.
to me the way devstack-plugin-ceph install cpeh is jsut an implementaion detail. its contract is that it will install and configure ceph for use with openstack. if you make it use cephadm for that its just and internal detail that should not affect the consomes of the plugin provide you maintain the interface to the devstack pluging mostly the same.
Starting with pacific the deployment of Ceph is moved from ceph-ansible to cephadm: the implication of this change it's not just on the deployment side but this new component (which interacts with the ceph orchestrator module) is able to maintain the lifecycle of the deployed containers, so I'd say the new approach it's not just an implementation detail but also changes the way some components interact with Ceph. Manila using ganesha, for instance, it's the first component that should start using the orchestrator interface, so I guess it's worth aligning (and extending) the most popular dev installer to support the new way (like other projects already did).
i would suggest addign a devstack macro initally to choose the backend but then eventually once the cephadm appoch is stable just swap the default.
+1 on choosing the backend and plan the switch when the cephadm approach is ready and works for all the openstack components
Having said this, I propose using the channel #openstack-cephadm in the OFTC network to talk about this and set up a first meeting with people interested in contributing to this effort.
ack im not sure i will get involed with this but the other option woudl be to just use #openstack-qa since that is the chanlle for devstack development.
Either #openstack-qa or a dedicated one works well , maybe #openstack-qa is useful to reach more people who can help / review the relevant changes .. wdyt
Thanks,
Victoria
[0] https://docs.ceph.com/en/pacific/cephadm/ [1] https://github.com/vkmc/devstack-plugin-cephadm
-- Francesco Pantano GPG KEY: F41BD75C
-- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>
Thanks everybody for your responses! I'll send an initial path with a macro to pick between implementations and we can go from there. Let's have a meeting next Friday at 3pm UTC in #openstack-qa at OFTC. Would that work for you? Thanks, Victoria On Thu, Jan 20, 2022 at 2:04 PM Sofia Enriquez <senrique@redhat.com> wrote:
Sounds good. I wanna join . I haven't tried cephadm yet but It would help us to make ceph new features more transparent to Cinder in the future. Thanks
++
On Thu, Jan 20, 2022 at 4:45 AM Francesco Pantano <fpantano@redhat.com> wrote:
Hi Victoria, thanks for starting this thread.
On Wed, Jan 19, 2022 at 2:03 PM Sean Mooney <smooney@redhat.com> wrote:
Hi all,
I'm reaching out to you to let you know that we will start the design and development of a Cephadm DevStack plugin.
Some of the reasons on why we want to take this approach:
- devstack-plugin-ceph worked for us for a lot of years, but the development of it relies on several hacks to adapt to the different Ceph versions we use and the different distros we support. This led to a monolithic script that sometimes is hard to debug and break our development environments and our CI - cephadm is the deployment tool developed and maintained by the Ceph community, it allows their users to get specific Ceph versions very easily and enforces good practices for Ceph clusters. From their docs, "Cephadm manages the full lifecycle of a Ceph cluster. It starts by bootstrapping a tiny Ceph cluster on a single node (one monitor and one manager) and
On Wed, 2022-01-19 at 12:04 +0100, Victoria Martínez de la Cruz wrote: then
uses the orchestration interface (“day 2” commands) to expand the cluster to include all hosts and to provision all Ceph daemons and services. [0]" - OpenStack deployment tools are starting to use cephadm as their way to deploy Ceph, so it would be nice to include cephadm in our development process to be closer with what is being done in the field
I started the development of this in [1], but it might be better to change devstack-plugin-ceph to do this instead of having a new plugin. This is something I would love to discuss in a first meeting. i would advocate for pivoting devstack-plugin-ceph. i dont think we have the capsity as a comunity to devleop, maintaine and debug/support 2 differnt ways of deploying ceph in our ci system in the long term.
++ let's pivot devstack-plugin-ceph to me the way devstack-plugin-ceph install cpeh is jsut an implementaion
detail. its contract is that it will install and configure ceph for use with openstack. if you make it use cephadm for that its just and internal detail that should not affect the consomes of the plugin provide you maintain the interface to the devstack pluging mostly the same.
Starting with pacific the deployment of Ceph is moved from ceph-ansible to cephadm: the implication of this change it's not just on the deployment side but this new component (which interacts with the ceph orchestrator module) is able to maintain the lifecycle of the deployed containers, so I'd say the new approach it's not just an implementation detail but also changes the way some components interact with Ceph. Manila using ganesha, for instance, it's the first component that should start using the orchestrator interface, so I guess it's worth aligning (and extending) the most popular dev installer to support the new way (like other projects already did).
i would suggest addign a devstack macro initally to choose the backend but then eventually once the cephadm appoch is stable just swap the default.
+1 on choosing the backend and plan the switch when the cephadm approach is ready and works for all the openstack components
Having said this, I propose using the channel #openstack-cephadm in the OFTC network to talk about this and set up a first meeting with people interested in contributing to this effort.
ack im not sure i will get involed with this but the other option woudl be to just use #openstack-qa since that is the chanlle for devstack development.
Either #openstack-qa or a dedicated one works well , maybe #openstack-qa is useful to reach more people who can help / review the relevant changes .. wdyt
++ let's meet in #openstack-qa
Thanks,
Victoria
[0] https://docs.ceph.com/en/pacific/cephadm/ [1] https://github.com/vkmc/devstack-plugin-cephadm
-- Francesco Pantano GPG KEY: F41BD75C
--
Sofía Enriquez
she/her
Software Engineer
Red Hat PnT <https://www.redhat.com>
IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>
Hi everyone, Just a reminder about this meeting, which will be next Friday at 3pm UTC in #openstack-qa at OFTC. Also, first patch is submitted in [0] Thanks, Victoria [0] https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/826484 On Fri, Jan 21, 2022 at 11:58 AM Victoria Martínez de la Cruz < victoria@vmartinezdelacruz.com> wrote:
Thanks everybody for your responses!
I'll send an initial path with a macro to pick between implementations and we can go from there.
Let's have a meeting next Friday at 3pm UTC in #openstack-qa at OFTC. Would that work for you?
Thanks,
Victoria
On Thu, Jan 20, 2022 at 2:04 PM Sofia Enriquez <senrique@redhat.com> wrote:
Sounds good. I wanna join . I haven't tried cephadm yet but It would help us to make ceph new features more transparent to Cinder in the future. Thanks
++
On Thu, Jan 20, 2022 at 4:45 AM Francesco Pantano <fpantano@redhat.com> wrote:
Hi Victoria, thanks for starting this thread.
On Wed, Jan 19, 2022 at 2:03 PM Sean Mooney <smooney@redhat.com> wrote:
Hi all,
I'm reaching out to you to let you know that we will start the design and development of a Cephadm DevStack plugin.
Some of the reasons on why we want to take this approach:
- devstack-plugin-ceph worked for us for a lot of years, but the development of it relies on several hacks to adapt to the different Ceph versions we use and the different distros we support. This led to a monolithic script that sometimes is hard to debug and break our development environments and our CI - cephadm is the deployment tool developed and maintained by the Ceph community, it allows their users to get specific Ceph versions very easily and enforces good practices for Ceph clusters. From their docs, "Cephadm manages the full lifecycle of a Ceph cluster. It starts by bootstrapping a tiny Ceph cluster on a single node (one monitor and one manager) and
On Wed, 2022-01-19 at 12:04 +0100, Victoria Martínez de la Cruz wrote: then
uses the orchestration interface (“day 2” commands) to expand the cluster to include all hosts and to provision all Ceph daemons and services. [0]" - OpenStack deployment tools are starting to use cephadm as their way to deploy Ceph, so it would be nice to include cephadm in our development process to be closer with what is being done in the field
I started the development of this in [1], but it might be better to change devstack-plugin-ceph to do this instead of having a new plugin. This is something I would love to discuss in a first meeting. i would advocate for pivoting devstack-plugin-ceph. i dont think we have the capsity as a comunity to devleop, maintaine and debug/support 2 differnt ways of deploying ceph in our ci system in the long term.
++ let's pivot devstack-plugin-ceph
to me the way devstack-plugin-ceph install cpeh is jsut an implementaion
detail. its contract is that it will install and configure ceph for use with openstack. if you make it use cephadm for that its just and internal detail that should not affect the consomes of the plugin provide you maintain the interface to the devstack pluging mostly the same.
Starting with pacific the deployment of Ceph is moved from ceph-ansible to cephadm: the implication of this change it's not just on the deployment side but this new component (which interacts with the ceph orchestrator module) is able to maintain the lifecycle of the deployed containers, so I'd say the new approach it's not just an implementation detail but also changes the way some components interact with Ceph. Manila using ganesha, for instance, it's the first component that should start using the orchestrator interface, so I guess it's worth aligning (and extending) the most popular dev installer to support the new way (like other projects already did).
i would suggest addign a devstack macro initally to choose the backend but then eventually once the cephadm appoch is stable just swap the default.
+1 on choosing the backend and plan the switch when the cephadm approach is ready and works for all the openstack components
Having said this, I propose using the channel #openstack-cephadm in
the
OFTC network to talk about this and set up a first meeting with people interested in contributing to this effort. ack im not sure i will get involed with this but the other option woudl be to just use #openstack-qa since that is the chanlle for devstack development.
Either #openstack-qa or a dedicated one works well , maybe #openstack-qa is useful to reach more people who can help / review the relevant changes .. wdyt
++ let's meet in #openstack-qa
Thanks,
Victoria
[0] https://docs.ceph.com/en/pacific/cephadm/ [1] https://github.com/vkmc/devstack-plugin-cephadm
-- Francesco Pantano GPG KEY: F41BD75C
--
Sofía Enriquez
she/her
Software Engineer
Red Hat PnT <https://www.redhat.com>
IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>
On Wed, 19 Jan 2022 at 11:08, Victoria Martínez de la Cruz <victoria@vmartinezdelacruz.com> wrote:
Hi all,
I'm reaching out to you to let you know that we will start the design and development of a Cephadm DevStack plugin.
Some of the reasons on why we want to take this approach:
- devstack-plugin-ceph worked for us for a lot of years, but the development of it relies on several hacks to adapt to the different Ceph versions we use and the different distros we support. This led to a monolithic script that sometimes is hard to debug and break our development environments and our CI - cephadm is the deployment tool developed and maintained by the Ceph community, it allows their users to get specific Ceph versions very easily and enforces good practices for Ceph clusters. From their docs, "Cephadm manages the full lifecycle of a Ceph cluster. It starts by bootstrapping a tiny Ceph cluster on a single node (one monitor and one manager) and then uses the orchestration interface (“day 2” commands) to expand the cluster to include all hosts and to provision all Ceph daemons and services. [0]" - OpenStack deployment tools are starting to use cephadm as their way to deploy Ceph, so it would be nice to include cephadm in our development process to be closer with what is being done in the field
I started the development of this in [1], but it might be better to change devstack-plugin-ceph to do this instead of having a new plugin. This is something I would love to discuss in a first meeting.
Having said this, I propose using the channel #openstack-cephadm in the OFTC network to talk about this and set up a first meeting with people interested in contributing to this effort.
Thanks,
Victoria
[0] https://docs.ceph.com/en/pacific/cephadm/ [1] https://github.com/vkmc/devstack-plugin-cephadm
In case it's useful as a reference, we built an Ansible collection that drives cephadm: https://github.com/stackhpc/ansible-collection-cephadm Feedback welcome, of course! Mark
participants (5)
-
Francesco Pantano
-
Mark Goddard
-
Sean Mooney
-
Sofia Enriquez
-
Victoria Martínez de la Cruz