[openstack-dev] [Fuel] [Shotgun] Decoupling Shotgun from Fuel

Adam Heczko aheczko at mirantis.com
Wed Jun 8 04:36:32 UTC 2016


Hi,
I'd like to ask what's the current state of Shotgun and what are the plans
for the future?
Is there any alternative chosen for Fuel diagnostic snapshot functionality
and being worked on?

On Mon, Apr 18, 2016 at 3:39 PM, Igor Kalnitsky <ikalnitsky at mirantis.com>
wrote:

> Evgeniy L. wrote:
> > I think such kind of tools should use as less as possible existing
> > infrastructure, because in case if something went wrong, you should
> > be able to easily get diagnostic information, even with broken RabbitMQ,
> > Astute and MCollective.
>
> It's a good point indeed! Moreover, troubleshooting scenarios may vary
> from case to case, so it should be easily extendable and changeable.
> So users can use various (probably, downloaded) scenarios to gather
> diagnostic info.
>
> That's why I think Ansible could really be helpful here. Such
> scenarios may be distributed as Ansible playbooks.
>
> On Mon, Apr 18, 2016 at 4:25 PM, Evgeniy L <eli at mirantis.com> wrote:
> >>> Btw, one of the ideas was to use Fuel task capabilities to gather
> >>> diagnostic snapshot.
> >
> > I think such kind of tools should use as less as possible existing
> > infrastructure, because in case if something went wrong, you should be
> able
> > to easily get diagnostic information, even with broken RabbitMQ, Astute
> and
> > MCollective.
> >
> > Thanks,
> >
> >
> > On Mon, Apr 18, 2016 at 2:26 PM, Vladimir Kozhukalov
> > <vkozhukalov at mirantis.com> wrote:
> >>
> >> Colleagues,
> >>
> >> Whether we are going to continue using Shotgun or
> >> substitute it with something else, we still need to
> >> decouple it from Fuel because Shotgun is a generic
> >> tool. Please review these [1], [2].
> >>
> >> [1] https://review.openstack.org/#/c/298603
> >> [2] https://review.openstack.org/#/c/298615
> >>
> >>
> >> Btw, one of the ideas was to use Fuel task capabilities
> >> to gather diagnostic snapshot.
> >>
> >> Vladimir Kozhukalov
> >>
> >> On Thu, Mar 31, 2016 at 1:32 PM, Evgeniy L <eli at mirantis.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Problems which I see with current Shotgun are:
> >>> 1. Luck of parallelism, so it's not going to fetch data fast enough
> from
> >>> medium/big clouds.
> >>> 2. There should be an easy way to run it manually (it's possible, but
> >>> there is no ready-to-use config), it would be really helpful in case if
> >>> Nailgun/Astute/MCollective are down.
> >>>
> >>> As far as I know 1st is partly covered by Ansible, but the problem is
> it
> >>> executes a single task in parallel, so there is probability that
> lagging
> >>> node will slow down fetching from entire environment.
> >>> Also we will have to build a tool around Ansible to generate playbooks.
> >>>
> >>> Thanks,
> >>>
> >>> On Wed, Mar 30, 2016 at 5:18 PM, Tomasz 'Zen' Napierala
> >>> <tnapierala at mirantis.com> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> Do we have any requirements for the new tool? Do we know what we don’t
> >>>> like about current implementation, what should be avoided, etc.?
> Before that
> >>>> we can only speculate.
> >>>> From my ops experience, shotgun like tools will not work conveniently
> on
> >>>> medium to big environments. Even on medium env amount of logs is just
> too
> >>>> huge to handle by such simple tool. In such environments better
> pattern is
> >>>> to use dedicated log collection / analysis tool, just like StackLight.
> >>>> At the other hand I’m not sure if ansible is the right tool for that.
> It
> >>>> has some features (like ‘fetch’ command) but in general it’s a
> configuration
> >>>> management tool, and I’m not sure how it would act under such heavy
> load.
> >>>>
> >>>> Regards,
> >>>>
> >>>> > On 30 Mar 2016, at 15:20, Vladimir Kozhukalov
> >>>> > <vkozhukalov at mirantis.com> wrote:
> >>>> >
> >>>> > Igor,
> >>>> >
> >>>> > I can not agree more. Wherever possible we should
> >>>> > use existent mature solutions. Ansible is really
> >>>> > convenient and well known solution, let's try to
> >>>> > use it.
> >>>> >
> >>>> > Yet another thing should be taken into account.
> >>>> > One of Shotgun features is diagnostic report
> >>>> > that could then be attached to bugs to identify
> >>>> > the content of env. This report could also be
> >>>> > used to reproduce env and then fight a bug.
> >>>> > I'd like we to have this kind of report.
> >>>> > Is it possible to implement such a feature
> >>>> > using Ansible? If yes, then let's switch to Ansible
> >>>> > as soon as possible.
> >>>> >
> >>>> >
> >>>> >
> >>>> > Vladimir Kozhukalov
> >>>> >
> >>>> > On Wed, Mar 30, 2016 at 3:31 PM, Igor Kalnitsky
> >>>> > <ikalnitsky at mirantis.com> wrote:
> >>>> > Neil Jerram wrote:
> >>>> > > But isn't Ansible also over-complicated for just running commands
> >>>> > > over SSH?
> >>>> >
> >>>> > It may be not so "simple" to ignore that. Ansible has a lot of
> modules
> >>>> > which might be very helpful. For instance, Shotgun makes a database
> >>>> > dump and there're Ansible modules with the same functionality [1].
> >>>> >
> >>>> > Don't think I advocate Ansible as a replacement. My point is, let's
> >>>> > think about reusing ready solutions. :)
> >>>> >
> >>>> > - igor
> >>>> >
> >>>> >
> >>>> > [1]: http://docs.ansible.com/ansible/list_of_database_modules.html
> >>>> >
> >>>> > On Wed, Mar 30, 2016 at 1:14 PM, Neil Jerram
> >>>> > <Neil.Jerram at metaswitch.com> wrote:
> >>>> > >
> >>>> > > FWIW, as a naive bystander:
> >>>> > >
> >>>> > > On 30/03/16 11:06, Igor Kalnitsky wrote:
> >>>> > >> Hey Fuelers,
> >>>> > >>
> >>>> > >> I know that you probably wouldn't like to hear that, but in my
> >>>> > >> opinion
> >>>> > >> Fuel has to stop using Shotgun. It's nothing more but a command
> >>>> > >> runner
> >>>> > >> over SSH. Besides, it has well known issues such as retrieving
> >>>> > >> remote
> >>>> > >> directories with broken symlinks inside.
> >>>> > >
> >>>> > > It makes sense to me that a command runner over SSH might not need
> >>>> > > to be
> >>>> > > a whole Fuel-specific component.
> >>>> > >
> >>>> > >> So I propose to find a modern alternative and reuse it. If we
> stop
> >>>> > >> supporting Shotgun, we can spend extra time to focus on more
> >>>> > >> important
> >>>> > >> things.
> >>>> > >>
> >>>> > >> As an example, we can consider to use Ansible. It should not be
> >>>> > >> tricky
> >>>> > >> to generate Ansible playbook instead of generating Shotgun one.
> >>>> > >> Ansible is a  well known tool for devops and cloud operators, and
> >>>> > >> they
> >>>> > >> we will only benefit if we provide possibility to extend
> diagnostic
> >>>> > >> recipes in usual (for them) way. What do you think?
> >>>> > >
> >>>> > > But isn't Ansible also over-complicated for just running commands
> >>>> > > over SSH?
> >>>> > >
> >>>> > >         Neil
> >>>> > >
> >>>> > >
> >>>> > >
> >>>> > >
> __________________________________________________________________________
> >>>> > > OpenStack Development Mailing List (not for usage questions)
> >>>> > > Unsubscribe:
> >>>> > > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>> >
> >>>> >
> >>>> >
> __________________________________________________________________________
> >>>> > OpenStack Development Mailing List (not for usage questions)
> >>>> > Unsubscribe:
> >>>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>> >
> >>>> >
> >>>> >
> __________________________________________________________________________
> >>>> > OpenStack Development Mailing List (not for usage questions)
> >>>> > Unsubscribe:
> >>>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>> --
> >>>> Tomasz 'Zen' Napierala
> >>>> Product Engineering - Poland
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>>
> >>>
> >>>
> >>>
> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Adam Heczko
Security Engineer @ Mirantis Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160608/b047c9e7/attachment.html>


More information about the OpenStack-dev mailing list