[openstack-dev] [Fuel] Getting rid of Docker containers on the Fuel master node
Bogdan Dobrelya
bdobrelia at mirantis.com
Fri Nov 20 16:57:27 UTC 2015
On 20.11.2015 17:31, Vladimir Kozhukalov wrote:
> Bogdan,
>
>>> So, we could only deprecate the docker feature for the 8.0.
>
> What do you mean exactly when saying 'deprecate docker feature'? I can
> not even imagine how we can live with and without docker containers at
> the same time. Deprecation is usually related to features which directly
> impact UX (maybe I am wrong).
I may be understood this [0] wrong, and the docker containers are not
user-visible, but that depends on the which type of users do we mean :-)
Sorry, for being not clear.
[0]
http://lists.openstack.org/pipermail/openstack-dev/2015-September/073576.html
>
> Guys,
>
> When you estimate risks of the docker removal, please take into account
> not only release deadlines but also the overall product quality. The
> thing is that continuing using containers makes it much more complicated
> (and thus less stable) to implement new upgrade flow (upgrade tarball
> can not be used any more, we need to re-install the host system).
> Switching from Centos 6 to Centos 7 is also much more complicated with
> docker. Every single piece of Fuel system is going to become simpler and
> easier to support.
>
> Of course, I am not suggesting to jump overboard into cold water without
> a life jacket. Transition plan, checklist, green tests, even spec etc.
> are assumed without saying (after all, I was not born yesterday). Of
> course, we won't merge changes until everything is green. What is the
> problem to try to do this and postpone if not ready in time? And please
> do not confuse these two cases: switching from plain deployment to
> containers is complicated, but switching from docker to plain is much
> simpler.
>
>
>
>
> Vladimir Kozhukalov
>
> On Fri, Nov 20, 2015 at 6:47 PM, Bogdan Dobrelya <bdobrelia at mirantis.com
> <mailto:bdobrelia at mirantis.com>> wrote:
>
> On 20.11.2015 15:10, Timur Nurlygayanov wrote:
> > Hi team,
> >
> > I think it too late to make such significant changes for MOS 8.0 now,
> > but I'm ok with the idea to remove docker containers in the future
> > releases if our dev team want to do this.
> > Any way, before we will do this, we need to plan how we will perform
> > updates between different releases with and without docker containers,
> > how we will manage requirements and etc. In fact we have a lot of
> > questions and haven't answers, let's prepare the spec for this change,
> > review it, discuss it with developers, users and project management team
> > and if we haven't requirements to keep docker containers on master node
> > let's remove them for the future releases (not in MOS 8.0).
> >
> > Of course, we can fix BVT / SWARM tests and don't use docker images in
> > our test suite (it shouldn't be really hard) but we didn't plan these
> > changes and in fact these changes can affect our estimates for many tasks.
>
> I can only add that features just cannot be removed without a
> deprecation period of 1-2 releases.
> So, we could only deprecate the docker feature for the 8.0.
>
> >
> > Thank you!
> >
> >
> > On Fri, Nov 20, 2015 at 4:44 PM, Alexander Kostrikov
> > <akostrikov at mirantis.com <mailto:akostrikov at mirantis.com>
> <mailto:akostrikov at mirantis.com <mailto:akostrikov at mirantis.com>>>
> wrote:
> >
> > Hello, Igor.
> >
> > >But I'd like to hear from QA how do we rely on container-based
> > infrastructure? Would it be hard to change our sys-tests in short
> > time?
> >
> > At first glance, system tests are using docker only to fetch logs
> > and run shell commands.
> > Also, docker is used to run Rally.
> >
> > If there is an action to remove docker containers with carefull
> > attention to bvt testing, it would take couple days to fix system tests.
> > But time may be highly affected by code freezes and active features
> > merging.
> >
> > QA team is going to have Monday (Nov 23) sync-up - and it is
> > possible to get more exact information from all QA-team.
> >
> > P.S.
> > +1 to remove docker.
> > -1 to remove docker without taking into account deadlines/other
> > features.
> >
> > On Thu, Nov 19, 2015 at 10:27 PM, Igor Kalnitsky
> > <ikalnitsky at mirantis.com <mailto:ikalnitsky at mirantis.com>
> <mailto:ikalnitsky at mirantis.com <mailto:ikalnitsky at mirantis.com>>>
> wrote:
> >
> > Hey guys,
> >
> > Despite the fact I like containers (as deployment unit), we
> > don't use
> > them so. That means I +1 idea to drop containers, just because I
> > believe that would
> >
> > * simplify a lot of things
> > * helps get rid of huge amount of hacks
> > * increase master node deployment
> > * release us from annoying support of upgrades / rollbacks that
> > proved
> > to be non-working well
> >
> > But I'd like to hear from QA how do we rely on container-based
> > infrastructure? Would it be hard to change our sys-tests in short
> > time?
> >
> > Thanks,
> > Igor
> >
> >
> > On Thu, Nov 19, 2015 at 10:31 AM, Vladimir Kuklin
> > <vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>
> <mailto:vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>>> wrote:
> > > Folks
> > >
> > > I guess it should be pretty simple to roll back - install
> > older version and
> > > restore the backup with preservation of /var/log directory.
> > >
> > > On Thu, Nov 19, 2015 at 7:38 PM, Sergii Golovatiuk
> > > <sgolovatiuk at mirantis.com
> <mailto:sgolovatiuk at mirantis.com> <mailto:sgolovatiuk at mirantis.com
> <mailto:sgolovatiuk at mirantis.com>>>
> > wrote:
> > >>
> > >> Hi,
> > >>
> > >> On Thu, Nov 19, 2015 at 5:50 PM, Matthew Mosesohn
> > <mmosesohn at mirantis.com <mailto:mmosesohn at mirantis.com>
> <mailto:mmosesohn at mirantis.com <mailto:mmosesohn at mirantis.com>>>
> > >> wrote:
> > >>>
> > >>> Vladimir,
> > >>>
> > >>> The old site.pp is long out of date and should just be
> > recreated from the
> > >>> content of all the other $service-only.pp files.
> > >>>
> > >>> My main question is how do we propose to do a rollback from
> > an update (in
> > >>> theory, from 8.0 to 9.0, then back to 8.0)? Should we
> > hardcode persistent
> > >>> data directories (or symlink them?) to
> > >>> /var/lib/fuel/$fuel_version/$service_name, as we are doing
> > behind the scenes
> > >>> currently with Docker? If we keep that mechanism in place,
> > all the existing
> > >>> puppet modules can be used without any modifications. On the
> > same note,
> > >>> upgrade/rollback is the same as backup and restore, that
> > means our restore
> > >>> should follow a similar approach.
> > >>> -Matthew
> > >>
> > >>
> > >> There only one idea I have is to do dual partitioning system.
> > The similar
> > >> approach is implemented in CoreOS.
> > >>
> > >>>
> > >>>
> > >>> On Thu, Nov 19, 2015 at 6:36 PM, Bogdan Dobrelya
> > <bdobrelia at mirantis.com <mailto:bdobrelia at mirantis.com>
> <mailto:bdobrelia at mirantis.com <mailto:bdobrelia at mirantis.com>>>
> > >>> wrote:
> > >>>>
> > >>>> On 19.11.2015 15:59, Vladimir Kozhukalov wrote:
> > >>>> > Dear colleagues,
> > >>>> >
> > >>>> > As might remember, we introduced Docker containers
> on the
> > master node
> > >>>> > a
> > >>>> > while ago when we implemented first version of Fuel
> > upgrade feature.
> > >>>> > The
> > >>>> > motivation behind was to make it possible to rollback
> > upgrade process
> > >>>> > if
> > >>>> > something goes wrong.
> > >>>> >
> > >>>> > Now we are at the point where we can not use our
> tarball
> > based upgrade
> > >>>> > approach any more and those patches that deprecate
> > upgrade tarball has
> > >>>> > been already merged. Although it is a matter of a
> > separate discussion,
> > >>>> > it seems that upgrade process rather should be based on
> > kind of backup
> > >>>> > and restore procedure. We can backup Fuel data on an
> > external media,
> > >>>> > then we can install new version of Fuel from
> scratch and
> > then it is
> > >>>> > assumed backed up Fuel data can be applied over
> this new Fuel
> > >>>> > instance.
> > >>>>
> > >>>> A side-by-side upgrade, correct? That should work as
> well.
> > >>>>
> > >>>> > The procedure itself is under active development,
> but it
> > is clear that
> > >>>> > rollback in this case would be nothing more than just
> > restoring from
> > >>>> > the
> > >>>> > previously backed up data.
> > >>>> >
> > >>>> > As for Docker containers, still there are potential
> > advantages of
> > >>>> > using
> > >>>> > them on the Fuel master node, but our current
> > implementation of the
> > >>>> > feature seems not mature enough to make us benefit
> from the
> > >>>> > containerization.
> > >>>> >
> > >>>> > At the same time there are some disadvantages like
> > >>>> >
> > >>>> > * it is tricky to get logs and other information (for
> > example, rpm
> > >>>> > -qa) for a service like shotgun which is run inside
> > one of
> > >>>> > containers.
> > >>>> > * it is specific UX when you first need to run
> > dockerctl shell
> > >>>> > {container_name} and then you are able to debug
> > something.
> > >>>> > * when building IBP image we mount directory from the
> > host file
> > >>>> > system
> > >>>> > into mcollective container to make image build
> faster.
> > >>>> > * there are config files and some other files which
> > should be shared
> > >>>> > among containers which introduces unnecessary
> > complexity to the
> > >>>> > whole system.
> > >>>> > * our current delivery approach assumes we wrap into
> > rpm/deb
> > >>>> > packages
> > >>>> > every single piece of the Fuel system. Docker
> images
> > are not an
> > >>>> > exception. And as far as they depend on other rpm
> > packages we
> > >>>> > forced
> > >>>> > to build docker-images rpm package using kind of
> > specific build
> > >>>> > flow. Besides this package is quite big (300M).
> > >>>> > * I'd like it to be possible to install Fuel not from
> > ISO but from
> > >>>> > RPM
> > >>>> > repo on any rpm based distribution. But it is
> double
> > work to
> > >>>> > support
> > >>>> > both Docker based and package based approach.
> > >>>>
> > >>>> There is another point, the containers long build
> time when
> > installing
> > >>>> the master node.
> > >>>>
> > >>>> >
> > >>>> > Probably some of you can give other examples.
> Anyway, the
> > idea is to
> > >>>> > get
> > >>>> > rid of Docker containers on the master node and
> switch to
> > plane
> > >>>> > package
> > >>>> > based approach that we used before.
> > >>>> >
> > >>>> > As far as there is nothing new here, we just need
> to use
> > our old
> > >>>> > site.pp
> > >>>> > (with minimal modifications), it looks like it is
> possible to
> > >>>> > implement
> > >>>> > this during 8.0 release cycle. If there are no
> principal
> > objections,
> > >>>> > please give me a chance to do this ASAP (during 8.0), I
> > know it is a
> > >>>> > huge risk for the release, but still I think I can
> do this.
> > >>>> >
> > >>>> >
> > >>>> >
> > >>>> >
> > >>>> > Vladimir Kozhukalov
> > >>>> >
> > >>>> >
> > >>>> >
> > >>>> >
> >
> __________________________________________________________________________
> > >>>> > OpenStack Development Mailing List (not for usage
> questions)
> > >>>> > Unsubscribe:
> > >>>> >
> >
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > >>>> >
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>>> >
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Best regards,
> > >>>> Bogdan Dobrelya,
> > >>>> Irc #bogdando
> > >>>>
> > >>>>
> > >>>>
> > __________________________________________________________________________
> > >>>> OpenStack Development Mailing List (not for usage questions)
> > >>>> Unsubscribe:
> > >>>>
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > >>>>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > __________________________________________________________________________
> > >>> OpenStack Development Mailing List (not for usage questions)
> > >>> Unsubscribe:
> > >>>
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > >>>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>>
> > >>
> > >>
> > >>
> > __________________________________________________________________________
> > >> OpenStack Development Mailing List (not for usage questions)
> > >> Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>
> > >
> > >
> > >
> > > --
> > > Yours Faithfully,
> > > Vladimir Kuklin,
> > > Fuel Library Tech Lead,
> > > Mirantis, Inc.
> > > +7 (495) 640-49-04 <tel:%2B7%20%28495%29%20640-49-04>
> > > +7 (926) 702-39-68 <tel:%2B7%20%28926%29%20702-39-68>
> > > Skype kuklinvv
> > > 35bk3, Vorontsovskaya Str.
> > > Moscow, Russia,
> > > www.mirantis.com <http://www.mirantis.com>
> <http://www.mirantis.com>
> > > www.mirantis.ru <http://www.mirantis.ru>
> <http://www.mirantis.ru>
> > > vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>
> <mailto:vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>>
> > >
> > >
> > __________________________________________________________________________
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> > --
> >
> > Kind Regards,
> >
> > Alexandr Kostrikov,
> >
> >
> > Mirantis, Inc.
> >
> > 35b/3, Vorontsovskaya St., 109147, Moscow, Russia
> >
> >
> > Tel.: +7 (495) 640-49-04 <tel:%2B7%20%28495%29%20640-49-04>
> > Tel.: +7 (925) 716-64-52 <tel:%2B7%20%28906%29%20740-64-79>
> >
> > Skype: akostrikov_mirantis
> >
> > E-mail:akostrikov at mirantis.com
> <mailto:E-mail%3Aakostrikov at mirantis.com>
> <mailto:elogutova at mirantis.com <mailto:elogutova at mirantis.com>>
> >
> > _www.mirantis.com <http://www.mirantis.com>
> <http://www.mirantis.ru/>__
> > __www.mirantis.ru <http://www.mirantis.ru>
> <http://www.mirantis.ru/>_
> >
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> > --
> >
> > Timur,
> > Senior QA Engineer
> > OpenStack Projects
> > Mirantis Inc
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> --
> Best regards,
> Bogdan Dobrelya,
> Irc #bogdando
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
--
Best regards,
Bogdan Dobrelya,
Irc #bogdando
More information about the OpenStack-dev
mailing list