Open Stack

It is not just about BVT. I'd suggest to monitor situation overall,
including failures of system tests [1]. If we see regressions there, or
some test cases will start flapping (what is even worse), then we'd have to
revert back to CentOS 7.1.

[1] https://github.com/openstack/fuel-qa

On Wed, Mar 2, 2016 at 10:16 AM Dmitry Borodaenko <dborodaenko at mirantis.com>
wrote:

> I agree with Mike's concerns, and propose to make these limitations (4
> weeks before FF for OS upgrades, 2 weeks for upgrades of key
> dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL,
> anything else?) official for 10.0/Newton.
>
> For 9.0/Mitaka, it is too late to impose them, so we just have to be
> very careful and conservative with this upgrade. First of all, we need
> to have a green BVT before and after switching to the CentOS 7.2 repo
> snapshot, so while I approved the spec, we can't move forward with this
> until BVT is green again, and right now it's red:
>
> https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/
>
> If we get it back to green but it becomes red after the upgrade, you
> must switch back to CentOS 7.1 *immediately*. If you are able to stick
> to this plan, there is still time to complete the transition today
> without requiring an FFE.
>
> --
> Dmitry Borodaenko
>
>
> On Wed, Mar 02, 2016 at 05:53:53PM +0000, Mike Scherbakov wrote:
> > Formally, we can merge it today. Historically, every update of OS caused
> us
> > instability for some time: from days to a couple of month.
> > Taking this into account and number of other exceptions requested,
> overall
> > stability of code, my opinion would be to postpone this to 10.0.
> >
> > Also, I'd suggest to change the process, and have freeze date for all OS
> > updates no later than a month before official FF date. This will give us
> > time to stabilize, and ensure that base on which all new code is being
> > developed is stable when approaching FF.
> >
> > I'd also propose to have freeze for major upgrades of 3rd party packages
> no
> > later than 2 weeks before FF, which Fuel depends heavily upon. For
> > instance, such will include RabbitMQ, MCollective, Puppet.
> >
> > On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat <imarnat at mirantis.com> wrote:
> >
> > > Igor,
> > > couple of points from my side.
> > >
> > > CentOS 7.2 will be getting updates for several more months, and we have
> > > snapshots and all the mechanics in place to switch to the next version
> when
> > > needed.
> > >
> > > Speaking of getting this update into 9.0, we actually don't need FFE,
> we
> > > can merge remaining staff today. It has enough reviews, so if you add
> your
> > > +1 today, we don't need FFE.
> > >
> > > https://review.openstack.org/#/c/280338/
> > > https://review.fuel-infra.org/#/c/17400/
> > >
> > >
> > >
> > > Regards,
> > > Igor Marnat
> > >
> > > On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin <
> dteselkin at mirantis.com>
> > > wrote:
> > >
> > >> Igor,
> > >>
> > >> Your statement about updates for 7.2 isn't correct - it will receive
> > >> updates,  because it's the latest release ATM. There is *no* pinning
> inside
> > >> ISO, and the only place where it was 8.0 were docker containers just
> > >> because we had to workaround some issues. But there are no docker
> > >> containers in 9.0, so that's not the case.
> > >> The proposed solution to switch to CentOS-7.2 in fact is based on
> > >> selecting the right snapshot with packages. There is no pinning in
> ISO (it
> > >> was in earlier versions of the spec but was removed).
> > >>
> > >> On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky <
> ikalnitsky at mirantis.com>
> > >> wrote:
> > >>
> > >>> Dmitry, Igor,
> > >>>
> > >>> > Very important thing is that CentOS 7.1 which master node is based
> now
> > >>> > don't get updates any longer.
> > >>>
> > >>> If you are using "fixed" release you must be ready that you won't get
> > >>> any updates. So with CentOS 7.2 the problem still the same.
> > >>>
> > >>> However, let's wait for Fuel PTL decision. I only shared my POV:
> > >>> that's not a critical feature, and taking into account the risks of
> > >>> regression - I'd prefer to do not accept it in 9.0.
> > >>>
> > >>> Regards,
> > >>> Igor
> > >>>
> > >>> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat <imarnat at mirantis.com>
> > >>> wrote:
> > >>> > Igor,
> > >>> > please note that this is pretty much not like update of master node
> > >>> which we
> > >>> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which
> team
> > >>> > tested for more than 2 months already.
> > >>> >
> > >>> > We don't expect it to require any additional efforts from core or
> qa
> > >>> team.
> > >>> >
> > >>> > Very important thing is that CentOS 7.1 which master node is based
> now
> > >>> don't
> > >>> > get updates any longer. Updates are only provided for CentOS 7.2.
> > >>> >
> > >>> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways.
> > >>> >
> > >>> > We can do it now for more or less free, later in release cycle for
> > >>> higher
> > >>> > risk and QA efforts and after the release for 2x price because of
> > >>> additional
> > >>> > QA cycle we'll need to pass through.
> > >>> >
> > >>> >
> > >>> >
> > >>> > Regards,
> > >>> > Igor Marnat
> > >>> >
> > >>> > On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin <
> > >>> dteselkin at mirantis.com>
> > >>> > wrote:
> > >>> >>
> > >>> >> Hi Igor,
> > >>> >>
> > >>> >> Postponing this till Fuel 10 means we have to elaborate a plan to
> do
> > >>> such
> > >>> >> upgrade for Fuel 9 after the release - the underlying system will
> not
> > >>> get
> > >>> >> updated on it's own, and the security issues will not close
> > >>> themselves. The
> > >>> >> problem here is that such upgrade of deployed master node
> requires a
> > >>> lot of
> > >>> >> QA work also.
> > >>> >>
> > >>> >> Since we are not going to update package we build on our own (they
> > >>> still
> > >>> >> targeted 7.1) switching master node base to CentOS-72 is not that
> > >>> dangerous
> > >>> >> as it seems.
> > >>> >>
> > >>> >> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky <
> > >>> ikalnitsky at mirantis.com>
> > >>> >> wrote:
> > >>> >>>
> > >>> >>> Hey Dmitry,
> > >>> >>>
> > >>> >>> No offence, but I rather against that exception. We have too many
> > >>> >>> things to do in Mitaka, and moving to CentOS 7.2 means
> > >>> >>>
> > >>> >>> * extra effort from core team
> > >>> >>> * extra effort from qa team
> > >>> >>>
> > >>> >>> Moreover, it might block development by introducing unpredictable
> > >>> >>> regressions. Remember 8.0? So I think it'd be better to reduce
> risk
> > >>> of
> > >>> >>> regressions that affects so many developers by postponing CentOS
> 7.2
> > >>> >>> till Fuel 10.
> > >>> >>>
> > >>> >>> Thanks,
> > >>> >>> Igor
> > >>> >>>
> > >>> >>>
> > >>> >>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin <
> > >>> dteselkin at mirantis.com>
> > >>> >>> wrote:
> > >>> >>> > I'd like to ask for a feature freeze exception for switching to
> > >>> >>> > CentOS-7.2
> > >>> >>> > feature [0].
> > >>> >>> >
> > >>> >>> > CentOS-7.2 ISO's have been tested periodically since the
> beginning
> > >>> of
> > >>> >>> > the
> > >>> >>> > year, and all major issues were addressed / fixed at the
> moment.
> > >>> During
> > >>> >>> > the
> > >>> >>> > last weekend I've made 70 BVT runs to verify that the  solution
> > >>> [2] for
> > >>> >>> > the
> > >>> >>> > last issue - e1000 transmit unit hangs works. And it works, 0
> > >>> tests of
> > >>> >>> > 35
> > >>> >>> > failed [3].
> > >>> >>> >
> > >>> >>> > Benefits of switching to CentOS-7.2 are quite obvious - we will
> > >>> return
> > >>> >>> > to
> > >>> >>> > latest supported CentOS release, will fix a lot of bugs /
> security
> > >>> >>> > issues
> > >>> >>> > [4] and will make further updates easier.
> > >>> >>> >
> > >>> >>> > Risk of regression still exists, but it's quite low, 35
> successful
> > >>> BVTs
> > >>> >>> > can't be wrong.
> > >>> >>> >
> > >>> >>> > To finish that feature the following should be done:
> > >>> >>> > * review and merge e1000 workaround [2]
> > >>> >>> > * review and merge spec [0]
> > >>> >>> > * review and merge request that switches build CI to
> CentOS-7.2 [5]
> > >>> >>> >
> > >>> >>> > I expect the last day it could be done is March, 4.
> > >>> >>> >
> > >>> >>> > [0] https://review.openstack.org/#/c/280338/
> > >>> >>> > [1] https://bugs.launchpad.net/fuel/+bug/1526544
> > >>> >>> > [2] https://review.openstack.org/#/c/285306/
> > >>> >>> > [3]
> > >>> https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350
> > >>> >>> > [4]
> > >>> https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630
> > >>> >>> > [5] https://review.fuel-infra.org/#/c/17400/
> > >>> >>> >
> > >>> >>> >
> > >>> >>> > --
> > >>> >>> > Thanks,
> > >>> >>> > Dmitry Teselkin
> > >>> >>> > Mirantis
> > >>> >>> > http://www.mirantis.com
> > >>> >>> >
> > >>> >>> >
> > >>> >>> >
> > >>>
> __________________________________________________________________________
> > >>> >>> > OpenStack Development Mailing List (not for usage questions)
> > >>> >>> > Unsubscribe:
> > >>> >>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > >>> >>> >
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>> >>> >
> > >>> >>>
> > >>> >>>
> > >>> >>>
> > >>>
> __________________________________________________________________________
> > >>> >>> OpenStack Development Mailing List (not for usage questions)
> > >>> >>> Unsubscribe:
> > >>> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > >>> >>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>> >>
> > >>> >>
> > >>> >>
> > >>> >>
> > >>> >> --
> > >>> >> Thanks,
> > >>> >> Dmitry Teselkin
> > >>> >> Mirantis
> > >>> >> http://www.mirantis.com
> > >>> >>
> > >>> >>
> > >>>
> __________________________________________________________________________
> > >>> >> OpenStack Development Mailing List (not for usage questions)
> > >>> >> Unsubscribe:
> > >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>> >>
> > >>> >
> > >>> >
> > >>> >
> > >>>
> __________________________________________________________________________
> > >>> > OpenStack Development Mailing List (not for usage questions)
> > >>> > Unsubscribe:
> > >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>> >
> > >>>
> > >>>
> > >>>
> __________________________________________________________________________
> > >>> OpenStack Development Mailing List (not for usage questions)
> > >>> Unsubscribe:
> > >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Thanks,
> > >> Dmitry Teselkin
> > >> Mirantis
> > >> http://www.mirantis.com
> > >>
> > >>
> __________________________________________________________________________
> > >> OpenStack Development Mailing List (not for usage questions)
> > >> Unsubscribe:
> > >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>
> > >>
> > >
> __________________________________________________________________________
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> > --
> > Mike Scherbakov
> > #mihgen
>
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 
Mike Scherbakov
#mihgen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160302/da1db3ad/attachment.html>