Open Stack

On Tue, Nov 10, 2015 at 11:10 AM, Vladimir Kuklin <vkuklin at mirantis.com> wrote:
> Alex
>
> Thanks for your very detailed answer - it clarified things a bit. So, if you
> would allow me to rephrase it - you are actually conducting a research on
> what is the actual gap between our downstream/fork and upstream
> UCA/puppet-openstack. This seems to be a very promising initiative. Are you
> going to come up with some external-user readable report soon?
>
> Also, regarding multiple distros support.  I think, we need to come up with
> an approach of making 'release manager' piece of Nailgun data driven and
> just allow a user to run any distribution he or she wants. Just register a
> set of repos with packages and run it. Actually, we already have it - we
> need to make base-image generation for RPM more flexible and any RPM/DEB
> based distro should work ok with it.
>
> The remaining piece is to actually support distro-specific things, e.g.
> CentOS/RHEL networking configuration, e.g. l23 network stored config puppet
> providers. But this is a distro-supporter/community burden.
>
>

Yes I hope to have something together by the end of the week.  I've
managed to get a controller and compute/cinder nodes up (and passing
basic OSTF tests) with what appears to be some minor adjustments to
the fuel-library code.  The one thing that gets dropped because of
lack of upstream support is nova floating network ranges. There's a
pending review that'll get that back in but I also don't know how
important it would be to support for this type of configuration.
Another issue is the upstream murano module is still a work in
progress so that won't work right now either. Hopefully that'll get
sorted out in time for the official release of the liberty puppet
modules.

As I've been working through this, I've noticed that it would be
possible to use fuel-plugins to only apply UCA packages to specific
nodes via a plugin role. An interesting follow on to this effort would
be to use MOS packages for controllers and UCA for Compute or vice
versa.  But that should probably be more an academic exercise rather
than production one.

-Alex

>
> On Tue, Nov 10, 2015 at 6:25 PM, Alex Schultz <aschultz at mirantis.com> wrote:
>>
>> Hey Vladimir,
>>
>> On Tue, Nov 10, 2015 at 5:56 AM, Vladimir Kuklin <vkuklin at mirantis.com>
>> wrote:
>> > Alex
>> >
>> > That's great to hear that. But be aware - making all of the components
>> > work
>> > exactly the way they work within MOS is actually identical to
>> > upstreaming
>> > MOS. We are using some components of different versions to satisfy many
>> > requirements for our Reference Architecutre implementation. It will not
>> > be
>> > so easy to base them upon existing 3rd party distributions. For example,
>> > `read timeout` for SQL requests is crucial for our HA as it handles
>> > cases
>> > when an SQL node dies while serving the request. And you will find
>> > myriads
>> > of them. And as we do not control things in upstream, we will always
>> > have
>> > some downstream divergence.
>> >
>>
>> Yes, I'm aware that it'll be an effort to make it work identically to
>> MOS.  Currently that's not my goal. My goal is to get it working at
>> all and be able to document the deficiencies when using upstream
>> packages/modules vs MOS provided ones.  Once we have documented these
>> differences we will be able to make decisions as to what efforts
>> should be made if we choose to address the differences.  The read
>> timeout thing seems to be an issue with what mysql python driver is
>> used so that could easily be configurable based on a packages or a
>> configuration option.
>>
>> > I guess, the optimal solution is to figure out the actual divergence
>> > between
>> > upstream and downstream and try to push things upstream as hard as we
>> > can,
>> > while retaining overrides for some packages and manifests on top of
>> > upstream
>> > versions. Do not get me wrong, but it seems there is exactly 0 (zero)
>> > ways
>> > you can get Fuel working with upstream packages unless they support
>> > exactly
>> > the same feature set and fix the same bugs in various components that
>> > Fuel
>> > expect them to support or fix. By 'working' I mean passing the same set
>> > of
>> > at least smoke and destructive tests, let alone passing scale tests.
>> >
>>
>> So I think this is where we are currently backwards in the way we're
>> doings. As we hope to push Fuel as a community project, we need to be
>> more open to supporting the distributions versions of these packages
>> as being supported.  If we want to continue with specific versions of
>> things we need to be able to modularize them and apply them to a
>> downstream version of Fuel that we can promote with the MOS package
>> set.  I agree that right now it's highly unlikely that one would be
>> able to use Fuel without any MOS packages. Because I don't think it's
>> possible right now, what I'm attempting to do is be able to deploy an
>> alternate OpenStack package set but not all of the other pieces as
>> they relate to our reference architecture.  That seems to be a more
>> achievable goal right now and allows us to document the gaps where
>> Fuel/MOS are a head (or behind depending on your views) with upstream.
>>
>> > Simon
>> >
>> > AFAIK, this patch is really simple and can be easily applied to any
>> > version
>> > of HAProxy. So it is not too hard handle it while it provides sufficient
>> > benefits to our deployment engineers. If you have any better solution,
>> > please feel free to share the code with us.
>> >
>>
>> Unfortunately the haproxy fork that we're currently running with also
>> includes our forked version of the puppet haproxy module. I'm not sure
>> the includes configs patch is really worth carrying the additional
>> work/forks but I think that is a conversation for another day.
>> Personally I think it would better if fuel-library supported the
>> currently available upstream versions of haproxy & haproxy puppet
>> module and if we want to continue using our copies of haproxy & puppet
>> haproxy module that we turn that into a downstream that is applied for
>> the MOS specific version of Fuel.  That's the advantage of the
>> Puppetfile we've been working on. We can change out the modules with
>> our own version downstream.
>>
>> Just to add some additional updates issues that I've run into.
>>
>> - nova-vncproxy package names (nova-consoleproxy vs nova-novncproxy)
>> but that's easily solved with the proposed os_package_type fact.
>> - heat-docker package does not exist in UCA. Haven't looked at what
>> exactly is in this package yet.
>> - horizon UCA package runs into
>> https://bugs.launchpad.net/cloud-archive/+bug/1479340 (we solved this
>> for MOS but it's still broken in UCA)
>> - neutron UCA package seems to suffer from
>> https://bugs.launchpad.net/mos/+bug/1510844
>> - Need to solve for the url issues with the openstack command that are
>> showing up in the CI as I just hit it with my deployment.
>>
>> Thanks,
>> -Alex
>>
>> > On Tue, Nov 10, 2015 at 12:54 PM, Stanislaw Bogatkin
>> > <sbogatkin at mirantis.com> wrote:
>> >>
>> >> Hi Simon,
>> >>
>> >> using 'include' in HAProxy is damn conveniently, I don't think it
>> >> should
>> >> die. There is just one way I know to use haproxy with several different
>> >> conf's - to construct looooooooooong command line with all of them -
>> >> and
>> >> it's really inconvenient.
>> >>
>> >> On Tue, Nov 10, 2015 at 12:20 PM, Simon Pasquier
>> >> <spasquier at mirantis.com>
>> >> wrote:
>> >>>
>> >>> Hello Alex!
>> >>>
>> >>> On Mon, Nov 9, 2015 at 9:29 PM, Alex Schultz <aschultz at mirantis.com>
>> >>> wrote:
>> >>>>
>> >>>> Hey folks,
>> >>>>
>> >>>> I'm testing[0] out flipping our current method of consuming upstream
>> >>>> puppet modules from using pinned versions hosted on fuel-infra to be
>> >>>> able to use the ones directly from upstream (master).  This work is
>> >>>> primarily to be closer aligned with the other OpenStack projects as
>> >>>> well as switching the current way we manage Fuel into a downstream of
>> >>>> the upstream community version. As part of this work we have also
>> >>>> been
>> >>>> working towards improving the upstream modules support different
>> >>>> package sets.  Specifically running Debian packages on Ubuntu[1][2].
>> >>>> This work is the start of being able to allow a user of Fuel to be
>> >>>> able to specify a specific package set and having it be able to work.
>> >>>> If we can properly split out the puppet modules and package
>> >>>> dependencies this will make Fuel a more flexible deployment engine as
>> >>>> I believe we would be better positioned to support multiple versions
>> >>>> of OpenStack for a given Fuel release.
>> >>>>
>> >>>> I'm currently working to get a PoC of Fuel consuming upstream puppet
>> >>>> modules and the UCA packages together and documenting all of the
>> >>>> issues so that we can address them.  So far I have been able to
>> >>>> deploy
>> >>>> the upstream modules via a custom ISO using the MOS package set and
>> >>>> it
>> >>>> works locally. Unfortunately the CI seems to be hitting some issues
>> >>>> that I think might be related to recently merged keystone changes but
>> >>>> I did not run into the same problem when running a manual deployment.
>> >>>> As I work through this PoC, I'm also attempting to develop a small
>> >>>> plugin that could be used to capture the work arounds to the
>> >>>> deployment process. I've run into a few issues so far as I work to
>> >>>> switch out the package sets.
>> >>>>
>> >>>> For the sake of providing additional visibility into this work, here
>> >>>> are the issues that I've hit so far.
>> >>>>
>> >>>> The first issue I ran across is that currently the MOS repositories
>> >>>> contain packages for both OpenStack and other system dependencies for
>> >>>> creating our HA implementation.  This is problematic when we want to
>> >>>> switch out the OpenStack packages but still want the MOS packages for
>> >>>> our HA items.  I'm working around this by adding the UCA repository
>> >>>> as
>> >>>> a higher priorities for deployments.  As such I've run into an issue
>> >>>> with the haproxy package that MOS provides vs the upstream Ubuntu
>> >>>> package.  To get around this, I've pinned the MOS version for now
>> >>>> until I can circle back around and figure out if the difference is a
>> >>>> config or functionality issue.
>> >>>
>> >>>
>> >>> I know at least for one difference between the MOS and Ubuntu
>> >>> versions:
>> >>> HAProxy from MOS has a patch to support the "include" configuration
>> >>> parameter.
>> >>> This is unrelated to your mail but I think this fork should die since
>> >>> it
>> >>> will never be accepted upstream and there are other ways to address
>> >>> the use
>> >>> case [0].
>> >>>
>> >>> HTH,
>> >>> Simon
>> >>>
>> >>> [0] http://marc.info/?l=haproxy&m=130817444025140&w=2
>> >>>
>> >>>>
>> >>>>
>> >>>> The second issue that I have run across is that we are appending
>> >>>> read_timeout=60 to our mysql connection strings for our
>> >>>> configurations. This seems to be unsupported by the libraries and
>> >>>> OpenStack components provided by the UCA package set.  I'm not sure
>> >>>> how the priority of python-pymsql vs python-mysqldb is resolved as I
>> >>>> had both packages installed but it continued to fail on read_timeout
>> >>>> being in the connection string.  For now I've updated the
>> >>>> fuel-library
>> >>>> code to remove this item from the connection strings and will be
>> >>>> circling back around to figure out the correct 'fix' for this issue.
>> >>>>
>> >>>> I'm hoping to be able to have a working ISO, plugin and a set of
>> >>>> instructions that can be used to deploy a basic cloud using Fuel by
>> >>>> the end of this week.
>> >>>>
>> >>>> Thanks,
>> >>>> -Alex
>> >>>>
>> >>>> [0] https://review.openstack.org/#/c/240325/
>> >>>> [1] https://review.openstack.org/#/c/241615/
>> >>>> [2] https://review.openstack.org/#/c/241741/
>> >>>>
>> >>>>
>> >>>>
>> >>>> __________________________________________________________________________
>> >>>> OpenStack Development Mailing List (not for usage questions)
>> >>>> Unsubscribe:
>> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> __________________________________________________________________________
>> >>> OpenStack Development Mailing List (not for usage questions)
>> >>> Unsubscribe:
>> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>>
>> >>
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >
>> >
>> >
>> > --
>> > Yours Faithfully,
>> > Vladimir Kuklin,
>> > Fuel Library Tech Lead,
>> > Mirantis, Inc.
>> > +7 (495) 640-49-04
>> > +7 (926) 702-39-68
>> > Skype kuklinvv
>> > 35bk3, Vorontsovskaya Str.
>> > Moscow, Russia,
>> > www.mirantis.com
>> > www.mirantis.ru
>> > vkuklin at mirantis.com
>> >
>> >
>> > __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com
> www.mirantis.ru
> vkuklin at mirantis.com
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>