Open Stack

Sorry for 1 day delay in response - busy prepping for Kolla midcycle next
week.  Responses inline.

On 2/5/16, 12:14 PM, "Steve Gordon" <sgordon at redhat.com> wrote:

>----- Original Message -----
>> From: "Steven Dake (stdake)" <stdake at cisco.com>
>> To: "OpenStack Development Mailing List (not for usage questions)"
>><openstack-dev at lists.openstack.org>
>> 
>> Steve,
>> 
>> Comments inline
>> 
>> On 2/3/16, 3:08 PM, "Steve Gordon" <sgordon at redhat.com> wrote:
>> 
>> >----- Original Message -----
>> >> From: "Hongbin Lu" <hongbin.lu at huawei.com>
>> >> To: "OpenStack Development Mailing List (not for usage questions)"
>> >><openstack-dev at lists.openstack.org>
>> >> 
>> >> I would vote for a quick fix + a blueprint.
>> >> 
>> >> BTW, I think it is a general consensus that we should move away from
>> >>Atomic
>> >> for various reasons (painful image building, lack of document, hard
>>to
>> >>use,
>> >> etc.). We are working on fixing the CoreOS templates which could
>>replace
>> >> Atomic in the future.
>> >> 
>> >> Best regards,
>> >> Hongbin
>> >
>> >Hi Hongbin,
>> >
>> >I had heard this previously in Tokyo and again when I was asking around
>> >about the image support on IRC last week, is there a list of the exact
>> >issues with image building etc. with regards to Atomic? When I was
>> >following up on this it seemed like the main issue is that the docs in
>> >the magnum repo are quite out of date (versus the upstream fedora
>>atomic
>> >docs) both with regards to the content of the image and the process
>>used
>> >to (re)build it - there didn't seem to be anything quantifiable that's
>> >wrong with the current Atomic images but perhaps I was asking the wrong
>> >folks. I was able to rebuild fairly trivially using the Fedora built
>> >artefacts [1][2].
>> 
>> Steve,
>> 
>> I hope you can forgive my directness and lack of diplomacy in this
>> message. :)
>> 
>> At least when I was heavily involved with Magnum, building atomic images
>> resulted in a situation in which the binaries built did not work
>>properly.
>>  I begged on the irc channels for help and begged on the mailing list
>>for
>> help for _ months _ on end and nobody listened.  It is almost as if
>>nobody
>> is actually working on Atomic.  If there are people, they do not
>>maintain
>> any kind of support footprint upstream to make Atomic a viable platform
>> for Magnum.
>
>Hi Steve,
>
>No worries about directness, it's actually what I am trying to elicit so
>I can understand and hopefully help address the issues. I did come across
>a couple of mailing list posts from you mainly asking about newer
>k8s/etcd/flannel versions and also etcdctl inclusion (there was a
>workaround posted at the time but I note it's now in the image) [1][2].
>These threads are actually what prompted me to ask "what else?" to try
>and determine if Magnum had other needs here, e.g. the rpm-ostree issue
>you refer to below is the type of hint I'm after to try and ensure
>everything is tracked somewhere.
>
>> I taught Tango how to build the images, who wrote the instructions down
>>in
>> the Magnum documentation.  That documentation ends up producing images
>> that randomly don't always work. The binaries return some weird system
>> call error, ebadlink I think but not sure.  Tango may remember.
>
>I was playing around with this earlier in the week and had some more
>success via a slightly different path, I'm in the process of attempting
>to re-produce on a clean system to ensure that (a) I didn't skip anything
>in the notes I took and (b) weed out any other setup that was peculiar to
>my system before I propose something against the docs.
>
>> Perhaps the rpm-ostree defect has been resolved now.  I have to be clear
>> that I was told "please wait 6 months for us to fix the build system and
>> bugs" while Atomic was our only distro implemented.  It was very
>> maddening.  I was so frustrated with Atomic, at the start of Mitaka I
>>was
>> going to propose deprecating Atomic because of a complete lack of
>>upstream
>> responsiveness.  I decided to let other folks make the call about what
>> they wanted to do with Atomic since I was myself unresponsive with the
>> Magnum upstream because of my full-time Kolla commitment.
>>
>> I am pretty sure a bug was filed about this issue in the Red Hat
>>bugzilla,
>> but I can't find it.
>
>Thanks, the hint that it regarded rpm-ostree was enough for me to find
>this:
>
>    https://bugzilla.redhat.com/show_bug.cgi?id=1177989

This is the bug that ailed us :)

>
>Does that seem like the right issue? That in turn led me to this bug
>which appears fixed in F21, but I need to follow up to work out what if
>anything happened w.r.t. the second half of Colin's comment in the above
>bug (or whether it is still relevant):
>
>    https://bugzilla.redhat.com/show_bug.cgi?id=1178208

I think the speculation around selinux may or may not be correct.  We run
Magnum without Selinux, but building the magnum images with rpm-ostree may
run with selinux, so I'm not really certain.

The best way is to build an image and see if it works properly without the
1277989 bug.

>
>Somewhat tangentially I was also wondering if there are specific version
>combinations of components (be they images, or the versions of specific
>components they contain) that are currently recommended/preferred for a
>given release. Obviously there is the custom image that is currently
>provided but I am thinking more for folks trying to roll their own etc. I
>didn't spot anything while poking around in the docs as yet but I may be
>looking in the wrong places. What I am thinking of is something like this
>but for the Magnum context:
>
>    https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix

A dependency matrix with version numbers for all the things Atomic depends
on with relationship to Magnum (specifically kubernetes, etcd, docker) in
the upstream Magnum documentation would be fantastic.  We went through a
similar problem in Kolla with dependency version problems because the
upstreams we depend on move so fast, and ended up with docs that look like
this:

http://docs.openstack.org/developer/kolla/quickstart.html

Search for the component table.

One problem Magnum suffers from which is not really Atomic's fault, is
that Magnum's heat templates must be built for specific versions of etcd,
kubernetes, flanneld, etc.  This is because the upstreams keep changing
the damn config options around for no good reason.

I suspect since Kubernetes 1.0 was cut some time ago, this problem has
subsided, but it was a major major pain point in the past.  This is why we
released fedora-atomic images ourselves - to match the heat templates.
Unfortunately there is no way to mix and match different versions of the
heat templates in Magnum with different fedora atomic versions.

I think the Magnum core review team was thinking that containerizing these
things and then launching the appropriate container via heat would solve
the problem - that way the configuration options matched the version of
the container services being deployed.

This would leave Atomic with only a copy of docker pre-installed, which
would be a fantastic improvement in usability for the Magnum use case (but
may not be for the OpenShift or general Atomic use case).  YMMV :)

Hope that helps.

Regards
-steve

>
>Thanks,
>
>Steve
>
>
>[1] 
>https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2015-Ap
>ril/msg00042.html
>[2] 
>https://lists.fedoraproject.org/pipermail/cloud/2015-January/004811.html
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev