Open Stack

I've been holding off, but I'll chime in now.

I believe Higgins should be about abstracting away differences in the container systems where there are needless differences the user really could care less about. IE,
"here, launch this container" can be done:
 * k8s pod create ...
 * docker run ...
 * docker-compose ...
 * nova boot --user-data "#!/bin/bash\ndocker run ..." ...
 * #insert mesos cli here... I don't know it off hand...

But, if you start building up COE like functionality on a base where there is no LCD functionality, you end up having to reimplement a COE as the underlying COE can't do the things you want and you use it as just a container launcher.

Now, since we're staring to talk about coming up with a new COE, I've got to insert obligatory standards xkcd here:
https://xkcd.com/927/

I recommend sticking to the stable, LCD like functionality for now. With the COE situation in flux, I think its likely either 1 of 4 things will happen.

1. Advanced features, like pods, will become so critical to developers that the COE's that don't support them will gain them, and then the LCD is reasonable again for advanced functionality. (likely)
2. The COE's that don't support features like pods will die out as they aren't useful, and then targeting the LCD is again reasonable. (also likely)
3. One COE rises to dominance pushing out the others. (possible)
4. People will forget about the need of features like pods as some other abstraction that ends up being more useful gets adopted. (unlikely)

Getting the basic, get containers launched, functionality is something that can be done today, and will immediately be useful to users. Doing COE advanced feature implementation will be a lot more work if you don't target the LCD, and may be needless for reasons listed above. Lets hold off and see where things settle.

In my personal experience converting apps to containers, I have not been able to live without the k8s notion of pods for a number of apps. I use either heat-templates + docker-compose or k8s to launch containers in sets that use unix sockets for communication. docker-swarm has been a non starter due to the lack of pods, and I haven't looked at mesos too much yet, as k8s has been working well for me.

Thanks,
Kevin

________________________________
From: Denis Makogon [lildee1991 at gmail.com]
Sent: Tuesday, May 31, 2016 11:05 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [higgins] Docker-compose support

Hello Hongbin.

I would disagree on what you are saying, because having Higgins doing too basic stuff is not very valuable. As for those of us who works with development and continuous delivery how can Higgins address, for example, micro-service chaining?

In any case Higgins eventually will end up having its own DSL (or TOSCA, or compose DSL) because there are not so much benefits from having API that only spin-up containers separately. Developers will, again, have to build solution over Higgins to support more advanced things like service chaining and that would mean that Higgins doesn't meet their requirements for further service consumption.


Kind regards,
Denys Makogon


2016-05-31 23:15 GMT+03:00 Hongbin Lu <hongbin.lu at huawei.com<mailto:hongbin.lu at huawei.com>>:
I don’t think it is a good to re-invent docker-compose in Higgins. Instead, we should leverage existing libraries/tools if we can.

Frankly, I don’t think Higgins should interpret any docker-compose like DSL in server, but maybe it is a good idea to have a CLI extension to interpret specific DSL and translate it to a set of REST API calls to Higgins server. The solution should be generic enough so that we can re-use it to interpret another DSL (e.g. pod, TOSCA, etc.) in the future.

Best regards,
Hongbin

From: Denis Makogon [mailto:lildee1991 at gmail.com<mailto:lildee1991 at gmail.com>]
Sent: May-31-16 3:25 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [higgins] Docker-compose support

Hello.

It is hard to tell if given API will be final version, but i tried to make it similar to CLI and its capabilities. So, why not?

2016-05-31 22:02 GMT+03:00 Joshua Harlow <harlowja at fastmail.com<mailto:harlowja at fastmail.com>>:
Cool good to know,

I see https://github.com/docker/compose/pull/3535/files#diff-1d1516ea1e61cd8b44d000c578bbd0beR66

Would that be the primary API? Hard to tell what is the API there actually, haha. Is it the run() method?

I was thinking more along the line that higgins could be a 'interpreter' of the same docker-compose format (or similar format); if the library that is being created takes a docker-compose file and turns it into a 'intermediate' version/format that'd be cool. The compiled version would then be 'executable' (and introspectable to) by say higgins (which could say traverse over that intermediate version and activate its own code to turn the intermediate versions primitives into reality), or a docker-compose service could or ...

What abou TOSCA? From my own perspective compose format is too limited, so it is really necessary to consider regarding use of TOSCA in Higgins workflows.


Libcompose also seems to be targeted at a higher level library, from at least reading the summary, neither seem to be taking a compose yaml file, turning it into a intermediate format, exposing that intermediate format to others for introspection/execution (and also likely providing a default execution engine that understands that format) but instead both just provide an equivalent of:

That's why i've started this thread, as community we have use cases for Higgins itself and for compose but most of them are not formalized or even written. Isn't this a good time to define them?

  project = make_project(yaml_file)
  project.run/up()

Which probably isn't the best API for something like a web-service that uses that same library to have. IMHO having a long running run() method

Well, compose allows to run detached executions for most of its API calls. By use of events, we can track service/containers statuses (but it is not really trivial).

exposed, without the necessary state tracking, ability to interrupt/pause/resume that run() method and such is not going to end well for users of that lib (especially a web-service that needs to periodically be `service webservice stop` or restart, or ...).

Yes, agreed. But docker or swarm by itself doesn't provide such API (can't tell the same for K8t).

Denis Makogon wrote:
Hello Stackers.


As part of discussions around what Higgins is and what its mission there
are were couple of you who mentioned docker-compose [1] and necessity of
doing the same thing for Higgins but from scratch.

I don't think that going that direction is the best way to spend
development cycles. So, that's why i ask you to take a look at recent
patchset submitted to docker-compose upstream [2] that makes this tool
(initially designed as CLI) to become a library with Python API.  The
whole idea is to make docker-compose look similar to libcompose [3]
(written on Go).

If we need to utilize docker-compose features in Higgins i'd recommend
to work on this with Docker community and convince them to land that
patch to upstream.

If you have any questions, please let me know.

[1] https://docs.docker.com/compose/
[2] https://github.com/docker/compose/pull/3535
[3] https://github.com/docker/libcompose


Kind regards,
Denys Makogon
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160601/4661f3fd/attachment-0001.html>