Open Stack

Murano folks,

I guess we should stop our debates with the Heat team about MuranoPL. What
we should do instead is to carefully read this thread couple of times and
collect and process all the feedback we've got. Steve and Zane did a very good
job helping us to find a way to align with Heat and Orhcestration program. Let
me outline the most important (imho) quotes:

# START quotes

Steven Dake:
------------
> I see no issue with HOT remaining simple and tidy focused entirely on
> orchestration (taking a desired state and converting that into reality) with
> some other imperative language layered on top to handle workflow and ALM.  I
> believe this separation of concerns is best for OpenStack and should be the
> preferred development path.


Zane Bitter:
------------
> I do think we should implement the hooks I mentioned at the start of this
> thread to allow tighter integration between Heat and a workflow engine
> (i.e. Mistral).
>
> So building a system on top of Heat is good. Building it on top of Mistral as
> well would also be good, and that was part of the feedback from the TC.
>
> To me, building on top means building on top of the languages (which users
> will have to invest a lot of work in learning) as well, rather than having a
> completely different language and only using the underlying implementation(s).
>
> To me that implies that Murano should be a relatively thin wrapper that ties
> together HOT and Mistral's DSL.


Steve Dake:
-----------
> I don't think HOT can do these things and I don't think we want HOT to do
> these things.  I am ok with that, since I don't see the pushback on having
> two languages for two different things in OpenStack.  I got from gokrove on
> iRC today that the rationale for the pushback was the TC wanted Murano folks
> to explore how to integrate better with Heat and possibly the orchestration
> program.  I don't see HOT as a place where there is an opportunity for scope
> expansion.  I see instead Murano creating HOT blobs and feeding them to Heat.


Zane Bitter:
------------
> Because there have to be some base types that you can use as building blocks.
> In the case of Heat, those base types are the set of things that you can
> create by talking to OpenStack APIs authenticated with the user's token.
> In the case of Mistral, I would expect it to be the set of actions that you
> can take by talking to OpenStack APIs authenticated with the user's token.
> And in the case of Murano, I would expect it to be the union of those two.
>
>
> Everything is a combination of existing resources, because the set of existing
> resources is the set of things which the operator provides as-a-Service. The
> set of things that the operator provides as a service plus the set of things
> that you can implement yourself on your own server (virtual or not) covers
> the entire universe of things. What you appear to be suggesting is that
> OpenStack must provide *Everything*-as-a-Service by allowing users to write
> their own services and have the operator execute them as-a-Service. This
> would be a breathtakingly ambitious undertaking, and I don't mean that in
> a good way.
>
> When the TC said "Murano is slightly too far up the stack at this point to
> meet the "measured progression of openstack as a whole" requirement", IMO
> one of the major things they meant was that you're inventing your own
> workflow thing, leading to duplication of effort between this and Workflow
> as a Service. (And Mistral folks are in turn doing the same thing by not
> using the same workflow library, taskflow, as the rest of OpenStack.)
> Candidly, I'm surprised that this is not the #1 thing on your priority list
> because IMO it's the #1 thing that will delay getting the project incubated.

# END quotes

Also I should quote what Georgy Okrokvertkhov said:
> Having this aligned I see a Murano package as an archive with all necessary
> definitions and resources and Murano service will just properly pass them to
> related services like Heat and Mistral. I think then Murano DSL will be much
> more simple and probably will be closer to declarative format with some
> specific data operations.


Summary:

I would like to propose further path for Murano evolution where it evolves as
an OpenStack project, aligned with others and developed in agreement between
all the interested sides. Here is the plan:

* Step forward and implement hooks (for workflow customization) in Heat. We
  will register a blueprint, discuss it with the Heat team and implement it
* Use our cross-project session on ATL summit to set clear goals and
  expectations
* Build pilot in Murano which:
   a. Uses new HOT Software components to do VM side software deployments
   b. Uses Mistral DSL to describe workflow. It'll require more focused
      discussion with Mistral team
   c. Bundles all that into an application package using new Murano DSL
   d. Allows users to combine applications in a single environenment
* Continuously align with the Heat team
* Continuously re-evaluate current state of pilot implementation and compare it
  with MuranoPL to keep track of features missing in pilot, but available in
  MuranoPL. If such feature exist - discuss it with the community. Find a place
  where the feature can be implemented and implement it
* Murano DSL will focus on:
  a. UI rendering
  b. HOT generation
  c. Setup other services (like put Mistral tasks to Mistral and bind
     them with events)

Speaking about new DSL for Murano. We're speaking about Application Lifecycle
Management. There are a lot of existing tools - Heat/HOT, Python, etc, but none
of them was designed with ALM in mind as a goal.

There are examples in OpenStack community when absolutely new project is
developed exactly for specific goal. For instance, there are a lot of task
trackers (including Launchpad), but none of them was designed to address the
needs of OpenStack community. That's why we're working on StoryBoard [0] - to
address specific domain area.

Clearly there is a need for a DSL to describe application packages. I believe
that almost everyone agrees that there should be a layer on top of HOT to
describe ALM. Feedback is - it should be simpler than our existing MuranoPL. So
let's design that layer together.


[0] https://wiki.openstack.org/wiki/StoryBoard

Thanks,
Ruslan