Open Stack

Hello, Drago!

I'm extremely interested in learning more about your HOT graphical
builder. The screenshots you had attached look gorgeous! Yours visual
representation of Heat resources is much more concise and simple than
I had drawn in Merlin PoC mock-ups [1]. On the other hand I have some
suspicions that D3.js is a good fit for a general purpose UI toolkit
Merlin aims to provide. Please don't get me wrong, D3.js is a great
library which can do fantastic things with data - in case your
data<->visualization use-case maps to the one of the facilities D3.js
provides out of the box. In case it doesn't, there are 2 options:
either change your approach to what should be visualized/how it should
be visualized, or tweak some inner machinery of D3.js

While bending the design towards the facilities of D3.js doesn't seem
a viable choice, changing D3.js from inside can be painful too. AFAIK
force-directed graph layout from D3.js doesn't provide the means to
represent composable entities (which isn't a big problem for Heat, but
is a very serious issue for Murano) out of the box. By composable I
mean something like [2] - but with much more complex inner structure
(imagine the Resource entity [3] having as its properties other
Resource entities which are shown as simple rounded rectangles with
labels on that picture, but are expanded into complex objects similar
to [3] once the user, say, clicks on them). As far as I understand,
you are visualizing that kind of composition via arrow links, but I'd
like to try another design options (especially in case of Murano) and
fear that D3.js will constrain me here. I've been thinking a bit about
using more low-level SVG js-framework like Raphael.js - it doesn't
offer most of the goodies D3.js does, but also it doesn't force me to
create the design based on some data transformations in a way that
D3.js does, providing the good old procedural API instead. Of course,
I may be wrong, perhaps more time and efforts invested into Merlin PoC
would allow me to realize it (or not).

Yet you are totally right having stressed the importance of right tool
for implementing the underlying object model (or JSON-wrapper as you
called it) - Barricade.js. That's the second big part of work Merlin
had to do, and I couldn't underestimate how it would be beneficial for
Merlin to leverage some of the facilities that Barricade.js provides.
I'll gladly look at the demo of template builder and Barricade. Is
there any chance I could take a look also at the source code of
Barricade.js, so I would better understand to which extent it suits
Merlin’s needs? I've searched through github.com and didn't found any
traces of Barricade.js repo, so it seems like some in-house project to
me. What are your plans for sharing this library with the community?

[1] https://wiki.openstack.org/wiki/Merlin/PoC
[2] http://mbostock.github.io/d3/talk/20111116/pack-hierarchy.html
[3] https://wiki.openstack.org/wiki/File:Merlin_poc_3.png

On Wed, Aug 27, 2014 at 6:14 PM, Drago Rosson
<drago.rosson at rackspace.com> wrote:
> Hello Timur,
>
> I have been developing a graphical Heat Orchestration Template builder. I
> first heard about the Merlin project around a week ago and found that what
> I have developed in the past few months is in line with what Merlin is
> trying to accomplish.
>
> Some of the features of the template builder (described further down)
> could be used as part of the UI framework Merlin aims to produce. However,
> its most important and useful component is a JavaScript library I have
> developed called Barricade.js, which powers the template builder’s data
> layer.
>
> Barricade.js aims to solve the problem of using JSON data across a web
> app. It creates data model objects out of JSON using a predefined schema
> (which looks similar to the schema used in your Murano Dynamic UI). This
> removes the disadvantages with JSON and introduces some very important
> advantages:
>
> - Encapsulation. JSON values can be set to any type or deleted entirely,
> which either causes errors when UI components expect these values to exist
> or be of a certain type, or forces the UI components to constantly check
> for correctness. Barricade instead wraps around the JSON and provides
> accessor methods to ensure type-safe data manipulation. Additionally,
> Barricade objects are observable, so changes made to their data trigger
> events that can be subscribed to by UI components.
>
> - Normalization. Whenever properties that are defined in the schema are
> missing in the JSON, Barricade will fill them in with default values. This
> way, UIs will always have valid values where they expect them, making
> their design much simpler. Optional properties are extremely common in
> Heat templates.
>
> - Metadata. Anything extra attached to JSON must be handled carefully
> (such as when converting back to the original YAML format). By wrapping
> the JSON with Barricade, metadata and convenience methods that UI
> components can use can be defined. For instance, the datatype of any value
> or a description to go along with each property in a Heat resource.
>
> - Validation. Soon, the schema will allow defining validators that will
> run whenever a new value is attempted to be set on data. Messages about
> failed validation will be available so that the UI can display it. This
> system seems to be very similar to dynamic UI’s.
>
> What this all boils down to is that all of the logic required to ensure
> JSON’s integrity is rolled into Barricade instead of sprinkled into all of
> the UI code. This way, UI components can be confident in the data that
> they are working with, which makes their code more concise and faster to
> develop.
>
> Barricade seems like a great fit for Merlin because the projects it
> targets (Heat, Solum, Mistral, Murano) use YAML files that can be used
> with Barricade once they are converted to JSON.
>
>
> About the template builder (see screenshots attached):
>
> - Uses an interactive canvas (powered by a D3.js force-directed graph) to
> display Heat resources and the dependencies in between them. New resources
> can be drag-and-dropped from a panel onto the canvas. Different resource
> types are attracted to different respective areas so that they
> automatically arrange themselves in a familiar network topology hierarchy.
>  A form/panel is displayed when a resource on the canvas is clicked so
> that the resource’s properties can be edited.
>
> - Has two―way data binding provided by Knockout.js interfacing with
> Barricade.js so that editing values in forms automatically updates the
> topology on the canvas (e.g. Changing resource IDs or creating/removing
> dependencies between resources).
>
> - Allows for direct-editing of the template (via text input) and loading
> templates via URLs.
>
>
> Please, let me know if you would like a demo of the template builder and
> Barricade!
>
> Thanks,
> Drago Rosson
>
>
>
>
> On 8/18/14, 5:19 AM, "Timur Sufiev" <tsufiev at mirantis.com> wrote:
>
>>David,
>>
>>I'm happy to hear that :)! After thinking a bit, I came up with the
>>following strategy for further Merlin development: make all the
>>commits into a separate repository (stackforge/merlin) at least until
>>the PoC is ready. This will allow to keep project history more
>>granular instead of updating one large commit inside openstack/horizon
>>gerrit (thus also lessening the burden on Horizon reviewers). Once the
>>Merlin proceeds from the experimental/PoC phase to the implementing of
>>a more elaborated spec, it will be just the time for it to join with
>>the Horizon.
>>
>>On Wed, Aug 13, 2014 at 2:48 AM, Lyle, David <david.lyle at hp.com> wrote:
>>> On 8/6/14, 1:41 PM, "Timur Sufiev" <tsufiev at mirantis.com> wrote:
>>>
>>>>Hi, folks!
>>>>
>>>>Two months ago there was an announcement in ML about gathering the
>>>>requirements for cross-project UI library for
>>>>Heat/Mistral/Murano/Solum [1]. The positive feedback in related
>>>>googledoc [2] and some IRC chats and emails that followed convinced me
>>>>that I'm not the only person interested in it :), so I'm happy to make
>>>>the next announcement.
>>>>
>>>>The project finally has got its name - 'Merlin' (making complex UIs is
>>>>a kind of magic), Openstack wiki page [3] and all other stuff like
>>>>stackforge repo, launchpad page and IRC channel (they are all
>>>>referenced in [3]). For those who don't like clicking the links, here
>>>>is quick summary.
>>>>
>>>>Merlin aims to provide a convenient client side framework for building
>>>>rich UIs for Openstack projects dealing with complex input data with
>>>>lot of dependencies and constraints (usually encoded in YAML format
>>>>via some DSL) - projects like Heat, Murano, Mistral or Solum. The
>>>>ultimate goal for such UI is to save users from reading comprehensive
>>>>documentation just in order to provide correct input data, thus making
>>>>the UI of these projects more user-friendly. If things go well for
>>>>Merlin, it could be eventually merged into Horizon library (I¹ll spare
>>>>another option for the end of this letter).
>>>>
>>>>The framework trying to solve this ambitious task is facing at least 2
>>>>challenges:
>>>>(1) enabling the proper UX patterns and
>>>>(2) dealing with complexities of different projects' DSLs.
>>>>
>>>>Having worked on DSL things in Murano project before, I'm planning at
>>>>first to deal with the challenge (2) in the upcoming Merlin PoC. So,
>>>>here is the initial plan: design an in-framework object model (OM)
>>>>that could translated forth and back into target project's DSL. This
>>>>OM is meant to be synchronised with visual elements shown on browser
>>>>canvas. Target project is the Heat with its HOT templates - it has the
>>>>most well-established syntax among other projects and comprehensive
>>>>documentation.
>>>>
>>>>Considering the challenge (1), not being a dedicated UX engineer, I'm
>>>>planning to start with some rough UI concepts [4] and gradually
>>>>improve them relying on community feedback, and especially, Openstack
>>>>UX group. If anybody from the UX team (or any other team!) is willing
>>>>to be involved to a greater degree than just giving some feedback,
>>>>you're are enormously welcome! Join Merlin, it will be fun :)!
>>>>
>>>>Finally, with this announcement I¹d like to start a discussion with
>>>>Horizon community. As far as I know, Horizon in its current state
>>>>lacks such UI toolkit as Merlin aims to provide. Would it be by any
>>>>chance possible for the Merlin project to be developed from the very
>>>>beginning as part of Horizon library? This choice has its pros and
>>>>cons I¹m aware of, but I¹d like to hear the opinions of Horizon
>>>>developers on that matter.
>>>
>>> I would like to see this toolset built into Horizon. That will make it
>>> accessible to integrated projects like Heat that Horizon already
>>>supports,
>>> but will also allow other projects to use the horizon library as a
>>> building block to providing managing project specific DSLs.
>>>
>>> David
>>>
>>>>
>>>>[1]
>>>>http://lists.openstack.org/pipermail/openstack-dev/2014-June/037054.html
>>>>[2]
>>>>https://docs.google.com/a/mirantis.com/document/d/19Q9JwoO77724RyOp7XkpY
>>>>mA
>>>>Lwmdb7JjoQHcDv4ffZ-I/edit#
>>>>[3] https://wiki.openstack.org/wiki/Merlin
>>>>[4] https://wiki.openstack.org/wiki/Merlin/SampleUI
>>>>
>>>>--
>>>>Timur Sufiev
>>>>
>>>>_______________________________________________
>>>>OpenStack-dev mailing list
>>>>OpenStack-dev at lists.openstack.org
>>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>--
>>Timur Sufiev
>>
>>_______________________________________________
>>OpenStack-dev mailing list
>>OpenStack-dev at lists.openstack.org
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Timur Sufiev