Open Stack

Hey folks.

Chad, IMO 1b and 2c

Jaromir,

thanks for your attention. I've answered inline, if you'll have some more
questions please ping me here or in IRC and I'll try to answer.

Thank you!

On Tue, Apr 22, 2014 at 3:38 PM, Jaromir Coufal <jcoufal at redhat.com> wrote:
> Hey Chad,
>
> thank you very much for starting this thread.
>
> Let me start with short introduction to my thoughts about OpenStack
> Dashboard's direction and our latest work there. I am working towards a
> higher goal of having the OpenStack UI a stable, simple and coherent story
> for our end users. Which means that it follows their workflows and enables
> them to be effective and happy when using it.
>
> Historically the UI resembled a little bit admin views, showing models and
> their objects in tables. We are improving here.
>
> One part which is leading towards the high level goal is information
> architecture reorganization which we started at last summit, but it got
> blocked a little bit by RBAC implementation. Though it is still an ongoing
> process.
>
> My concern is that the UI is starting to become a place where other projects
> just add another dashboard or panel in order to get incubated. I don't want
> this to happen, so I would like to put more thoughts where the Sahara
> functionality belongs and how it works together in relationship with other
> functionality and projects which are already in.
>
> To understand Sahara more, I read wiki page, various documentations and saw
> screencasts. After going through all of the materials there appeared various
> questions, which might be caused with lack of my knowledge about Sahara. So
> in the following paragraphs, I would like to put some questions (some might
> be rhetorical) and suggest few solutions to improve the placement of Sahara
> features.
>
> 1) What is the relationship of Sahara to Heat? Isn't Heat supposed to deal
> with provisioning of such a group of VMs? Is Hadoop cluster somehow
> different from other stacks that it can't be handled by Heat?

[SL] Sahara uses Heat for resources provisioning, than installs management tool
on them and this management tool installs Hadoop on all provided VMs. Cluster
in terms of Sahara is not just resources, but some
workflows/lifecycles on top of
it.
On the other side Heat will support Sahara Clusters as one of its resources,
so, you'll be able to use OS::Sahara::Cluster in Heat templates soon.

>
> The reason why I am asking is, that it looks to me, that what you are doing
> with Sahara in the first phase is actually designing your cluster with
> specific node types, you save it as a template (which only specifies which
> node types and how meany instances should run) and then you provision this
> cluster. For me it looks more or less the same as how Heat is supposed to
> work:
> * Node Type == Heat Resource (Type)
> * Cluster Template == Heat Template
> * Cluster == Heat Stack
>
> Therefore I am a little bit hesitative to adding another views, which are
> having similar purpose. If they are different, can you please be a bit more
> specific about the differences?

The main difference is that Node Group Templates are only storages for
Hadoop configurations.
Cluster Templates storing some cluster-wide configurations and cluster
topology, additionally it contains a number of node group templates with
possible overrides.
Clusters created from the cluster templates with possible overrides and
it's the real cluster object.

>
> 2) Lot of views (menu items) are reflecting mostly data model, but I don't
> think that there needs to be that many of them.
>
> Current views:
> * Clusters, Cluster Templates, Node Group Templates, Job Executions, Jobs,
> Job Binaries, Data Sources, Image Registry, Plugins
>
> Suggested structure A:
> * Orchestration
>     - Clusters / Stacks (one view including Clusters, Cluster Templates)
>     - Node Group Templates
> * Data Processing
>     - Overview
>     - Jobs (one view including Job Executions, Jobs, Job Binaries)
>     - Data Sources
> ? Image Registry - should be part of already existing Images catalog (why to
> separate them?)
[SL] It's a bit obsolete part that add tagging to glance images for
filtering them
in drop downs on Sahara pages.
> ? Plugins - Do they have to be managed via UI? Can't be all enabled by
> default and configured somewhere else in the preferences? I think it will
> confuse users to manage plugins from the top level main navigation - it's
> not something what with the user visit regularly.
[SL] Plugins page just shows list of enabled plugins.
>
> The question here is, if Clusters, Cluster Templates and Node Group
> Templates aren't supposed be reflected somehow with Heat, since Heat is the
> tool for Orchestration and it already deals with that. Because if we do
> this, then Clusters look like duplication for Heat Stacks (already described
> above in paragraph #1).
[SL] IMO it's not related to Heat. You'll be able to use OS::Sahara::Cluster in
Heat.
>
> if the answer is no, then suggested structure B:
> * Data Processing (specific only for Hadoop):
>    - Clusters
>    - Node Group Templates
>    - Jobs
>    - Data Sources
[SL] I like this structure, small correction: Clusters, Templates,
Jobs, Data Sources.
>
> I lean towards the A suggestion so far so I understand Sahara.
>
>
> Few comments inline:
>
>
> On 2014/17/04 21:06, Chad Roberts wrote:
>>
>> Per blueprint
>> https://blueprints.launchpad.net/horizon/+spec/merge-sahara-dashboard we are
>> merging the Sahara Dashboard UI code into the Horizon code base.
>>
>> Over the last week, I have been working on making this merge happen and
>> along the way some interesting questions have come up.  Hopefully, together
>> we can make the best possible decisions.
>>
>> Sahara is the Data Processing platform for Openstack.  During incubation
>> and prior to that, a horizon dashboard plugin was developed to work with the
>> data processing api.  Our original implementation was a separate dashboard
>> that we would activate by adding to HORIZON_CONFIG and INSTALLED_APPS.  The
>> layout gave us a root of "Sahara" on the same level as Admin and Project.
>> Under Sahara, we have 9 panels that make-up the entirety of the
>> functionality for the Sahara dashboard.
>>
>> Over the past week there seems to be at least 2 questions that have come
>> up.  I'd like to get input from anyone interested.
>>
>> 1)  Where should the functionality live within the Horizon UI? So far, 2
>> options have been presented.
>>      a)  In a separate dashboard (same level as Admin and Project).  This
>> is what we had in the past, but it doesn't seem to fit the flow of Horizon
>> very well.  I had a review up for this method at one point, but it was shot
>> down, so it is currently abandoned.
>
>
> I disagree with this option. Dashboards are used for different purposes
> (like very different use cases, contexts, earlier very different personas).
>
>
>>      b)  In a panel group under Project.  This is what I have stared work
>> on recently. This seems to mimic the way other things have been integrated,
>> but more than one person has disagreed with this approach.
>
>
> This is similar approach as was used lately in OpenStack Dashboard and I
> would like to avoid it. I would suggest option C, but this is based on my
> understanding from the top mentioned assumptions.
>
>>      c)  Any other options?
>
>
> It looks to me, that, Sahara belongs partly under orchestration panel,
> because it orchestrates VMs to deploy Hadoop cluster and partly under new
> panel call Data Processing (jobs executions).
>
>
>
>> 2)  Where should the code actually reside?
>>      a)  Under openstack_dashboards/dashboards/sahara  (or
>> data_processing).  This was the initial approach when the target was a
>> separate dashboard.
>>      b)  Have all 9 panels reside in
>> openstack_dashboards/dashboards/project.  To me, this is likely to
>> eventually make a mess of /project if more and more things are integrated
>> there.
>>      c)  Place all 9 data_processing panels under
>> openstack_dashboards/dashboards/project/data_processing  This essentially
>> groups the code by panel group and might make for a bit less mess.
>>      d)  Somewhere else?
>>
>>
>> The current plan is to discuss this at the next Horizon weekly meeting,
>> but even if you can't be there, please do add your thoughts to this thread.
>>
>> Thanks,
>> Chad Roberts (crobertsrh on irc)
>
>
> Cheers
> -- Jarda
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Sincerely yours,
Sergey Lukjanov
Sahara Technical Lead
(OpenStack Data Processing)
Mirantis Inc.