Open Stack

Hey Chad,

thank you very much for starting this thread.

Let me start with short introduction to my thoughts about OpenStack 
Dashboard's direction and our latest work there. I am working towards a 
higher goal of having the OpenStack UI a stable, simple and coherent 
story for our end users. Which means that it follows their workflows and 
enables them to be effective and happy when using it.

Historically the UI resembled a little bit admin views, showing models 
and their objects in tables. We are improving here.

One part which is leading towards the high level goal is information 
architecture reorganization which we started at last summit, but it got 
blocked a little bit by RBAC implementation. Though it is still an 
ongoing process.

My concern is that the UI is starting to become a place where other 
projects just add another dashboard or panel in order to get incubated. 
I don't want this to happen, so I would like to put more thoughts where 
the Sahara functionality belongs and how it works together in 
relationship with other functionality and projects which are already in.

To understand Sahara more, I read wiki page, various documentations and 
saw screencasts. After going through all of the materials there appeared 
various questions, which might be caused with lack of my knowledge about 
Sahara. So in the following paragraphs, I would like to put some 
questions (some might be rhetorical) and suggest few solutions to 
improve the placement of Sahara features.

1) What is the relationship of Sahara to Heat? Isn't Heat supposed to 
deal with provisioning of such a group of VMs? Is Hadoop cluster somehow 
different from other stacks that it can't be handled by Heat?

The reason why I am asking is, that it looks to me, that what you are 
doing with Sahara in the first phase is actually designing your cluster 
with specific node types, you save it as a template (which only 
specifies which node types and how meany instances should run) and then 
you provision this cluster. For me it looks more or less the same as how 
Heat is supposed to work:
* Node Type == Heat Resource (Type)
* Cluster Template == Heat Template
* Cluster == Heat Stack

Therefore I am a little bit hesitative to adding another views, which 
are having similar purpose. If they are different, can you please be a 
bit more specific about the differencies?

2) Lot of views (menu items) are reflecting mostly data model, but I 
don't think that there needs to be that many of them.

Current views:
* Clusters, Cluster Templates, Node Group Templates, Job Executions, 
Jobs, Job Binaries, Data Sources, Image Registry, Plugins

Suggested structure A:
* Orchestration
     - Clusters / Stacks (one view including Clusters, Cluster Templates)
     - Node Group Templates
* Data Processing
     - Overview
     - Jobs (one view including Job Executions, Jobs, Job Binaries)
     - Data Sources
? Image Registry - should be part of already existing Images catalog 
(why to separate them?)
? Plugins - Do they have to be managed via UI? Can't be all enabled by 
default and configured somewhere else in the preferences? I think it 
will confuse users to manage plugins from the top level main navigation 
- it's not something what with the user visit regularly.

The question here is, if Clusters, Cluster Templates and Node Group 
Templates aren't supposed be reflected somehow with Heat, since Heat is 
the tool for Orchestration and it already deals with that. Because if we 
do this, then Clusters look like duplication for Heat Stacks (already 
described above in paragraph #1).

if the answer is no, then suggested structure B:
* Data Processing (specific only for Hadoop):
    - Clusters
    - Node Group Templates
    - Jobs
    - Data Sources

I lean towards the A suggestion so far so I understand Sahara.


Few comments inline:

On 2014/17/04 21:06, Chad Roberts wrote:
> Per blueprint  https://blueprints.launchpad.net/horizon/+spec/merge-sahara-dashboard we are merging the Sahara Dashboard UI code into the Horizon code base.
>
> Over the last week, I have been working on making this merge happen and along the way some interesting questions have come up.  Hopefully, together we can make the best possible decisions.
>
> Sahara is the Data Processing platform for Openstack.  During incubation and prior to that, a horizon dashboard plugin was developed to work with the data processing api.  Our original implementation was a separate dashboard that we would activate by adding to HORIZON_CONFIG and INSTALLED_APPS.  The layout gave us a root of "Sahara" on the same level as Admin and Project.  Under Sahara, we have 9 panels that make-up the entirety of the functionality for the Sahara dashboard.
>
> Over the past week there seems to be at least 2 questions that have come up.  I'd like to get input from anyone interested.
>
> 1)  Where should the functionality live within the Horizon UI? So far, 2 options have been presented.
>      a)  In a separate dashboard (same level as Admin and Project).  This is what we had in the past, but it doesn't seem to fit the flow of Horizon very well.  I had a review up for this method at one point, but it was shot down, so it is currently abandoned.

I disagree with this option. Dashboards are used for different purposes 
(like very different use cases, contexts, earlier very different personas).

>      b)  In a panel group under Project.  This is what I have stared work on recently. This seems to mimic the way other things have been integrated, but more than one person has disagreed with this approach.

This is similar approach as was used lately in OpenStack Dashboard and I 
would like to avoid it. I would suggest option C, but this is based on 
my understanding from the top mentioned assumptions.

>      c)  Any other options?

It looks to me, that, Sahara belongs partly under orchestration panel, 
because it orchestrates VMs to deploy Hadoop cluster and partly under 
new panel call Data Processing (jobs executions).


> 2)  Where should the code actually reside?
>      a)  Under openstack_dashboards/dashboards/sahara  (or data_processing).  This was the initial approach when the target was a separate dashboard.
>      b)  Have all 9 panels reside in openstack_dashboards/dashboards/project.  To me, this is likely to eventually make a mess of /project if more and more things are integrated there.
>      c)  Place all 9 data_processing panels under openstack_dashboards/dashboards/project/data_processing  This essentially groups the code by panel group and might make for a bit less mess.
>      d)  Somewhere else?
>
>
> The current plan is to discuss this at the next Horizon weekly meeting, but even if you can't be there, please do add your thoughts to this thread.
>
> Thanks,
> Chad Roberts (crobertsrh on irc)

Cheers
-- Jarda