Bah, sent it via my phone and didn't hit reply all. On 26/08/19 10:55 PM, Jim Rollenhagen wrote:

Fair points. In case you didn't realize, you only sent this to me, not the list :)

// jim

On Sat, Aug 24, 2019 at 1:45 AM Adrian Turjak <adriant@catalyst.net.nz <mailto:adriant@catalyst.net.nz>> wrote:

On 24 Aug. 2019 00:08, Jim Rollenhagen <jim@jimrollenhagen.com <mailto:jim@jimrollenhagen.com>> wrote:

On Fri, Aug 23, 2019 at 3:21 AM Adrian Turjak <adriant@catalyst.net.nz <mailto:adriant@catalyst.net.nz>> wrote:

Hello Douglas!

As someone who has struggled against the performance issues in Horizon I can easily feel your pain, and an effort to making something better is good. Sadly, this doesn't sound like a safe direction, even if for admin only purposes.

The first major issue is that you connect to the databases of the services directly. That's a major issue, both for long term compatibility, and security. The APIs should always be the main point of contact and the ONLY contract that the services have to maintain. By connecting to the database directly you are now relying on a data structure that can, and likely will change, and any security and sanity checking on filters and queries is now handled on your layer rather than the application itself. Not only that, but your dashboard also now needs passwords for all the databases, and by the sounds of it all the message queues.

I would highly encourage you to try and work with the community to fix the issues at the API layers rather than bypassing them. We can add better query and filtering to the APIs, and we can work on improving performance if we know the pain points, and this is likely where contributions would be welcome.

I think we do need an alternative to Horizon, and the ideal solution in my mind is to make a new dashboard built on either React or Vue, with a thin proxy app (likely in Flask) that serves the initial javascript page, and proxies any API requests to the services themselves. The filter issues should made better by implementing more complex filtering in the APIs themselves, and having the Dashboard layer better at exposing those dynamically. React or Vue would do a much better job of dynamically and quickly reloading and querying the services, and it would make the whole experience much nicer.

Michael Krotscheck did a bunch of work to put CORS config in many API services. Why bother with a proxy when we can continue that? :) 

We deploy our dashboard publically, but our APIs are behind a whitelist only firewall. I bet we're not the only one. So we need the proxy like nature of the dashboard API calls.

I personally like the idea of the browser able to talk to the APIs directly, but I have a feeling there may be other reasons to avoid that. Not to mention you get a clearer picture of which API calls come from the dashboard. Plus, a proxy via the dashboard app itself (assuming it is deployed near the cluster) won't exactly add much extra to the response time.

But if we as a community do end up going down this route, we can discuss if we need the app to proxy or not. If not, then really we are just building a pure JavaScript app, but if we do need to proxy, then the proxy layer would still be very simple with the core logic all in the front-end.

The one of the best parts of Horizon was that it was a 'dumb' dashboard built around your token. It can be deployed anywhere, by anyone, and only needs access to the cluster to work, no secrets to any database.

I know this is a huge issue, and we do need to solve it, but I hope we can work on something better that doesn't bypass the APIs, because that isn't a safe solution. :(

Cheers,

Adrian Turjak

On 20/08/19 10:14 PM, Douglas Zhang wrote:

Hello everyone,

To help users interact with openstack, we’re currently developing a client-side web tool which enables administrators to manage their openstack cluster in a more efficient and convenient way. (Since we have not named it officially yet, I’m going to call it openstack-admin)

*# Introduction*

Some may ask, “Why do we need an extra web-based user interface since we have horizon?” Well, although horizon is a mature and powerful dashboard, it is far not efficient enough on big clusters (a simple |list| operation could take seconds to complete). What’s more, its flexibility of searching could not match our requirements. To overcome obstacles above, a more efficient tool is urgently required, that’s why we started to develop openstack-admin.

*# Highlights*

Comparing with the current user interface, openstack-admin has following advantages:

*

*Fast*: openstack-admin gets data straightly from SQL databases instead of calling standard openstack API, which accelerates the querying period to a large extent (especially when we’re dealing with a large amount of data).

*

*Flexible*: openstack-admin supports the fuzzy search for any important field(e.g. display_name/uuid/ip_address/project_name of an instance), which enables users to locate a particular object in no time.

*

*User-friendly*: the backend of openstack-admin gets necessary messages from the message queue used by nova, and send them to the frontend using websocket. This way, not only more realistic progress bars could be implemented, but more detailed information could be provided to users as well.

*# Issues*

To make this tool more efficient and provide better support for concurrency, we chose Golang to implement openstack-admin. As I’ve asked before (truly appreciate advises from Jeremy and Ghanshyam), a project written by an unofficial language may be accepted only if existing languages have been proven to not meet the technical requirements, so we’re considering re-implementing openstack-admin using python if we can’t come to an agreement on the language issue.

So that’s all. How do you guys think of this project?

Thanks,

Douglas Zhang