On 20/08/2019 19:05, Matt Riedemann wrote:

On 8/20/2019 7:02 AM, Mohammed Naser wrote:

...

...

Flexible: openstack-admin supports the fuzzy search for any important field(e.g. display_name/uuid/ip_address/project_name of an instance), which enables users to locate a particular object in no time. This is really useful to be honest, but we probably can work around it by using the filtering that APIs provide.

Isn't this what searchlight integration into horizon was for?

Yes - AFAIK, there was even an idea of integrating searchlight into the OSC / nova CLI for list / search operations. I really liked the idea of searchlight for fuzzy search, being able to search for an IP and finding the floating IP, server, and DNS Record seemed like a great tool.

On Mon, 2019-08-26 at 09:28 -0400, Camille Rodriguez wrote:

I like the idea and would like to see a demo if you get one ready. I agree with the previous suggestions about using and ameliorating the APIs instead of querying directly to the databases.

take this with a grain of salt as i dont know how alive/dead it is but if you are using go instead of using the api directly which you can do with the go sdk you could also look at oaktree which was ment to provide a grpc enpoint to wrap the api. https://opendev.org/x/oaktree (no update in 2 years with no updates so im thinking pretty dead.) from go grpc might be an easier way to interact but there is a go client https://opendev.org/x/golang-client too. the database are considerd internal to the project so while you might be able to read from them you certenly cannot write to them. maintain models that work over time would be non trivaila but if you use teh api this should be less of an issue. go is not really supported as a first class language for clients/proejcts so the tools and libs for go support are more or less non existent.

In my previous organization, we also struggled with the management of numerous big clusters and we developed a home-made Django web portal to satisfies queries and automation that Horizon could not do. I would be curious to see which functions have been developed for 'openstack-admin' and how similar the needs are.

Cheers, Camille Rodriguez

On Mon, Aug 26, 2019 at 7:26 AM Graham Hayes <gr@ham.ie> wrote:

...

On 20/08/2019 19:05, Matt Riedemann wrote:

...

On 8/20/2019 7:02 AM, Mohammed Naser wrote:

...

...

Flexible: openstack-admin supports the fuzzy search for any important field(e.g. display_name/uuid/ip_address/project_name of an instance), which enables users to locate a particular object in no time.

This is really useful to be honest, but we probably can work around it by using the filtering that APIs provide.

Isn't this what searchlight integration into horizon was for?

Yes - AFAIK, there was even an idea of integrating searchlight into the OSC / nova CLI for list / search operations.

I really liked the idea of searchlight for fuzzy search, being able to search for an IP and finding the floating IP, server, and DNS Record seemed like a great tool.

On Fri, Aug 23, 2019 at 3:21 AM Adrian Turjak <adriant@catalyst.net.nz> wrote:

Hello Douglas!

As someone who has struggled against the performance issues in Horizon I can easily feel your pain, and an effort to making something better is good. Sadly, this doesn't sound like a safe direction, even if for admin only purposes.

The first major issue is that you connect to the databases of the services directly. That's a major issue, both for long term compatibility, and security. The APIs should always be the main point of contact and the ONLY contract that the services have to maintain. By connecting to the database directly you are now relying on a data structure that can, and likely will change, and any security and sanity checking on filters and queries is now handled on your layer rather than the application itself. Not only that, but your dashboard also now needs passwords for all the databases, and by the sounds of it all the message queues.

I would highly encourage you to try and work with the community to fix the issues at the API layers rather than bypassing them. We can add better query and filtering to the APIs, and we can work on improving performance if we know the pain points, and this is likely where contributions would be welcome.

I think we do need an alternative to Horizon, and the ideal solution in my mind is to make a new dashboard built on either React or Vue, with a thin proxy app (likely in Flask) that serves the initial javascript page, and proxies any API requests to the services themselves. The filter issues should made better by implementing more complex filtering in the APIs themselves, and having the Dashboard layer better at exposing those dynamically. React or Vue would do a much better job of dynamically and quickly reloading and querying the services, and it would make the whole experience much nicer.

Michael Krotscheck did a bunch of work to put CORS config in many API services. Why bother with a proxy when we can continue that? :)

The one of the best parts of Horizon was that it was a 'dumb' dashboard built around your token. It can be deployed anywhere, by anyone, and only needs access to the cluster to work, no secrets to any database.

I know this is a huge issue, and we do need to solve it, but I hope we can work on something better that doesn't bypass the APIs, because that isn't a safe solution. :(

Cheers,

Adrian Turjak On 20/08/19 10:14 PM, Douglas Zhang wrote:

Hello everyone,

To help users interact with openstack, we’re currently developing a client-side web tool which enables administrators to manage their openstack cluster in a more efficient and convenient way. (Since we have not named it officially yet, I’m going to call it openstack-admin)

*# Introduction*

Some may ask, “Why do we need an extra web-based user interface since we have horizon?” Well, although horizon is a mature and powerful dashboard, it is far not efficient enough on big clusters (a simple list operation could take seconds to complete). What’s more, its flexibility of searching could not match our requirements. To overcome obstacles above, a more efficient tool is urgently required, that’s why we started to develop openstack-admin.

*# Highlights*

Comparing with the current user interface, openstack-admin has following advantages:

-

*Fast*: openstack-admin gets data straightly from SQL databases instead of calling standard openstack API, which accelerates the querying period to a large extent (especially when we’re dealing with a large amount of data). -

*Flexible*: openstack-admin supports the fuzzy search for any important field(e.g. display_name/uuid/ip_address/project_name of an instance), which enables users to locate a particular object in no time. -

*User-friendly*: the backend of openstack-admin gets necessary messages from the message queue used by nova, and send them to the frontend using websocket. This way, not only more realistic progress bars could be implemented, but more detailed information could be provided to users as well.

*# Issues*

To make this tool more efficient and provide better support for concurrency, we chose Golang to implement openstack-admin. As I’ve asked before (truly appreciate advises from Jeremy and Ghanshyam), a project written by an unofficial language may be accepted only if existing languages have been proven to not meet the technical requirements, so we’re considering re-implementing openstack-admin using python if we can’t come to an agreement on the language issue.

So that’s all. How do you guys think of this project?

Thanks,

Douglas Zhang