- openstack-discuss - lists.openstack.org

[oslo] Reminder: Feature Freeze is this Friday
by Ben Nemec 28 Aug '19

28 Aug '19

This is your friendly neighborhood PTL reminding you that if you need a feature to merge in Oslo this cycle, it should be in by this Friday. We feature freeze earlier than other projects so they have a chance to make use of any features we add before their feature freeze. After Friday, any new features in Oslo will require an exception from me. I don't see much in the way of feature reviews open against Oslo right now (at least none that are mergeable), but if I missed anything let me know. Thanks. -Ben

1 0

[requirements][placement] orjson instead of stdlib json
by Chris Dent 28 Aug '19

28 Aug '19

Requirements people: For the past few months we've been doing profiling of placement, modeling large clouds where many results (7000 resource providers) will be returned in response to common requests. One of the last remaining chunks of slowness is serializing the very large dicts representing those providers (and associated allocation requests) to JSON (greater than 100k lines when pretty printed). Using the oslo jsonutils (which is a light wrapper over the stdlib 'json' module) approximately 25% of time is consumed by dumps(). Using orjson [1] instead, time consumption drops to 1%, so this is something it would be good to make real in placement (and perhaps other projects that have large result sets). There's a WIP that demonstrates its use at [2]. I'm posting about it because JSON serialization seems like it might be an area that the requirements team has greater interest than some other areas. Also, orjson is only Python 3, so use needs to be adapted for Python 2, as shown in the WIP [2]. Otherwise it's fine: active, Apache2 or MIT license. Thoughts? [1] https://pypi.org/project/orjson/ [2] https://review.opendev.org/#/c/674661/ -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

7 9

Re: [openstack-dev] Dropping lazy translation support
by Shengjing Zhu 28 Aug '19

28 Aug '19

Sorry for replying the old mail, and please cc me when reply. Matt Riedemann <mriedemos at gmail.com> writes: > This is a follow up to a dev ML email [1] where I noticed that some > implementations of the upgrade-checkers goal were failing because some > projects still use the oslo_i18n.enable_lazy() hook for lazy log message > translation (and maybe API responses?). > > The very old blueprints related to this can be found here [2][3][4]. > > If memory serves me correctly from my time working at IBM on this, this > was needed to: > > 1. Generate logs translated in other languages. > > 2. Return REST API responses if the "Accept-Language" header was used > and a suitable translation existed for that language. > > #1 is a dead horse since I think at least the Ocata summit when we > agreed to no longer translate logs since no one used them. > > #2 is probably something no one knows about. I can't find end-user > documentation about it anywhere. It's not tested and therefore I have no > idea if it actually works anymore. > > I would like to (1) deprecate the oslo_i18n.enable_lazy() function so > new projects don't use it and (2) start removing the enable_lazy() usage > from existing projects like keystone, glance and cinder. > > Are there any users, deployments or vendor distributions that still rely > on this feature? If so, please speak up now. I was pointed to this discussion when I tried to fix this feature in keystone, https://review.opendev.org/677117 For #2 translated API response, this feature probably hasn't been working for some time, but it's still a valid user case. Has the decision been settled? -- Regards, Shengjing Zhu

3 4

[openstack-ansible] weekly update
by Mohammed Naser 28 Aug '19

28 Aug '19

Hi everyone, Here’s the update of what happened in this week’s OpenStack Ansible Office Hours. - The uwsgi role is mostly done but there’s an issue with ansible-lint checks so we’re thinking of moving it to the integrated repository and towards using collections in the long term. - We’re still working on the os_murano role. - We finally fixed Rocky POST_FAILURES (thanks infra!) - Rabbit and Galera are still stuck on bind-to-mgmt work.. - We will also need new code in order to build images in test jobs for Octavia since right now they’re only usable in Master. Thanks! Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

1 0

No manila meeting 29 August 2019
by Tom Barron 28 Aug '19

28 Aug '19

We have quite a few folks unavailable this Thursday and will cancel this week's Manila community meeting. The next meeting will be the following week, 5 September, at 1500 UTC as usual.

1 0

[nova] ComputeDriver.spawn getting a power_on kwarg
by Matt Riedemann 28 Aug '19

28 Aug '19

This is just an FYI to out of tree virt driver maintainers that the ComputeDriver.spawn interface is getting a new power_on kwarg with this chnage: https://review.opendev.org/#/c/642590/ -- Thanks, Matt

1 0

FWAAS V2 doesn't work with DVR
by Salman Khan 28 Aug '19

28 Aug '19

Hi Guys, I asked this question over #openstack-neutron channel but didn't get any answer, so asking here in a hope that someone might read this email and reply. The problem is: I have enabled FWAAS_V2 with DVR and that doesn't seem to work. I debugged things down to router namespaces and it looks like iptables rules are applied to rfp-<network-id> interface which doesn't exist in that namespace. So rules are completely wrong as they are applied to an interface that doesn't exist, I mean there is rfp-* interface but the <network-id> that fwaas expecting is not what it should be. I tried applying the rules to qr-* interfaces in the namespace but that didn't work as well, packets are dropping on "invalid" state rule. That's probably because of nat rules from dvr. Can someone please help me to understand this behaviour. Is it really suppose to work or not. If there is any bug or fix pending or there is any work ongoing to support this. Regards, Salman

2 2

[scientific-sig] SIG IRC meeting today 1100 UTC
by Stig Telfer 28 Aug '19

28 Aug '19

Hi All - We have an IRC meeting today at 1100 UTC (about an hour’s time) in channel #openstack-meeting. Everyone is welcome. Today’s agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_August_28th_2019 <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_August_28th_2019> Cheers, Stig

1 0

[glance][neutron] Glance migrations broken for postgresql; impact to Neutron periodic jobs
by Nate Johnston 28 Aug '19

28 Aug '19

Glance developers, I am investigating failures with the neutron-tempest-postgres-full periodic job in Zuul, and it looks like the failing step is actually a database migration for Glance that occurs during devstack setup. Pasted below is a representative example [1]. You can see the same issue causing failures for this neutron periodic job by looking at the Zuul failures [2]; the error is in logs/devstacklog.txt.gz. The last successful job was 2019-08-15T06:07:06; the next one on 2019-08-16 failed. But there were no changes merged in glance during that time period [3]. So if you could help debug the source of this issue then we can hopefully get this job - and probably whatever others depend on postgresql - back working. 2019-08-27 06:35:44.963 | INFO [alembic.runtime.migration] Running upgrade rocky_expand02 -> train_expand01, empty expand for symmetry with train_contract01 2019-08-27 06:35:44.967 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl. 2019-08-27 06:35:44.967 | INFO [alembic.runtime.migration] Will assume transactional DDL. 2019-08-27 06:35:44.970 | Upgraded database to: train_expand01, current revision(s): train_expand01 2019-08-27 06:35:44.971 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl. 2019-08-27 06:35:44.971 | INFO [alembic.runtime.migration] Will assume transactional DDL. 2019-08-27 06:35:44.975 | INFO [alembic.runtime.migration] Context impl PostgresqlImpl. 2019-08-27 06:35:44.975 | INFO [alembic.runtime.migration] Will assume transactional DDL. 2019-08-27 06:35:44.995 | CRITI [glance] Unhandled error 2019-08-27 06:35:44.996 | Traceback (most recent call last): 2019-08-27 06:35:44.996 | File "/usr/local/bin/glance-manage", line 10, in <module> 2019-08-27 06:35:44.996 | sys.exit(main()) 2019-08-27 06:35:44.996 | File "/opt/stack/new/glance/glance/cmd/manage.py", line 563, in main 2019-08-27 06:35:44.996 | return CONF.command.action_fn() 2019-08-27 06:35:44.996 | File "/opt/stack/new/glance/glance/cmd/manage.py", line 395, in sync 2019-08-27 06:35:44.996 | self.command_object.sync(CONF.command.version) 2019-08-27 06:35:44.996 | File "/opt/stack/new/glance/glance/cmd/manage.py", line 166, in sync 2019-08-27 06:35:44.996 | self.migrate(online_migration=False) 2019-08-27 06:35:44.996 | File "/opt/stack/new/glance/glance/cmd/manage.py", line 294, in migrate 2019-08-27 06:35:44.996 | if data_migrations.has_pending_migrations(db_api.get_engine()): 2019-08-27 06:35:44.996 | File "/opt/stack/new/glance/glance/db/sqlalchemy/alembic_migrations/data_migrations/__init__.py", line 61, in has_pending_migrations 2019-08-27 06:35:44.997 | return any([x.has_migrations(engine) for x in migrations]) 2019-08-27 06:35:44.997 | File "/opt/stack/new/glance/glance/db/sqlalchemy/alembic_migrations/data_migrations/train_migrate01_backend_to_store.py", line 28, in has_migrations 2019-08-27 06:35:44.997 | metadata_backend = con.execute(sql_query) 2019-08-27 06:35:44.997 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 982, in execute 2019-08-27 06:35:44.997 | return self._execute_text(object_, multiparams, params) 2019-08-27 06:35:44.997 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1155, in _execute_text 2019-08-27 06:35:44.997 | parameters, 2019-08-27 06:35:44.997 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context 2019-08-27 06:35:44.997 | e, statement, parameters, cursor, context 2019-08-27 06:35:44.997 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1464, in _handle_dbapi_exception 2019-08-27 06:35:44.997 | util.raise_from_cause(newraise, exc_info) 2019-08-27 06:35:44.997 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause 2019-08-27 06:35:44.997 | reraise(type(exception), exception, tb=exc_tb, cause=cause) 2019-08-27 06:35:44.997 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context 2019-08-27 06:35:44.998 | cursor, statement, parameters, context 2019-08-27 06:35:44.998 | File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 552, in do_execute 2019-08-27 06:35:44.998 | cursor.execute(statement, parameters) 2019-08-27 06:35:44.998 | DBError: (psycopg2.errors.UndefinedFunction) function instr(text, unknown) does not exist 2019-08-27 06:35:44.998 | LINE 1: select meta_data from image_locations where INSTR(meta_data,... 2019-08-27 06:35:44.998 | ^ 2019-08-27 06:35:44.998 | HINT: No function matches the given name and argument types. You might need to add explicit type casts. Thanks, Nate [1] also http://paste.openstack.org/show/765665/ if that is an easier view [2] http://zuul.openstack.org/builds?project=openstack%2Fneutron&job_name=neutr… [3] https://review.opendev.org/#/q/project:openstack/glance+status:merged

2 1

Re: [ops] [glance] Need to move images to a different backend
by Massimo Sgaravatto 28 Aug '19

28 Aug '19

Hi Abhishek Yes, before trying the: glance location-add --url rbd://8162f291-00b6-4b40-a8b4-1981a8c09b64/images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628/snap 6bcc4eab-ed35-42dc-88bd-1d45de73b628 command, I uploaded the image to ceph. And indeed I can see it: [root@ceph-mon-01 ~]# rbd info images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628 rbd image '6bcc4eab-ed35-42dc-88bd-1d45de73b628': size 10GiB in 1280 objects order 23 (8MiB objects) block_name_prefix: rbd_data.b7b56aa1e4f0bc format: 2 features: layering flags: create_timestamp: Tue Aug 27 11:42:32 2019 [root@ceph-mon-01 ~]# rbd snap ls images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628 SNAPID NAME SIZE TIMESTAMP 455500 snap 10GiB Tue Aug 27 11:46:37 2019 [root@ceph-mon-01 ~]# Cheers, Massimo On Tue, Aug 27, 2019 at 2:41 PM Abhishek Kekane <akekane(a)redhat.com> wrote: > Hi Massimo, > > I need to reproduce this first, but pretty much sure this issue is not > because you configured multiple backends. What you are doing is downloading > existing image to your local storage and then uploading it to glance using > add-location operation. > > So before adding this using add-location operation have you manually > uploaded this image to ceph/rbd? if not then how you are building your > location url which you are mentioning in the add-location command? If this > location is not existing then it might be a problem. > > Thank you, > > Abhishek > > On Tue, 27 Aug 2019 at 5:09 PM, Massimo Sgaravatto < > massimo.sgaravatto(a)gmail.com> wrote: > >> I have a Rocky installation where glance is configured with multiple >> backends. This [*] is the relevant part of the glance configuration. >> >> I now need to dismiss the file backend and move the images-snapshots >> stored there to rbd. >> >> >> I had in mind to follow this procedure (already successfully tested with >> an older version of OpenStack): >> >> 1) download the image from the file backend (glance image-download --file >> <file> <imageid> >> 2) upload the file <file> to rbd, relying on this function: >> https://github.com/openstack/glance_store/blob/stable/rocky/glance_store/_d… >> 3) glance location-add --url rbd://xyz <imageid> >> 4) glance location-delete --url file://abc <imageid> >> >> I have problems with 3: >> >> # glance --debug location-add --url >> rbd://8162f291-00b6-4b40-a8b4-1981a8c09b64/images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628/snap >> 6bcc4eab-ed35-42dc-88bd-1d45de73b628 >> ... >> ... >> DEBUG:keystoneauth.session:PATCH call to image for >> https://cloud-areapd.pd.infn.it:9292/v2/images/6bcc4eab-ed35-42dc-88bd-1d45… >> used request id req-6c0598cc-582c-4ce7-a14c-5d1bb6ec4f14 >> Request returned failure status 400. >> DEBUG:glanceclient.common.http:Request returned failure status 400. >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/glanceclient/shell.py", line >> 687, in main >> OpenStackImagesShell().main(argv) >> File "/usr/lib/python2.7/site-packages/glanceclient/shell.py", line >> 591, in main >> args.func(client, args) >> File "/usr/lib/python2.7/site-packages/glanceclient/v2/shell.py", line >> 749, in do_location_add >> image = gc.images.add_location(args.id, args.url, metadata) >> File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line >> 448, in add_location >> response = self._send_image_update_request(image_id, add_patch) >> File "/usr/lib/python2.7/site-packages/glanceclient/common/utils.py", >> line 598, in inner >> return RequestIdProxy(wrapped(*args, **kwargs)) >> File "/usr/lib/python2.7/site-packages/glanceclient/v2/images.py", line >> 432, in _send_image_update_request >> data=json.dumps(patch_body)) >> File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line >> 340, in patch >> return self.request(url, 'PATCH', **kwargs) >> File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", >> line 377, in request >> return self._handle_response(resp) >> File "/usr/lib/python2.7/site-packages/glanceclient/common/http.py", >> line 126, in _handle_response >> raise exc.from_response(resp, resp.content) >> HTTPBadRequest: 400 Bad Request: Invalid location (HTTP 400) >> 400 Bad Request: Invalid location (HTTP 400) >> >> >> As far as I can see with the "openstack" client there is not something to >> add/delete a location. >> So I guess it is necessary to change the 'direct_url' and 'locations' >> properties. >> >> If I try to change the direct_url property: >> >> # openstack image set --property >> direct_url='rbd://8162f291-00b6-4b40-a8b4-1981a8c09b64/images-prod/6bcc4eab-ed35-42dc-88bd-1d45de73b628/snap' >> 6bcc4eab-ed35-42dc-88bd-1d45de73b628 >> 403 Forbidden: Attribute 'direct_url' is read-only. (HTTP 403) >> >> Any hints ? >> Thanks, Massimo >> >> [*] >> >> [default] >> ... >> enabled_backends = file:file,http:http,rbd:rbd >> show_image_direct_url = true >> show_multiple_locations = true >> >> [glance_store] >> default_backend = rbd >> >> [file] >> filesystem_store_datadir = /var/lib/glance/images/ >> >> >> [rbd] >> rbd_store_chunk_size = 8 >> rbd_store_ceph_conf = /etc/ceph/ceph.conf >> rbd_store_user = glance-prod >> rbd_store_pool = images-prod >> >> -- > Thanks & Best Regards, > > Abhishek Kekane >

2 3

Zuul log location changing
by corvus＠inaugust.com 28 Aug '19

28 Aug '19

Hi, We've been working for some time[1] to retire our current static log server in favor of storing Zuul job logs in Swift. We're just about ready to do that. This means that not only do we get to use another great OpenStack project, but we also can stop worrying about fscking our 14TB log partition whenever it gets hit with a comsic ray. The change will happen in two phases, only the first of which should be readily apparent. Phase 1: On Monday August 12, 2019 we will change the URL that Zuul reports back to Gerrit. Instead of being a direct link to the log server, it will be a link to the Zuul Build page for the job. This is part of Zuul's web interface which has been around for a while, but isn't well known since we haven't linked to it yet. The build page shows a summary of information about the build, including output snippets for failed tasks. Next to the "Summary" tab, you'll find the "Logs" tab. This contains an expandable index of all the log files uploaded for the build. If they are text files, they can be rendered in-app with line-number hyperlinks and severity filtering. There are also direct links to the files on the log server. Links to preview sites (e.g., for docs builds) will show up in the "Artifacts" section. We also plan to further enhance the build page with additional features. Here are some links to sample build pages so you can see what it's like: https://zuul.opendev.org/t/openstack/build/a6e13a8098fc4a1fbff43d8f2c27ad29 https://zuul.opendev.org/t/openstack/build/75d1e8d4ffaf477db00520d7bfd77246 This step is necessary because our static log server implements a number of features as WSGI middleware in Apache. We have re-implemented the these on the Zuul build page, so there should be no loss in functionality (in fact, we think this is an improvement). Once in place, we can change the backend storage options without impacting the user interface. Phase 2: Shortly afterwards (depending on how phase 1 goes), we will configure jobs to upload logs to Swift instead of the static log server. At this point, there should be no user-visible change, since the main interface for interacting with logs is now the Zuul build page. However, you may notice that log urls have changed from our static log server to one of six different Swift instances. The Swift instance used to store the logs for any given build is chosen at random from among our providers, and is yet another really cool multi-cloud feature we get by using OpenStack. Thanks to our amazing providers and all of the folks who have helped with this effort over the years[1]. Please let us know if you have any questions or encounter any issues, either here, or in #openstack-infra on IRC. -Jim [1] Years. So. Many. Years.

5 7

[placement][plt][elections] Non-nomination for U
by Chris Dent 28 Aug '19

28 Aug '19

tl;dr: I'd prefer not to be Placement PTL for the U cycle. I've accomplished most of what I set out to do and I think we have other members of the team who would make excellent PTLs (notably tetsuro and gibi). Longer version: Getting placement standing on its own feet, with a mostly reasonable architecture, reasonable performance, and a fairly complete set of features has pretty much been my single focus since the start of 2016. From both a personal and professional standpoint I need to focus some energy elsewhere, lest the deep ruts in my brain become permanent. I think it is important to not be shy about the goal of that three and a half year process: Get placement out of nova and get nova out of placement. The architectural differences necessary to make placement work well, be maintainable, and easily deployable [1] we're difficult to do within nova (for many reasons), but once extracted took a matter of days. Nova, neutron, blazar, zun, watcher, and cyborg all use, or are in the process of starting to use, placement. Ironic is thinking about it. So: some success. But it's not all gravy, there are still some things left to do: While the fundamental concept of placement (presenting and keeping track of inventory of resources) is pretty straightforward, using it most effectively can be complicated, especially if nested resource providers are involved. There's a lot of work to be done to document how to best model data and queries. As the number of resources in the cloud grows it becomes increasingly important to "ask better questions"; knowing how to do that shouldn't require 3 years of experience. The "consumer types" feature is in flight. It will allow placement to be used to keep track of resource usage for some types of quota, which makes good sense: If we're already tracking "user X is using N VCPU" in placement it's wasteful tracking it somewhere else as well. Placement sharding, so that one placement can track multiple domains, is a feature that we've discussed in the last couple of years, thinking perhaps it could be useful to edge setups. Until the recent performance improvements it wouldn't have worked well, but now it might. However, until real users present themselves with a need for this feature, there's not much point moving forward with it. We should continue to do benchmarking and profiling. That work has done wonders to reveal unexpected areas for improvement. Being willing and able to analyse and adjust the system and the code, continuously, is key to ensuring the project does not become moribund or calcified. Thanks to everyone who has helped get placement to a healthy place. [1] For some examples: We've taken a common benchmark from 17s to .5s. It's now possible to run placement without a config file or without a manual database sync. Every module has a maintainability index of "A". Placement doesn't use eventlet (and is not accidentally infected by it) so it can benefit from whatever threading or multi-processing model whatever chosen WSGI server provides. Placement is a simple web app over a (not so simple) persistence and query layer. The web app and the associated database server(s) can be scaled independently, up and out. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

3 2

[octavia] Stepping down from Octavia core team
by Nir Magnezi 28 Aug '19

28 Aug '19

Hi all, As some of you already know, I'm no longer involved in Octavia development and therefore stepping down from the core team. It has been a pleasure to be a part of the Octavia (and neutron-lbaas) core team for nearly three years. I learned a lot. Octavia is an excellent project with a strong core team, and I'm sure it will continue to grow and gain more adoption. Thanks, Nir

4 3

Re: [all][tc][horizon] A web tool which helps administrators in managing openstack clusters
by Adrian Turjak 28 Aug '19

28 Aug '19

Bah, sent it via my phone and didn't hit reply all. On 26/08/19 10:55 PM, Jim Rollenhagen wrote: > Fair points. In case you didn't realize, you only sent this to me, not > the list :) > > // jim > > > On Sat, Aug 24, 2019 at 1:45 AM Adrian Turjak <adriant(a)catalyst.net.nz > <mailto:adriant@catalyst.net.nz>> wrote: > > > > On 24 Aug. 2019 00:08, Jim Rollenhagen <jim(a)jimrollenhagen.com > <mailto:jim@jimrollenhagen.com>> wrote: > > On Fri, Aug 23, 2019 at 3:21 AM Adrian Turjak > <adriant(a)catalyst.net.nz <mailto:adriant@catalyst.net.nz>> wrote: > > Hello Douglas! > > As someone who has struggled against the performance > issues in Horizon I can easily feel your pain, and an > effort to making something better is good. Sadly, this > doesn't sound like a safe direction, even if for admin > only purposes. > > The first major issue is that you connect to the databases > of the services directly. That's a major issue, both for > long term compatibility, and security. The APIs should > always be the main point of contact and the ONLY contract > that the services have to maintain. By connecting to the > database directly you are now relying on a data structure > that can, and likely will change, and any security and > sanity checking on filters and queries is now handled on > your layer rather than the application itself. Not only > that, but your dashboard also now needs passwords for all > the databases, and by the sounds of it all the message queues. > > I would highly encourage you to try and work with the > community to fix the issues at the API layers rather than > bypassing them. We can add better query and filtering to > the APIs, and we can work on improving performance if we > know the pain points, and this is likely where > contributions would be welcome. > > I think we do need an alternative to Horizon, and the > ideal solution in my mind is to make a new dashboard built > on either React or Vue, with a thin proxy app (likely in > Flask) that serves the initial javascript page, and > proxies any API requests to the services themselves. The > filter issues should made better by implementing more > complex filtering in the APIs themselves, and having the > Dashboard layer better at exposing those dynamically. > React or Vue would do a much better job of dynamically and > quickly reloading and querying the services, and it would > make the whole experience much nicer. > > Michael Krotscheck did a bunch of work to put CORS config in > many API services. Why bother with a proxy when we can > continue that? :) > > > We deploy our dashboard publically, but our APIs are behind a > whitelist only firewall. I bet we're not the only one. So we need > the proxy like nature of the dashboard API calls. > > I personally like the idea of the browser able to talk to the APIs > directly, but I have a feeling there may be other reasons to avoid > that. Not to mention you get a clearer picture of which API calls > come from the dashboard. Plus, a proxy via the dashboard app > itself (assuming it is deployed near the cluster) won't exactly > add much extra to the response time. > > But if we as a community do end up going down this route, we can > discuss if we need the app to proxy or not. If not, then really we > are just building a pure JavaScript app, but if we do need to > proxy, then the proxy layer would still be very simple with the > core logic all in the front-end. > > > The one of the best parts of Horizon was that it was a > 'dumb' dashboard built around your token. It can be > deployed anywhere, by anyone, and only needs access to the > cluster to work, no secrets to any database. > > I know this is a huge issue, and we do need to solve it, > but I hope we can work on something better that doesn't > bypass the APIs, because that isn't a safe solution. :( > > Cheers, > > Adrian Turjak > > On 20/08/19 10:14 PM, Douglas Zhang wrote: > > Hello everyone, > > To help users interact with openstack, we’re currently > developing a client-side web tool which enables > administrators to manage their openstack cluster in a > more efficient and convenient way. (Since we have not > named it officially yet, I’m going to call it > openstack-admin) > > *# Introduction* > > Some may ask, “Why do we need an extra web-based user > interface since we have horizon?” Well, although > horizon is a mature and powerful dashboard, it is far > not efficient enough on big clusters (a simple |list| > operation could take seconds to complete). What’s > more, its flexibility of searching could not match our > requirements. To overcome obstacles above, a more > efficient tool is urgently required, that’s why we > started to develop openstack-admin. > > *# Highlights* > > Comparing with the current user interface, > openstack-admin has following advantages: > > * > > *Fast*: openstack-admin gets data straightly from > SQL databases instead of calling standard > openstack API, which accelerates the querying > period to a large extent (especially when we’re > dealing with a large amount of data). > > * > > *Flexible*: openstack-admin supports the fuzzy > search for any important field(e.g. > display_name/uuid/ip_address/project_name of an > instance), which enables users to locate a > particular object in no time. > > * > > *User-friendly*: the backend of openstack-admin > gets necessary messages from the message queue > used by nova, and send them to the frontend using > websocket. This way, not only more realistic > progress bars could be implemented, but more > detailed information could be provided to users as > well. > > *# Issues* > > To make this tool more efficient and provide better > support for concurrency, we chose Golang to implement > openstack-admin. As I’ve asked before (truly > appreciate advises from Jeremy and Ghanshyam), a > project written by an unofficial language may be > accepted only if existing languages have been proven > to not meet the technical requirements, so we’re > considering re-implementing openstack-admin using > python if we can’t come to an agreement on the > language issue. > > So that’s all. How do you guys think of this project? > > Thanks, > > Douglas Zhang > >

1 0

Re: Re: [all][tc][horizon] A web tool which helps administrators in managing openstack clusters
by Douglas Zhang 28 Aug '19

28 Aug '19

Hello everyone, Thanks for your attention and advice for this project. We have read each reply thoroughly and the good news is that we’re able to give answers to some questions raised by them. As Lingxian Kong said: I have a few questions/suggestions: 1. It'd be great and gain more attractions if you could provide a demo about how "openstack-admin" looks like 2. What OpenStack services has "openstack-admin" already integrated? Is it easy to integrate with others? 1. We have deployed openstack-admin <http://218.205.220.13:9384/login> on a mini openstack cluster as a demo, any access would be welcomed. - Username: openstack-admin - Password: @Dem0 2. Since openstack-admin gets information it needs by querying the sql database, it’s fairly easy to integrate with all openstack services. - As the demo shows, openstack-admin has integrated *nova*(almost all GET and part of POST), *cinder*(GET and snapshot-creation), *neutron*(subnets and ports), *keystone*(projects) and *glance*(images), that’s all we need in our own working environment. - If we need to integrate more services to openstack-admin(e.g. adding a create instance button or integrating with swift), it would not be a complex task, either. As Adrian Turjak said: The first major issue is that you connect to the databases of the services directly. That's a major issue, both for long term compatibility, and security. The APIs should always be the main point of contact and the ONLY contract that the services have to maintain. By connecting to the database directly you are now relying on a data structure that can, and likely will change, and any security and sanity checking on filters and queries is now handled on your layer rather than the application itself. Not only that, but your dashboard also now needs passwords for all the databases, and by the sounds of it all the message queues. And as Mohammed Naser said: While I agree with you that querying database is much faster, this introduces two issues that I imagine for users: - Dashboards generally having direct access via SQL is a scary thing from an operators perspective - also, it will make maintaining the project quite hard because I don't think any projects expose a *stable* database API. Well, we’re not surprised that our querying approach would be challenged since it does sound unsafe. However, we have made some efforts to solve problems which have been posed: - We use a ORM library to create all queries, which ensures that only those instructions we have specified in the backend(i.e. select, order by, where and other harmless querying instructions) could be executed, protecting our databases from dangerous attacks like SQL injection. All sanity or security checkings would be automatically completed by those library functions. - All instructions that *may* change the database(e.g. start, shutoff, migrate) would be executed by calling standard openstack API, only pure GET instructions were implemented by querying databases directly. We have wrapped each API call with a go func() { ... }() to avoid the extremely long calling period. The results of API calls would be sent back to frontend by websocket asynchronously. - Passwords of databases and message queues(and many other kinds of information) are stored in a config file which would be loaded by openstack-admin. Simply by modifying this file, we could be consistent with all changes about sql databases and MQs. I hope my explanation is clear enough, and we’re willing to solve other possible issues existing. Cheers, Douglas Zhang

4 3

[Watcher] team meeting at 08:00 UTC today
by li.canwei2＠zte.com.cn 28 Aug '19

28 Aug '19

Hi, Watcher team will have a meeting at 08:00 UTC today in the #openstack-meeting-alt channel. The agenda is available on https://wiki.openstack.org/wiki/Watcher_Meeting_Agenda feel free to add any additional items. Thanks! Canwei Li

1 0

[horizon] Horizon's openstack_auth is completely broken with Django 2.2, breaking all the dashboad stack in Debian Sid/Bullseye
by Thomas Goirand 28 Aug '19

28 Aug '19

Hi, Django 2.2 was uploaded to Debian Sid/Bullseye a few days after Buster was released, removing at the same time any support for Python 2. This broke lots of packages, but I managed to upload more than 40 times to fix the situation. Though Horizon is still completely broken. There's a bunch of issues related to Django 2.2 for Horizon. I managed to patch Horizon for a few of them, but there's one bigger issue which I can't solve myself, because I'm not skilled enough in Django, and don't know enough about Horizon's internal. Let me explain. In Django 1.11, the login() function in openstack_auth/views.py was deprecated in the favor of a LoginView class. In Django 2.2, the login() function is now not supported anymore. This means that, if openstack_auth/views.py doesn't get rewritten completely, Horizon will continue to be completely broken in Debian Sid/Bullseye. Unfortunately, the Horizon team is understaffed, and the PTL told me that they don't plan anything before Train+1. As a consequence, Horizon is completely broken in Debian Sid/Bullseye, and will stay as it is if nobody steps up for the work. After a month and a half with this situation not being solved by anyone, I'm hereby calling for help. I could attempt to work on this, though I need help and pointers at some example implementation of this. It'd be nicer if someone more skilled than me in Django was working on this anyways. Cheers, Thomas Goirand (zigo)

3 4

[designate][ptl][elections] Non Candidacy for U cycle
by Graham Hayes 28 Aug '19

28 Aug '19

Hi All, As I laid out in my Train nomination[1], Train was my last cycle as PTL for Designate. Change is good for projects, and Designate is well overdue for it.[2] The 4 years (with a 6 month break[3]), 4 or 5 different companies, and 2 API versions were a real experience - we did some great work as a team. As a project Designate has come a long way from the start, and team over the years really deserves congratulations. From something that was an outsider project in Folsem and Grizzly, to having a Trademark program now, we are in completely different space, and that is a testament to the wide group of people that have contributed (in terms of code, but also in running it, reporting bugs, helping us with docs, and translating it into many different languages) The project is used by some of the largest installs of OpenStack, and provides really useful service for cloud environments. We do have some work left to do, and as "cloud native" evolves, Designate will as well. I am not going to disappear - I still will be around to work on the project, explain why we did somethings that look weird[4], and help people set it up, but just not as a PTL. If anyone has any questions on what the role entails, or are interested but want some more info, please reach out, via email, mugsie on freenode, or on the #openstack-dns channel Thanks everyone, - Graham 1 - In 7 years, we have had 3 PTLs, and one of them was for a single cycle 2 - https://opendev.org/openstack/election/raw/branch/master/candidates/train/D… 3 - thanks timsim for giving me a break :) 4 - Most of the time, its my fault

2 1

[all][elections][ptl][tc] Combined PTL/TC Nominations Kickoff
by Jeremy Stanley 28 Aug '19

28 Aug '19

Nominations for OpenStack PTLs (Project Team Leads) and TC (Technical Committee) positions (6 positions) are now open and will remain open until Sep 03, 2019 23:45 UTC. All nominations must be submitted as a text file to the openstack/election repository as explained at https://governance.openstack.org/election/#how-to-submit-a-candidacy Please make sure to follow the candidacy file naming convention: candidates/u/<Project_Name or "TC">/<email_address> (for example, "candidates/u/TC/stacker(a)example.org"). The name of the file should match an email address for your current OpenStack Foundation Individual Membership. Take this opportunity to ensure that your OSF member profile contains current information: https://www.openstack.org/profile/ Any OpenStack Foundation Individual Member can propose their candidacy for an available, directly-elected seat on the Technical Committee. In order to be an eligible candidate for PTL you must be an OpenStack Foundation Individual Member. PTL candidates must also have contributed to the corresponding team during the Stein to Train timeframe, Aug 10, 2018 00:00 UTC - Sep 03, 2019 00:00 UTC. Your Gerrit account must also have a verified email address matching the one used in your candidacy filename. Both PTL and TC elections will be held from Sep 10, 2019 23:45 UTC through to Sep 17, 2019 23:45 UTC. The electorate for the TC election are the OpenStack Foundation Individual Members who have a code contribution one of the official teams over the Stein to Train timeframe, Aug 10, 2018 00:00 UTC - Sep 03, 2019 00:00 UTC, as well as any Extra ATCs who are acknowledged by the TC. The electorate for a PTL election are the OpenStack Foundation Individual Members who have a code contribution over the Stein to Train timeframe, Aug 10, 2018 00:00 UTC - Sep 03, 2019 00:00 UTC, in a deliverable repository maintained by the team for which the PTL would lead, as well as the Extra ATCs who are acknowledged by the TC for that specific team. The list of project teams can be found at https://governance.openstack.org/tc/reference/projects/ and their individual team pages include lists of corresponding Extra ATCs. Please find below the timeline: nomination starts @ Aug 27, 2019 23:45 UTC nomination ends @ Sep 03, 2019 23:45 UTC campaigning starts @ Sep 03, 2019 23:45 UTC campaigning ends @ Sep 10, 2019 23:45 UTC elections start @ Sep 10, 2019 23:45 UTC elections end @ Sep 17, 2019 23:45 UTC Shortly after election officials approve candidates, they will be listed on the https://governance.openstack.org/election/ page. The electorate is requested to confirm their email addresses in Gerrit, prior to 2019-09-03 00:00:00+00:00 so that the emailed ballots are mailed to the correct email address. This email address should match one which was provided in your foundation member profile as well. Gerrit account information and OSF member profiles can be checked at https://review.openstack.org/#/settings/contact and https://www.openstack.org/profile/ accordingly. If you have any questions please be sure to either ask them on the mailing list or to the elections officials: https://governance.openstack.org/election/#election-officials -- Jeremy Stanley, on behalf of the technical election officials

1 0

[all[ Please Do Not Translate Etherpads ...
by Jay Bryant 28 Aug '19

28 Aug '19

All, Not sure how else to address this, but we have had Cinder's etherpads repeatedly converted into Japanese recently. Want to make a public plea here to please be careful with translation tools and ensure that you aren't changing the etherpads in the process. Rebuilding the content to its previous state is tedious. Thank you! Jay (irc: jungleboyj)

7 12

[cinder] Mid-Cycle Summary and Video Posted ...
by Jay Bryant 27 Aug '19

27 Aug '19

All, I have gotten the Summary from the Cinder Mid-Cycle that we held at Lenovo's RTP campus last week posted. [1] Please take a look at the summary and see if you have any action items. Also note that I have tagged actions without an owner with the 'HELP NEEDED' tag. If you can take any of the action items it would be greatly appreciated. Thanks to all who participated and made it another successful mid-cycle! Jay (irc: jungleboyj) [1] https://wiki.openstack.org/wiki/CinderTrainMidCycleSummary

1 0

[oslo] No meeting next week, Sep. 2
by Ben Nemec 27 Aug '19

27 Aug '19

Hi, Next Monday is a US holiday and as a result I won't be available to run the meeting. Since that affects a number of our other Oslo contributors, I propose that we just skip the meeting for that week. It's the first day after feature freeze anyway so there shouldn't be _too_ much to discuss. As always, if anything comes up in the meantime you can bring it up in #openstack-oslo or here on the list. No need to wait for the meeting. Thanks. -Ben

1 0

[tripleo] PTL non-candidacy
by Alex Schultz 27 Aug '19

27 Aug '19

Hey folks, The PTL election cycle is upon us. I will not be running again for the TripleO PTL. I think this last cycle has been fairly successful from a stability stand point. We've continued to improve the deployment process and address a decent amount of tech debt. I will still be around in the community, but I think it's time for others to step up. If anyone has any questions about the position, feel free to reach out to me. Thanks, -Alex

1 0

[all][api-sig] Update on SIG status and seeking guidance from the community
by Michael McCune 27 Aug '19

27 Aug '19

hello all, I am writing on behalf of the API-SIG to update the community on our status. We have recently learned that Ed Leafe will be leaving the membership of the SIG as his new working circumstances are pulling him away from this work. This will leave us with only 2 active members, Dmitry Tantsur and myself. Firstly, I want to extend a heartfelt thanks to Ed. thank you =) I believe that Ed is the longest standing member of the SIG, having been active from the days when it was just a smol working group. His contributions have been numerous over the years and his guidance and wisdom will be missed. Second, this change in our membership has given rise to some questions about the future of the SIG and how we might best serve the OpenStack community. We currently hold an office hour each week and see relatively low traffic during this hour. Our activities have mainly been focused on writing and maintaining the guidelines and the support structure around those guidelines. As most of the new writing has ended (there are a few still in flight), we have been paying most of our attention to maintenance and supporting the community. As our ability to maintain an active membership has diminished, and many of our initial goals met, we have started to question whether we should still maintain the "SIG" status and also how we might best serve the community going forward. We will still need core members to review and approve changes, but perhaps we should adjust our workflow to be of greater service. I don't have any answers to this more existential question about the SIG, but we as a group felt it was a good idea to reach out to the wider community for advice and guidance. So, what do you all have to say? What would you like to see from the API-SIG moving forward? Should we re-evaluate the purpose of this group and how it relates to the OpenStack community? thank you all for the wonderful spirit of openness that we build with, and thanks for listening. and lastly, thanks again to Ed. You have acted as an example of the best aspects of open collaboration. peace o/

5 8

Getting the power state
by Mauricio Tavares 27 Aug '19

27 Aug '19

2 1