[openstack-dev] [nova] config options help text improvement: current status

Matt Riedemann mriedem at linux.vnet.ibm.com
Thu Mar 3 02:50:18 UTC 2016



On 3/2/2016 11:45 AM, Markus Zoeller wrote:
> TL;DR: From ~600 nova specific config options are:
>              ~140 at a central location with an improved help text
>              ~220 options in open reviews (currently on hold)
>              ~240 options todo
>
>
> Background
> ==========
> Nova has a lot of config options. Most of them weren't well
> documented and without looking in the code you probably don't
> understand what they do. That's fine for us developers but the ops
> had more problems with the interface we provide for them [1]. After
> the Mitaka summit we came to the conclusion that this should be
> improved, which is currently in progress with blueprint [2].
>
>
> Current Status
> ==============
> After asking on the ML for help [3] the progress improved a lot.
> The goal is clear now and we know how to achieve it. The organization
> is done via [4] which also has a section of "odd config options".
> This section is important for a later step when we want do deprecate
> config options to get rid of unnecessary ones.
>
> As we reached the Mitaka-3 milestone we decided to put the effort [5]
> on hold to stabilize the project and focus the review effort on bug
> fixes. When the Newton cycle opens, we can continue the work. The
> current result can be seen in the sample "nova.conf" file generated
> after each commit [6]. The appendix at the end of this post shows an
> example.
>
> All options we have will be treated that way and moved to a central
> location at "nova/conf/". That's the central location which hosts
> now the interface to the ops. It's easier to get an overview now.
> The appendix shows how the config options were spread at the beginning
> and how they are located now.
>
> I initially thought that we have around 800 config options in Nova
> but I learned meanwhile that we import a lot from other libs, for
> example from "oslo.db" and expose them as Nova options. We have around
> 600 Nova specific config options, and ~140 are already treaded like
> described above and ca. 220 are in the pipeline of open reviews.
> Which leaves us ~240 which are not looked at yet.
>
>
> Outlook
> =======
> The numbers of the beginning of this ML post make me believe that we
> can finish the work in the upcoming Newton cycle. "Finished" means
> here:
> * all config options we provide to our ops have proper and usable docs
> * we have an understanding which options don't make sense anymore
> * we know which options should get stronger validation to reduce errors
>
> I'm looking forward to it :)
>
>
> Thanks
> ======
> I'd like to thank all the people who are working on this and making
> this possible. A special thanks goes to Ed Leafe, Esra Celik and
> Stephen Finucane. They put a tremendous amount of work in it.
>
>
> References:
> ===========
> [1]
> http://lists.openstack.org/pipermail/openstack-operators/2016-January/009301.html
> [2] https://blueprints.launchpad.net/nova/+spec/centralize-config-options
> [3]
> http://lists.openstack.org/pipermail/openstack-dev/2015-December/081271.html
> [4] https://etherpad.openstack.org/p/config-options
> [5] Gerrit reviews for this topic:
> https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/centralize-config-options
> [6] The sample config file which gets generated after each commit:
>      http://docs.openstack.org/developer/nova/sample_config.html
>
>
> Appendix
> ========
>
> Example of the help text improvement
> -----------------------------------
> As an example, compare the previous documentation of the scheduler
> option "scheduler_tracks_instance_changes".
> Before we started:
>
>      # Determines if the Scheduler tracks changes to instances to help
>      # with its filtering decisions. (boolean value)
>      #scheduler_tracks_instance_changes = true
>
> After the improvement:
>
>      # The scheduler may need information about the instances on a host
>      # in order to evaluate its filters and weighers. The most common
>      # need for this information is for the (anti-)affinity filters,
>      # which need to choose a host based on the instances already running
>      # on a host.
>      #
>      # If the configured filters and weighers do not need this information,
>      # disabling this option will improve performance. It may also be
>      # disabled when the tracking overhead proves too heavy, although
>      # this will cause classes requiring host usage data to query the
>      # database on each request instead.
>      #
>      # This option is only used by the FilterScheduler and its subclasses;
>      # if you use a different scheduler, this option has no effect.
>      #
>      # * Services that use this:
>      #
>      #     ``nova-scheduler``
>      #
>      # * Related options:
>      #
>      #     None
>      #  (boolean value)
>      #scheduler_tracks_instance_changes = true
>
>
> The spread of config options in the tree
> ----------------------------------------
> We started with this in November 2015. It's the Nova project tree and
> the numbers behind the package name are the numbers of config options
> declared in that package (config options declared in sub-packages are
> not accumulated).
>
>      Based on:
>      commit 201090b0bcb
>      Date: Thu Nov 19 04:08:51 2015 +0000
>
>       nova [87]
>      --- CA [0]
>      ------ newcerts [0]
>      ------ private [0]
>      ------ projects [0]
>      ------ reqs [0]
>      --- api [3]
>      ------ ec2 [14]
>      ------ metadata [6]
>      ------ openstack [6]
>      --------- compute [2]
>      ------------ legacy_v2 [2]
>      --------------- contrib [6]
>      ------------ schemas [0]
>      ------------ views [0]
>      ------ validation [0]
>      --- cells [25]
>      ------ filters [0]
>      ------ weights [3]
>      --- cert [2]
>      --- cloudpipe [6]
>      --- cmd [15]
>      --- compute [49]
>      ------ monitors [2]
>      --------- cpu [0]
>      ------ resources [0]
>      --- conductor [5]
>      ------ tasks [1]
>      --- conf [0]
>      --- console [16]
>      --- consoleauth [3]
>      --- db [4]
>      ------ sqlalchemy [13]
>      --------- api_migrations [0]
>      ------------ migrate_repo [0]
>      --------------- versions [0]
>      --------- migrate_repo [0]
>      ------------ versions [0]
>      --- hacking [0]
>      --- image [14]
>      ------ download [3]
>      --- ipv6 [1]
>      --- keymgr [5]
>      --- mks [2]
>      --- network [68]
>      ------ neutronv2 [14]
>      ------ security_group [1]
>      --- objects [2]
>      --- objectstore [3]
>      --- openstack [0]
>      ------ common [4]
>      --- pci [2]
>      --- rdp [2]
>      --- scheduler [14]
>      ------ client [0]
>      ------ filters [15]
>      ------ weights [6]
>      --- servicegroup [1]
>      ------ drivers [4]
>      --- spice [6]
>      --- virt [18]
>      ------ disk [2]
>      --------- mount [1]
>      --------- vfs [1]
>      ------ hyperv [16]
>      ------ image [0]
>      ------ ironic [10]
>      ------ libvirt [45]
>      --------- storage [2]
>      --------- volume [19]
>      ------ vmwareapi [24]
>      ------ xenapi [36]
>      --------- client [2]
>      --------- image [8]
>      --- vnc [8]
>      --- volume [8]
>      ------ encryptors [0]
>      --- wsgi [0]
>      ----------------------------
>      Number of total options: 637
>
>
> As of now, we have these numbers (please note the "nova/conf/" folder)
>
>      Based on:
>      commit 040df06bfc4
>      Date: Wed Mar 2 11:09:24 2016 +0000
>
>       nova [72]
>      --- CA [0]
>      ------ newcerts [0]
>      ------ private [0]
>      ------ projects [0]
>      ------ reqs [0]
>      --- api [3]
>      ------ ec2 [0]
>      ------ metadata [6]
>      ------ openstack [7]
>      --------- compute [2]
>      ------------ legacy_v2 [2]
>      --------------- contrib [6]
>      ------------ schemas [0]
>      ------------ views [0]
>      ------ validation [0]
>      --- cells [0]
>      ------ filters [0]
>      ------ weights [0]
>      --- cert [0]
>      --- cloudpipe [6]
>      --- cmd [11]
>      --- compute [41]
>      ------ monitors [2]
>      --------- cpu [0]
>      ------ resources [0]
>      --- conductor [1]
>      ------ tasks [1]
>      --- conf [136]
>      --- console [11]
>      --- consoleauth [3]
>      --- db [4]
>      ------ sqlalchemy [13]
>      --------- api_migrations [0]
>      ------------ migrate_repo [0]
>      --------------- versions [0]
>      --------- migrate_repo [0]
>      ------------ versions [0]
>      --- hacking [0]
>      --- image [15]
>      ------ download [3]
>      --- ipv6 [1]
>      --- keymgr [5]
>      --- mks [2]
>      --- network [68]
>      ------ neutronv2 [7]
>      ------ security_group [1]
>      --- objects [2]
>      --- openstack [0]
>      ------ common [0]
>      --- pci [0]
>      --- rdp [2]
>      --- scheduler [0]
>      ------ client [0]
>      ------ filters [0]
>      ------ weights [0]
>      --- servicegroup [1]
>      ------ drivers [0]
>      --- spice [6]
>      --- virt [7]
>      ------ disk [0]
>      --------- mount [0]
>      --------- vfs [1]
>      ------ hyperv [15]
>      ------ image [0]
>      ------ ironic [0]
>      ------ libvirt [48]
>      --------- storage [2]
>      --------- volume [19]
>      ------ vmwareapi [24]
>      ------ xenapi [36]
>      --------- client [2]
>      --------- image [8]
>      --- vnc [0]
>      --- volume [8]
>      ------ encryptors [0]
>      --- wsgi [0]
>      ----------------------------
>      Number of total options: 610
>
>
> Regards, Markus Zoeller (markus_z)
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Thanks, this looks a lot better.

On a side note, can we get someone to herd the cats around this bug?

https://bugs.launchpad.net/nova/+bug/1532228

There are a few duplicates and at least 3 patches up for the same thing, 
but all three are incomplete. Basically all of the 
nova.virt.libvirt.volume.* options are missing from the generated sample 
config. It would be nice if someone drove those 3 patches into one and 
resolved all of the missing libvirt volume options rather than a couple 
here and there.

-- 

Thanks,

Matt Riedemann




More information about the OpenStack-dev mailing list