Open Stack

I don't like how this discussion is framed. The initial premise that we have
only two controversial options to choose from is lazy. If there is no
consensus, we should look for more options, not for a popular vote.

Secondly, current level of UX is not negotiable. You can't take something that
we already have and that works (and current Fuel UI is the best out there by a
wide margin), and make it worse just to add a new feature. Even something as
important as plugins must be an incremental improvement.

With that premise, lets decompose the problem.

1. There are two levels of settings related to any plugin: a) first you have to
enable enable the plugin itself; b) when the plugin is enabled, it may expose
additional settings.

- How can it be acceptable to have all plugins always enabled in all
  environments? Do you really trust all plugin writers to carefully check for
  plugin-specific options and ensure there is zero impact on an environment if
  none of its options are enabled?

- If all your plugins are enabled everywhere, you can't uninstall any of them
  because all environments you deployed would become unmanageable.

- If you ignore uninstallation, soon you will be stuck with plugins that cannot
  be made removable even when Fuel itself begins to support it.

- To break away from unremovable plugins, you're likely going to have to break
  backwards compatibility (unless you already have a forward-compatible design
  that allows for removable plugins in the future, but then you wouldn't have
  to exclude removing plugins from MVP).

- And if a Fuel upgrade ever requires uninstalling a plugin due to
  irreconcilable incompatibility, and they're enabled in all of your
  environments, you're stuck unable to upgrade.

So, lets not enable any plugins by default. And if we can come up with a way to
make them removable (when they're not enabled in any deployed environments), we
should at least leave room for that in the design.

2. Either level of plugin settings (enable or configure) may be exposed in
setup wizard, settings page, or both.

- Yes, additional plugin settings also may show up in the wizard (e.g. vCenter
  credentials).

- Yes, we should maintain the settings page as the SSoT, and that means
  reflecting as many of setup wizard options in it as possible.

- Yes, for some options (like choice of operating system or network topology),
  our settings page is not dynamic enough to allow user to go back and revert
  them without recreating the environment.

- No, plugin writer shouldn't have to explicitly describe a checkbox to enable
  their plugin. They only should provide name and description of the plugin.
  Plugin engine should be able to produce a catalogue of installed plugins, and
  UI should generate enable checkboxes from that catalogue.

- If a plugin doesn't affect any available environment configuration options
  outside of the settings tab (i.e. setup wizard, network settings, node roles,
  disk & nic configuration), there's no reason to limit it to setup wizard, the
  "enable" checkbox and whatever other options it has should all be present in
  the settings page.

- Do we have any plugins in 6.0 that have to be present in setup wizard because
  they affect UI outside of settings page? I'm not aware of any.

If so, lets start by representing all plugin settings in the settings page. But
leave the room in the metadata to force some or all of plugin's settings to
show up in the setup wizard (or even to present plugin configuration options
differently in the wizard than in the settings).

Just my $2,
-DmitryB

On Wed, Oct 8, 2014 at 9:18 AM, Nikolay Markov <nmarkov at mirantis.com> wrote:
> Vitaly,
>
> Once again, as a plugin developer I don't care about how Sahara or Murano is
> implemented. I don't care about checkboxes, either. I just want one simple
> command to run on target nodes and I should be provided with the simplest
> possible interface to:
> 1) Write this command in some YAML and don't care about anything else
> 2) Enable my plugin for particular environment and see if it's really
> enabled both on UI and CLI (and through pure API by simple field checking)
>
> If it provides some separate service - this doesn't change anything, I just
> need it to be listed somewhere in "plugins" inside cluster data to know that
> it'll be executed.
>
> How will it work with your approach?
>
> 08 Окт 2014 г. 20:00 пользователь "Vitaly Kramskikh"
> <vkramskikh at mirantis.com> написал:
>
>> Hi, responses inline.
>>
>> 2014-10-08 21:09 GMT+07:00 Evgeniy L <eli at mirantis.com>:
>>>
>>> Hi,
>>>
>>> Vitaly, I understand your concerns about UX.
>>> But there are technical problems and statements which affect
>>> plugin developer and makes his live more complicated.
>>>
>>> My opinion is we definitely should know, if plugin is disabled
>>> or if it's enabled for specific environment.
>>>
>>> 1. plugin developer defines tasks, like "I want the script to be
>>>     executed on nodes with controller role" and I don't think that
>>>     he expects this task to be run on all of the nodes, also
>>>     I don't think that we want ask plugin developer to parse
>>>     yaml with bash in order to understand if his plugin is enabled,
>>>     it's a bad design
>>
>> Bash script shouldn't be even run if the conditions to run it are not met.
>> I described above how it could be done.
>>>
>>> 2. there will be no way to uninstall the plugin, because all of the
>>>     plugins are enabled on the environments, even if user doesn't
>>>     use them
>>
>> Well, this is the only issue that I see with the first approach and I
>> still don't know how to solve it.
>>>
>>> Also I don't think that it's a good interface, to ask plugin developr
>>> to include checkbox in each plugin.
>>>
>> It should be included only in plugins which affect the installation. For
>> example, if OSTF was a plugin it wouldn't need such a checkbox. We can also
>> make kind of plugin bootstrap or a sample plugin whcih will include a single
>> control.
>>>
>>> The second solution is discussed because it's the most explicit
>>> way to solve described problem.
>>>
>>> Let's try to figure out the solution which will work well for user
>>> and for plugin developer.
>>>
>>> For example, for each plugin generate section on UI with checkbox
>>> Like:
>>
>> Well, first Nikolay disliked need for a checkbox for any plugin and now
>> you want to autogenerate a section. Why woudn't we give plugin writers
>> ability to describe the controls themselves? For example, LBaaS would
>> require a single checkbox in "Additional Services" section.
>>>
>>>
>>> GlusterFS [ ] - disable all of the fields for the section
>>> Here is some description of the section, which we can take from
>>> description field.
>>>
>>> IP address [127.0.0.1] - this field provides plugin developer
>>>
>>> If plugin is set, we add env <-> plugin relation, if it's unset, we
>>> delete it.
>>> Also when user checks the checkbox, UI will be able to retrieve
>>> attributes which plugin provides. But it's not so easy todo, I'm not
>>> sure if we can do it with hooks, but it's possible with some separate
>>> model and handlers.
>>>
>>> Thanks,
>>>
>>> On Wed, Oct 8, 2014 at 12:48 PM, Vitaly Kramskikh
>>> <vkramskikh at mirantis.com> wrote:
>>>>
>>>> Nikolay,
>>>>
>>>> Currently every thing that can be turned into a plugin (Ceph, vCenter,
>>>> Sahara, Murano, Ceilometer) provides a checkbox (or more complicated
>>>> controls) for the settings tab. Why change this approach for plugins? The
>>>> settings tab (cluster attributes) currently is a SSOT, and you want to ruin
>>>> it for no reason.
>>>>
>>>> Of course it makes no sense to generate anything. Checkboxes on the
>>>> settings tab can be added using simple YAML mixin and if you want to check
>>>> this checkbox to determine whether to perform some action or not and don't
>>>> want to write any python code, try to add to plugin's YAML "restrictions"
>>>> section which we already have for the settings tab, the wizard and roles.
>>>>
>>>> 2014-10-08 14:50 GMT+07:00 Nikolay Markov <nmarkov at mirantis.com>:
>>>>>
>>>>> >>> Right now we already have like 2 types of plugins (extensions),
>>>>> >>> classified by usage of settings tab:
>>>>> >>> 1. Some kind of backend for service (swift/ceph, lvm/ceph,
>>>>> >>> ovs/nsx), or hypervisor (lvm/qemu/vmware)
>>>>> >>> 2. Self-contained service that just needs to be installed (sahara,
>>>>> >>> murano, zabbix)
>>>>>
>>>>> That's not quite right. In 6.0 and after that there will be a lot of
>>>>> small plugins which only modify some config and/or install some
>>>>> package. There is nothing to configure here, and I as a plugin
>>>>> developer don't even want to know anything about checkboxes on UI. I
>>>>> just want two things: role to execute my command on and command
>>>>> itself. That's one small YAML.
>>>>>
>>>>> And autogenerating checkboxes for such plugins on UI is bad, because
>>>>> explicit is better than implicit (and all our settings are explicitly
>>>>> defined in openstack.yaml).
>>>>>
>>>>> On Wed, Oct 8, 2014 at 11:43 AM, Dmitriy Shulyak
>>>>> <dshulyak at mirantis.com> wrote:
>>>>> > If there is no checkboxes (read configuration) and plugin is
>>>>> > installed - all
>>>>> > deployment tasks will be applied
>>>>> > to every environment, but why do you think that there will be no
>>>>> > checkboxes
>>>>> > in most cases?
>>>>> >
>>>>> > Right now we already have like 2 types of plugins (extensions),
>>>>> > classified
>>>>> > by usage of settings tab:
>>>>> > 1. Some kind of backend for service (swift/ceph, lvm/ceph, ovs/nsx),
>>>>> > or
>>>>> > hypervisor (lvm/qemu/vmware)
>>>>> > 2. Self-contained service that just needs to be installed (sahara,
>>>>> > murano,
>>>>> > zabbix)
>>>>> >
>>>>> > In 1st case you need to provide shared configuration storage (like
>>>>> > cluster
>>>>> > attributes right now), in order for plugin
>>>>> > to be able to exclude part of core workflow from running (not
>>>>> > installing
>>>>> > swift for example).
>>>>> > In case if the plugin is self-contained entity, like Sahara, Murano
>>>>> > right
>>>>> > now - checkboxes would be simply required.
>>>>> > It works this way right now, and it doesnot look like huge overhead.
>>>>> >
>>>>> > So what do you think, will it work or no?
>>>>> >
>>>>> > On Wed, Oct 8, 2014 at 8:42 AM, Nikolay Markov <nmarkov at mirantis.com>
>>>>> > wrote:
>>>>> >>
>>>>> >> Hi,
>>>>> >>
>>>>> >> Frankly speaking, I'm not sure on how 1st approach will even work.
>>>>> >> What if plugin doesn't provide any checkboxes (and in most cases it
>>>>> >> won't)? How should we determine in serializer, which plugins should
>>>>> >> be
>>>>> >> applied while generating astute.yaml and tasks.yaml? Should we
>>>>> >> autogenerate some stuff for plugins which are not even enabled and
>>>>> >> do
>>>>> >> needless work?
>>>>> >>
>>>>> >> This looks too complicated for me from the backend side, and option
>>>>> >> with enabling/disabling plugins in wizard for specific environment
>>>>> >> (we
>>>>> >> can invent mechanism to disable them on env which is not deployed
>>>>> >> yet,
>>>>> >> besides, for API it's just one PUT) is MUCH simpler and much more
>>>>> >> obvious, as I see.
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Wed, Oct 8, 2014 at 8:34 AM, Vitaly Kramskikh
>>>>> >> <vkramskikh at mirantis.com> wrote:
>>>>> >> > Hi,
>>>>> >> >
>>>>> >> > I would go with the 1st approach. The thing I don't like in the
>>>>> >> > 2nd
>>>>> >> > approach
>>>>> >> > is that we have to make the user enable plugin twice. For example,
>>>>> >> > we
>>>>> >> > have
>>>>> >> > to enable Ceph as a plugin and then add Ceph role to nodes and
>>>>> >> > choose
>>>>> >> > what
>>>>> >> > we want to store in Ceph (images, objects). Why we would need to
>>>>> >> > explicitly
>>>>> >> > enable Ceph plugin? Let's always show plugin options in wizard and
>>>>> >> > settings
>>>>> >> > tab, and if the user just doesn't want to enable Ceph, he would
>>>>> >> > just
>>>>> >> > leave
>>>>> >> > all the checkboxes unchecked. The 2nd approach would also lead to
>>>>> >> > some
>>>>> >> > kind
>>>>> >> > of inconsistency in case the user enabled Ceph plugin but left all
>>>>> >> > the
>>>>> >> > Ceph-related checkboxes unchecked and didn't add Ceph nodes.
>>>>> >> >
>>>>> >> > 2014-10-07 21:17 GMT+07:00 Evgeniy L <eli at mirantis.com>:
>>>>> >> >>
>>>>> >> >> Hi,
>>>>> >> >>
>>>>> >> >> We had a meeting today about plugins on UI, as result of the
>>>>> >> >> meeting
>>>>> >> >> we have two approaches and this approaches affect not only UX but
>>>>> >> >> plugins itself.
>>>>> >> >>
>>>>> >> >> 1st - disable/enable plugin on settings tab
>>>>> >> >>
>>>>> >> >> user installs the plugin
>>>>> >> >> creates a cluster
>>>>> >> >> configures and enables/disables plugins on settings tab
>>>>> >> >>
>>>>> >> >> For user it will look like Ceph plugin checkboxes on settings
>>>>> >> >> tab,
>>>>> >> >> if he enables checkbox, then we pass the parameter to
>>>>> >> >> orchestrator
>>>>> >> >> as `true`.
>>>>> >> >>
>>>>> >> >> Cons:
>>>>> >> >>
>>>>> >> >> plugin developer should define a checkbox in each plugin (for
>>>>> >> >> plugin
>>>>> >> >> disabling/enabling)
>>>>> >> >> on the backend we have to enable all of the plugins for
>>>>> >> >> environment,
>>>>> >> >> because user can define any name for his checkbox and we won't be
>>>>> >> >> able
>>>>> >> >> to
>>>>> >> >> find it and make appropriate mapping plugin <-> env
>>>>> >> >> since all of the plugins are always "enabled" we have to run
>>>>> >> >> tasks for
>>>>> >> >> all
>>>>> >> >> of the plugins, and each plugin should parse astute.yaml in order
>>>>> >> >> to
>>>>> >> >> figure
>>>>> >> >> out if it's required to run task current script
>>>>> >> >>
>>>>> >> >> Pros:
>>>>> >> >>
>>>>> >> >> it won't require additional setting or step for wizard
>>>>> >> >> user will be able to disable plugin after environment creation
>>>>> >> >>
>>>>> >> >> 2nd - enable plugins in wizard
>>>>> >> >>
>>>>> >> >> user installs the plugin
>>>>> >> >> now he can choose specific plugins for his environment in wizard
>>>>> >> >> after cluster is created, he can configure additional parameters
>>>>> >> >> on
>>>>> >> >> settings tab, if plugin provides any
>>>>> >> >>
>>>>> >> >> Cons:
>>>>> >> >>
>>>>> >> >> user won't be able to disable plugin after cluster is created
>>>>> >> >> additional step or configuration subcategory in wizard
>>>>> >> >>
>>>>> >> >> Pros:
>>>>> >> >>
>>>>> >> >> On backend we always know which plugin is disabled and which is
>>>>> >> >> enabled.
>>>>> >> >>
>>>>> >> >> it means we don't provide settings for plugins which are disabled
>>>>> >> >> we don't run tasks on slaves if it's not required
>>>>> >> >>
>>>>> >> >> Thanks,
>>>>> >> >>
>>>>> >> >> _______________________________________________
>>>>> >> >> OpenStack-dev mailing list
>>>>> >> >> OpenStack-dev at lists.openstack.org
>>>>> >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> >> >>
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > --
>>>>> >> > Vitaly Kramskikh,
>>>>> >> > Software Engineer,
>>>>> >> > Mirantis, Inc.
>>>>> >> >
>>>>> >> > _______________________________________________
>>>>> >> > OpenStack-dev mailing list
>>>>> >> > OpenStack-dev at lists.openstack.org
>>>>> >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> >> >
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Best regards,
>>>>> >> Nick Markov
>>>>> >>
>>>>> >> _______________________________________________
>>>>> >> OpenStack-dev mailing list
>>>>> >> OpenStack-dev at lists.openstack.org
>>>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > OpenStack-dev mailing list
>>>>> > OpenStack-dev at lists.openstack.org
>>>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Nick Markov
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Vitaly Kramskikh,
>>>> Software Engineer,
>>>> Mirantis, Inc.
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Vitaly Kramskikh,
>> Software Engineer,
>> Mirantis, Inc.
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Dmitry Borodaenko