Open Stack

Vitaly, what do you think about that?

On Fri, Dec 12, 2014 at 5:58 PM, Evgeniy L <eli at mirantis.com> wrote:
>
> Hi,
>
> I don't agree with many of your statements but, I would like to
> continue discussion about really important topic i.e. UI flow, my
> suggestion was to add groups, for plugin in metadata.yaml plugin
> developer can have description of the groups which it belongs to:
>
> groups:
>   - id: storage
>     subgroup:
>       - id: cinder
>
> With this information we can show a new option on UI (wizard),
> if option is selected, it means that plugin is enabled, if plugin belongs
> to several groups, we can use OR statement.
>
> The main point is, for environment creation we must specify
> ids of plugins. Yet another reason for that is plugins multiversioning,
> we must know exactly which plugin with which version
> is used for environment, and I don't see how "conditions" can help
> us with it.
>
> Thanks,
>
>>
>>>
>>>
> On Wed, Dec 10, 2014 at 8:23 PM, Vitaly Kramskikh <vkramskikh at mirantis.com
> > wrote:
>>
>>
>>
>> 2014-12-10 19:31 GMT+03:00 Evgeniy L <eli at mirantis.com>:
>>
>>>
>>>
>>> On Wed, Dec 10, 2014 at 6:50 PM, Vitaly Kramskikh <
>>> vkramskikh at mirantis.com> wrote:
>>>
>>>>
>>>>
>>>> 2014-12-10 16:57 GMT+03:00 Evgeniy L <eli at mirantis.com>:
>>>>
>>>>> Hi,
>>>>>
>>>>> First let me describe what our plans for the nearest release. We want
>>>>> to deliver
>>>>> role as a simple plugin, it means that plugin developer can define his
>>>>> own role
>>>>> with yaml and also it should work fine with our current approach when
>>>>> user can
>>>>> define several fields on the settings tab.
>>>>>
>>>>> Also I would like to mention another thing which we should probably
>>>>> discuss
>>>>> in separate thread, how plugins should be implemented. We have two
>>>>> types
>>>>> of plugins, simple and complicated, the definition of simple - I can
>>>>> do everything
>>>>> I need with yaml, the definition of complicated - probably I have to
>>>>> write some
>>>>> python code. It doesn't mean that this python code should do absolutely
>>>>> everything it wants, but it means we should implement stable,
>>>>> documented
>>>>> interface where plugin is connected to the core.
>>>>>
>>>>> Now lets talk about UI flow, our current problem is how to get the
>>>>> information
>>>>> if plugins is used in the environment or not, this information is
>>>>> required for
>>>>> backend which generates appropriate tasks for task executor, also this
>>>>> information can be used in the future if we decide to implement
>>>>> plugins deletion
>>>>> mechanism.
>>>>>
>>>>> I didn't come up with a some new solution, as before we have two
>>>>> options to
>>>>> solve the problem:
>>>>>
>>>>> # 1
>>>>>
>>>>> Use conditional language which is currently used on UI, it will look
>>>>> like
>>>>> Vitaly described in the example [1].
>>>>> Plugin developer should:
>>>>>
>>>>> 1. describe at least one element for UI, which he will be able to use
>>>>> in task
>>>>>
>>>>> 2. add condition which is written in our own programming language
>>>>>
>>>>> Example of the condition for LBaaS plugin:
>>>>>
>>>>>     condition: settings:lbaas.metadata.enabled == true
>>>>>
>>>>> 3. add condition to metadata.yaml a condition which defines if plugin
>>>>> is enabled
>>>>>
>>>>>     is_enabled: settings:lbaas.metadata.enabled == true
>>>>>
>>>>> This approach has good flexibility, but also it has problems:
>>>>>
>>>>> a. It's complicated and not intuitive for plugin developer.
>>>>>
>>>> It is less complicated than python code
>>>>
>>>
>>> I'm not sure why are you talking about python code here, my point
>>> is we should not force developer to use this conditions in any language.
>>>
>>> But that's how current plugin-like stuff works. There are various tasks
>> which are run only if some checkboxes are set, so stuff like Ceph and
>> vCenter will need conditions to describe tasks.
>>
>>> Anyway I don't agree with the statement there are more people who know
>>> python than "fuel ui conditional language".
>>>
>>>
>>>> b. It doesn't cover case when the user installs 3rd party plugin
>>>>>     which doesn't have any conditions (because of # a) and
>>>>>     user doesn't have a way to disable it for environment if it
>>>>>     breaks his configuration.
>>>>>
>>>> If plugin doesn't have conditions for tasks, then it has invalid
>>>> metadata.
>>>>
>>>
>>> Yep, and it's a problem of the platform, which provides a bad interface.
>>>
>> Why is it bad? It plugin writer doesn't provide plugin name or version,
>> then metadata is invalid also. It is plugin writer's fault that he didn't
>> write metadata properly.
>>
>>>
>>>
>>>>
>>>>> # 2
>>>>>
>>>>> As we discussed from the very beginning after user selects a release
>>>>> he can
>>>>> choose a set of plugins which he wants to be enabled for environment.
>>>>> After that we can say that plugin is enabled for the environment and
>>>>> we send
>>>>> tasks related to this plugin to task executor.
>>>>>
>>>>> >> My approach also allows to eliminate "enableness" of plugins which
>>>>> will cause UX issues and issues like you described above. vCenter and Ceph
>>>>> also don't have "enabled" state. vCenter has hypervisor and storage, Ceph
>>>>> provides backends for Cinder and Glance which can be used simultaneously or
>>>>> only one of them can be used.
>>>>>
>>>>> Both of described plugins have enabled/disabled state, vCenter is
>>>>> enabled
>>>>> when vCenter is selected as hypervisor. Ceph is enabled when it's
>>>>> selected
>>>>> as a backend for Cinder or Glance.
>>>>>
>>>> Nope, Ceph for Volumes can be used without Ceph for Images. Both of
>>>> these plugins can also have some granular tasks which are enabled by
>>>> various checkboxes (like VMware vCenter for volumes). How would you
>>>> determine whether tasks which installs VMware vCenter for volumes should
>>>> run?
>>>>
>>>
>>> Why "nope"? I have "Cinder OR Glance".
>>>
>> Oh, I missed it. So there are 2 checkboxes, how would you determine
>> "enableness"?
>>
>>> It can be easily handled in deployment script.
>>>
>> I don't know much about the status of granular deployment blueprint, but
>> AFAIK that's what we are going to get rid of.
>>
>>>
>>>
>>>>> If you don't like the idea of having Ceph/vCenter checkboxes on the
>>>>> first page,
>>>>> I can suggest as an idea (research is required) to define groups like
>>>>> Storage Backend,
>>>>> Network Manager and we will allow plugin developer to embed his option
>>>>> in radiobutton
>>>>> field on wizard pages. But plugin developer should not describe
>>>>> conditions, he should
>>>>> just write that his plugin is a Storage Backend, Hypervisor or new
>>>>> Network Manager.
>>>>> And the plugins e.g. Zabbix, Nagios, which don't belong to any of this
>>>>> groups
>>>>> should be shown as checkboxes on the first page of the wizard.
>>>>>
>>>> Why don't you just ditch "enableness" of plugins and get rid of this
>>>> complex stuff? Can you explain why do you need to know if plugin is
>>>> "enabled"? Let me summarize my opinion on this:
>>>>
>>>
>>> I described why we need it many times. Also it looks like you skipped
>>> another option
>>> and I would like to see some more information why you don't like it and
>>> why it's
>>> a bad from UX stand point of view.
>>>
>> Yes, I skipped it. You said "research is required", so please do it,
>> write a proposal and then we will compare it with condition approach. You
>> still don't have your proposal, so there is nothing to compare and discuss.
>> From the first perspective it seems complex and restrictive.
>>
>>>
>>>>    - You don't need to know whether plugin is enabled or not. You need
>>>>    to know what tasks should be run and whether plugin is removable (anything
>>>>    else?). These conditions can be described by the DSL.
>>>>
>>>> I do need to know if plugin is enabled to figure out if it's removable,
>>> in fact those are the same things.
>>>
>> So there is nothing else you need "enableness", right? If you "described
>> why we need it many times", I think you need to do it one more time (in
>> form of a list). If we need "enableness" just to determine whether the
>> plugin is removable, then it is not the reason to ruin our UX.
>>
>>>
>>>>    -
>>>>    - Explicitly asking the user to enable plugin for new environment
>>>>    should be considered as a last resort solution because it significantly
>>>>    impair our UX for inexperienced user. Just imagine: a new user which barely
>>>>    knows about OpenStack chooses a name for the environment, OS release and
>>>>    then he needs to choose plugins. Really?
>>>>
>>>> I really think that it's absolutely ok to show checkbox with LBaaS for
>>> the user who found the
>>> plugin, downloaded it on the master and installed it with CLI.
>>>
>>> And right now this user have to go to this settings tab with attempts to
>>> find this checkbox,
>>> also he may not find it for example because of incompatible release
>>> version, and it's clearly
>>> a bad UX.
>>>
>> I like how it is done in modern browsers - after upgrade of master node
>> there is notification about incompatible plugins, and in list of plugins
>> there is a message that plugin is incompatible. We need to design how we
>> will handle it. Currently it is definitely a bad UX because nothing is done
>> for this.
>>
>>> My proposal for "complex" plugin interface: there should be python
>>>> classes with exactly the same fields from yaml files: plugin name, version,
>>>> etc. But condition for cluster deletion and for tasks which are written in
>>>> DSL in case of "simple" yaml config should become methods which plugin
>>>> writer can make as complex as he wants.
>>>>
>>> Why do you want to use python to define plugin name, version etc? It's a
>>> static data which are
>>> used for installation, I don't think that in fuel client (or some other
>>> installation tool) we want
>>> to unpack the plugin and import this module to get the information which
>>> is required for installation.
>>>
>> It is just a proposal in which I try to solve problems which you see in
>> my approach. If you want a "complex" interface with arbitrary python code,
>> that's how I see it. All fields are the same here, the approach is the
>> same, just conditions are in python. And YAML config can be converted to
>> this class, and all other code won't need to handle 2 different interfaces
>> for plugins.
>>
>>>
>>>>> [1]
>>>>> https://github.com/vkramskikh/fuel-plugins/commit/1ddb166731fc4bf614f502b276eb136687cb20cf
>>>>>
>>>>> On Sun, Nov 30, 2014 at 3:12 PM, Vitaly Kramskikh <
>>>>> vkramskikh at mirantis.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> 2014-11-28 23:20 GMT+04:00 Dmitriy Shulyak <dshulyak at mirantis.com>:
>>>>>>
>>>>>>>
>>>>>>>>    - environment_config.yaml should contain exact config which
>>>>>>>>    will be mixed into cluster_attributes. No need to implicitly generate any
>>>>>>>>    controls like it is done now.
>>>>>>>>
>>>>>>>>  Initially i had the same thoughts and wanted to use it the way it
>>>>>>> is, but now i completely agree with Evgeniy that additional DSL will cause
>>>>>>> a lot
>>>>>>> of problems with compatibility between versions and developer
>>>>>>> experience.
>>>>>>>
>>>>>> As far as I understand, you want to introduce another approach to
>>>>>> describe UI part or plugins?
>>>>>>
>>>>>>> We need to search for alternatives..
>>>>>>> 1. for UI i would prefer separate tab for plugins, where user will
>>>>>>> be able to enable/disable plugin explicitly.
>>>>>>>
>>>>>> Of course, we need a separate page for plugin management.
>>>>>>
>>>>>>> Currently settings tab is overloaded.
>>>>>>> 2. on backend we need to validate plugins against certain env before
>>>>>>> enabling it,
>>>>>>>    and for simple case we may expose some basic entities like
>>>>>>> network_mode.
>>>>>>> For case where you need complex logic - python code is far more
>>>>>>> flexible that new DSL.
>>>>>>>
>>>>>>>>
>>>>>>>>    - metadata.yaml should also contain "is_removable" field. This
>>>>>>>>    field is needed to determine whether it is possible to remove installed
>>>>>>>>    plugin. It is impossible to remove plugins in the current implementation.
>>>>>>>>    This field should contain an expression written in our DSL which we already
>>>>>>>>    use in a few places. The LBaaS plugin also uses it to hide the checkbox if
>>>>>>>>    Neutron is not used, so even simple plugins like this need to utilize it.
>>>>>>>>    This field can also be autogenerated, for more complex plugins plugin
>>>>>>>>    writer needs to fix it manually. For example, for Ceph it could look like
>>>>>>>>    "settings:storage.volumes_ceph.value == false and
>>>>>>>>    settings:storage.images_ceph.value == false".
>>>>>>>>
>>>>>>>> How checkbox will help? There is several cases of plugin removal..
>>>>>>>
>>>>>> It is not a checkbox, this is condition that determines whether the
>>>>>> plugin is removable. It allows plugin developer specify when plguin can be
>>>>>> safely removed from Fuel if there are some environments which were created
>>>>>> after the plugin had been installed.
>>>>>>
>>>>>>> 1. Plugin is installed, but not enabled for any env - just remove
>>>>>>> the plugin
>>>>>>> 2. Plugin is installed, enabled and cluster deployed - forget about
>>>>>>> it for now..
>>>>>>> 3. Plugin is installed and only enabled - we need to maintain state
>>>>>>> of db consistent after plugin is removed, it is problematic, but possible
>>>>>>>
>>>>>> My approach also allows to eliminate "enableness" of plugins which
>>>>>> will cause UX issues and issues like you described above. vCenter and Ceph
>>>>>> also don't have "enabled" state. vCenter has hypervisor and storage, Ceph
>>>>>> provides backends for Cinder and Glance which can be used simultaneously or
>>>>>> only one of them can be used.
>>>>>>
>>>>>>> My main point that plugin is enabled/disabled explicitly by user,
>>>>>>> after that we can decide ourselves can it be removed or not.
>>>>>>>
>>>>>>>>
>>>>>>>>    - For every task in tasks.yaml there should be added new
>>>>>>>>    "condition" field with an expression which determines whether the task
>>>>>>>>    should be run. In the current implementation tasks are always run for
>>>>>>>>    specified roles. For example, vCenter plugin can have a few tasks with
>>>>>>>>    conditions like "settings:common.libvirt_type.value == 'vcenter'" or
>>>>>>>>    "settings:storage.volumes_vmdk.value == true". Also, AFAIU, similar
>>>>>>>>    approach will be used in implementation of Granular Deployment feature.
>>>>>>>>
>>>>>>>> I had some thoughts about using DSL, it seemed to me especially
>>>>>>> helpfull when you need to disable part of embedded into core functionality,
>>>>>>> like deploying with another hypervisor, or network dirver (contrail
>>>>>>> for example). And DSL wont cover all cases here, this quite similar to
>>>>>>> metadata.yaml, simple cases can be covered by some variables in tasks (like
>>>>>>> group, unique, etc), but complex is easier to test and describe in python.
>>>>>>>
>>>>>> Could you please provide example of such conditions? vCenter and Ceph
>>>>>> can be turned into plugins using this approach.
>>>>>>
>>>>>> Also, I'm not against python version of plugins. It could look like a
>>>>>> python class with exactly the same fields form YAML files, but conditions
>>>>>> will be written in python.
>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing list
>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Vitaly Kramskikh,
>>>>>> Software Engineer,
>>>>>> Mirantis, Inc.
>>>>>>
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>> On Sun, Nov 30, 2014 at 3:12 PM, Vitaly Kramskikh <
>>>>> vkramskikh at mirantis.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> 2014-11-28 23:20 GMT+04:00 Dmitriy Shulyak <dshulyak at mirantis.com>:
>>>>>>
>>>>>>>
>>>>>>>>    - environment_config.yaml should contain exact config which
>>>>>>>>    will be mixed into cluster_attributes. No need to implicitly generate any
>>>>>>>>    controls like it is done now.
>>>>>>>>
>>>>>>>>  Initially i had the same thoughts and wanted to use it the way it
>>>>>>> is, but now i completely agree with Evgeniy that additional DSL will cause
>>>>>>> a lot
>>>>>>> of problems with compatibility between versions and developer
>>>>>>> experience.
>>>>>>>
>>>>>> As far as I understand, you want to introduce another approach to
>>>>>> describe UI part or plugins?
>>>>>>
>>>>>>> We need to search for alternatives..
>>>>>>> 1. for UI i would prefer separate tab for plugins, where user will
>>>>>>> be able to enable/disable plugin explicitly.
>>>>>>>
>>>>>> Of course, we need a separate page for plugin management.
>>>>>>
>>>>>>> Currently settings tab is overloaded.
>>>>>>> 2. on backend we need to validate plugins against certain env before
>>>>>>> enabling it,
>>>>>>>    and for simple case we may expose some basic entities like
>>>>>>> network_mode.
>>>>>>> For case where you need complex logic - python code is far more
>>>>>>> flexible that new DSL.
>>>>>>>
>>>>>>>>
>>>>>>>>    - metadata.yaml should also contain "is_removable" field. This
>>>>>>>>    field is needed to determine whether it is possible to remove installed
>>>>>>>>    plugin. It is impossible to remove plugins in the current implementation.
>>>>>>>>    This field should contain an expression written in our DSL which we already
>>>>>>>>    use in a few places. The LBaaS plugin also uses it to hide the checkbox if
>>>>>>>>    Neutron is not used, so even simple plugins like this need to utilize it.
>>>>>>>>    This field can also be autogenerated, for more complex plugins plugin
>>>>>>>>    writer needs to fix it manually. For example, for Ceph it could look like
>>>>>>>>    "settings:storage.volumes_ceph.value == false and
>>>>>>>>    settings:storage.images_ceph.value == false".
>>>>>>>>
>>>>>>>> How checkbox will help? There is several cases of plugin removal..
>>>>>>>
>>>>>> It is not a checkbox, this is condition that determines whether the
>>>>>> plugin is removable. It allows plugin developer specify when plguin can be
>>>>>> safely removed from Fuel if there are some environments which were created
>>>>>> after the plugin had been installed.
>>>>>>
>>>>>>> 1. Plugin is installed, but not enabled for any env - just remove
>>>>>>> the plugin
>>>>>>> 2. Plugin is installed, enabled and cluster deployed - forget about
>>>>>>> it for now..
>>>>>>> 3. Plugin is installed and only enabled - we need to maintain state
>>>>>>> of db consistent after plugin is removed, it is problematic, but possible
>>>>>>>
>>>>>> My approach also allows to eliminate "enableness" of plugins which
>>>>>> will cause UX issues and issues like you described above. vCenter and Ceph
>>>>>> also don't have "enabled" state. vCenter has hypervisor and storage, Ceph
>>>>>> provides backends for Cinder and Glance which can be used simultaneously or
>>>>>> only one of them can be used.
>>>>>>
>>>>>>> My main point that plugin is enabled/disabled explicitly by user,
>>>>>>> after that we can decide ourselves can it be removed or not.
>>>>>>>
>>>>>>>>
>>>>>>>>    - For every task in tasks.yaml there should be added new
>>>>>>>>    "condition" field with an expression which determines whether the task
>>>>>>>>    should be run. In the current implementation tasks are always run for
>>>>>>>>    specified roles. For example, vCenter plugin can have a few tasks with
>>>>>>>>    conditions like "settings:common.libvirt_type.value == 'vcenter'" or
>>>>>>>>    "settings:storage.volumes_vmdk.value == true". Also, AFAIU, similar
>>>>>>>>    approach will be used in implementation of Granular Deployment feature.
>>>>>>>>
>>>>>>>> I had some thoughts about using DSL, it seemed to me especially
>>>>>>> helpfull when you need to disable part of embedded into core functionality,
>>>>>>> like deploying with another hypervisor, or network dirver (contrail
>>>>>>> for example). And DSL wont cover all cases here, this quite similar to
>>>>>>> metadata.yaml, simple cases can be covered by some variables in tasks (like
>>>>>>> group, unique, etc), but complex is easier to test and describe in python.
>>>>>>>
>>>>>> Could you please provide example of such conditions? vCenter and Ceph
>>>>>> can be turned into plugins using this approach.
>>>>>>
>>>>>> Also, I'm not against python version of plugins. It could look like a
>>>>>> python class with exactly the same fields form YAML files, but conditions
>>>>>> will be written in python.
>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> OpenStack-dev mailing list
>>>>>>> OpenStack-dev at lists.openstack.org
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Vitaly Kramskikh,
>>>>>> Software Engineer,
>>>>>> Mirantis, Inc.
>>>>>>
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Vitaly Kramskikh,
>>>> Software Engineer,
>>>> Mirantis, Inc.
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Vitaly Kramskikh,
>> Software Engineer,
>> Mirantis, Inc.
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141217/18258801/attachment.html>