[openstack-dev] [Fuel][Plugins] Multi release packages

Ilya Kutukov ikutukov at mirantis.com
Fri Feb 12 12:37:37 UTC 2016


Excuse me, i mean multi-release package. We already have release directives
in plugin metadata.yaml that defines releases compatible with the plugin.
As far as i understand "multi-release package" suppose ability to define
custom configuration/code for each of this releases.


On Fri, Feb 12, 2016 at 3:19 PM, Evgeniy L <eli at mirantis.com> wrote:

> >> We have package format <=4.0 where all files have fixed names and
> locations. This is the defaults.
>
> The thing is for 5.0 there should be no default, those fields from now on
> should be specified explicitly.
>
> >> Igor want to provide multi-package
>
> I'm not familiar with this idea, could you please clarify what
> multi-package is?
>
> Thanks,
>
> On Fri, Feb 12, 2016 at 2:57 PM, Ilya Kutukov <ikutukov at mirantis.com>
> wrote:
>
>>
>>
>> On Fri, Feb 12, 2016 at 2:03 PM, Evgeniy L <eli at mirantis.com> wrote:
>>
>>> Ilya,
>>>
>>> What do you mean by "default"? From the data format I see that we don't
>>> "override defaults" we specify the data for specific release, the way it
>>> was always done for deployment scripts and repositories.
>>>
>>>
>> We have package format <=4.0 where all files have fixed names and
>> locations. This is the defaults.
>>
>> 1. The maintenance team wants ability to specify folder instead plugin
>> configuration files so there should be ability to change this paths to
>> define a folder or other non-standard location. Yes, plugin developer could
>> have whatever source structure and then translate it to the file structure
>> required for the FPB with scripts or build system, but ability to split
>> e.g. tasks files looks useful for me.
>>
>> 2. Igor want to provide multi-package, so, according to spec, this custom
>> release-specific paths to other package files could be specified in release
>> records.
>>
>> I don't see any reasons to complicate format even more and to have some
>>> things which are related to the deployment specified in the root and some
>>> in specific release.
>>>
>>> There is consistent mechanism to specify such kind of things, lets just
>>> use it.
>>>
>>> Thanks,
>>>
>>> On Fri, Feb 12, 2016 at 1:24 PM, Ilya Kutukov <ikutukov at mirantis.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Feb 12, 2016 at 11:47 AM, Evgeniy L <eli at mirantis.com> wrote:
>>>>
>>>>> Ilya,
>>>>>
>>>>> >> My opinion that i've seen no example of multiple software of
>>>>> plugins versions shipped in one package or other form of bundle. Its not a
>>>>> common practice.
>>>>>
>>>>> With plugins we extend Fuel capabilities, which supports multiple
>>>>> operating system releases, so it's absolutely natural to extend multiple
>>>>> releases at the same time.
>>>>>
>>>>>
>>>> I just warning against idea when to merge content of several plugin
>>>> distributions in one bundle. But it's ok for plugin code to support
>>>> multiple releases one or another way.
>>>>
>>>>
>>>>
>>>>> >> Anyway we need to provide ability to override paths in manifest
>>>>> (metadata.yaml).
>>>>>
>>>>> Could you please provide more information on that? I'm not sure if I
>>>>> understand your solution.
>>>>>
>>>>>
>>>> https://review.openstack.org/#/c/271417/5/specs/9.0/plugins-v5.rst
>>>> L150 and further
>>>> We are overriding default path with special per-release path attributes.
>>>> The question is to use per-release way described in spec or don't
>>>> bother and specify this overrides only in metadata.yaml root.
>>>>
>>>>
>>>>> Also I'm not sure what we are arguing about, if plugin developer (or
>>>>> certification process of some company) requires to have plugin per release,
>>>>> it's *very easy* and straight forward to do it even now *without any*
>>>>> changes.
>>>>>
>>>>> If plugin developer wants to deliver plugin for CentOS, Ubuntu, RH
>>>>> etc, let them do it, and again when we get full support
>>>>> of multi-version environments it's going to be even more crucial for UX to
>>>>> have a single plugin with multi-release support.
>>>>>
>>>>>
>>>> Thanks,
>>>>>
>>>>> On Thu, Feb 11, 2016 at 9:35 PM, Ilya Kutukov <ikutukov at mirantis.com>
>>>>> wrote:
>>>>>
>>>>>> My opinion that i've seen no example of multiple software of plugins
>>>>>> versions shipped in one package or other form of bundle. Its not a common
>>>>>> practice.
>>>>>>
>>>>>> Anyway we need to provide ability to override paths in manifest
>>>>>> (metadata.yaml).
>>>>>>
>>>>>> So the plugin developers could use this approaches to provide
>>>>>> multiple versions support:
>>>>>>
>>>>>> * tasks logic (do the plugin developers have access to current
>>>>>> release info?)
>>>>>> * hooks in pre-build process. Its not a big deal to preprocess source
>>>>>> folder to build different packages with scripts that adding or removing
>>>>>> some files or replacing some paths.
>>>>>> * and, perhaps, logic anchors with YACL or other DSL in tasks
>>>>>> dependancies if this functionality will be added this in theory could allow
>>>>>> to use or not to use some graph parts depending on release.
>>>>>>
>>>>>> I think its already better than nothing and more flexible than any
>>>>>> standardised approach.
>>>>>>
>>>>>> On Thu, Feb 11, 2016 at 6:31 PM, Simon Pasquier <
>>>>>> spasquier at mirantis.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Thu, Feb 11, 2016 at 11:46 AM, Igor Kalnitsky <
>>>>>>> ikalnitsky at mirantis.com> wrote:
>>>>>>>
>>>>>>>> Hey folks,
>>>>>>>>
>>>>>>>> The original idea is to provide a way to build plugin that are
>>>>>>>> compatible with few releases. It makes sense to me, cause it looks
>>>>>>>> awful if you need to maintain different branches for different Fuel
>>>>>>>> releases and there's no difference in the sources. In that case,
>>>>>>>> each
>>>>>>>> bugfix to deployment scripts requires:
>>>>>>>>
>>>>>>>> * backport bugfix to other branches (N backports)
>>>>>>>> * build new packages for supported releases (N builds)
>>>>>>>> * release new packages (N releases)
>>>>>>>>
>>>>>>>> It's somehow.. annoying.
>>>>>>>>
>>>>>>>
>>>>>>> A big +1 on Igor's remark. I've already expressed it in another
>>>>>>> thread but it should be expected that plugin developers want to support 2
>>>>>>> consecutive versions of Fuel for a given version of their plugin.
>>>>>>> That being said, I've never had issues to do it with the current
>>>>>>> plugin framework. Except when Fuel breaks the backward compatibility but
>>>>>>> it's another story...
>>>>>>>
>>>>>>> Simon
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> However, I starting agree that having all-in-one RPM when deployment
>>>>>>>> scripts are different, tasks are different, roles/volumes are
>>>>>>>> different, probably isn't a good idea. It basically means that your
>>>>>>>> sources are completely different, and that means you have different
>>>>>>>> implementations of the same plugin. In that case, in order to avoid
>>>>>>>> mess in source tree, it'd be better to separate such implementations
>>>>>>>> on VCS level.
>>>>>>>>
>>>>>>>> But I'd like to hear more opinion from plugin developers.
>>>>>>>>
>>>>>>>> - Igor
>>>>>>>>
>>>>>>>> On Thu, Feb 11, 2016 at 9:16 AM, Bulat Gaifullin
>>>>>>>> <bgaifullin at mirantis.com> wrote:
>>>>>>>> > I agree with Stas, one rpm - one version.
>>>>>>>> >
>>>>>>>> > But plugin builder allows to specify several releases as
>>>>>>>> compatible. The
>>>>>>>> > deployment tasks and repositories can be specified per release,
>>>>>>>> at the same
>>>>>>>> > time the deployment graph is one for all releases.
>>>>>>>> > Currently it looks like half-implemented feature.  Can we drop
>>>>>>>> this feature?
>>>>>>>> > or should we finish implementation of this feature.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Regards,
>>>>>>>> > Bulat Gaifullin
>>>>>>>> > Mirantis Inc.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On 11 Feb 2016, at 02:41, Andrew Woodward <xarses at gmail.com>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Wed, Feb 10, 2016 at 2:23 PM Dmitry Borodaenko <
>>>>>>>> dborodaenko at mirantis.com>
>>>>>>>> > wrote:
>>>>>>>> >>
>>>>>>>> >> +1 to Stas, supplanting VCS branches with code duplication is a
>>>>>>>> path to
>>>>>>>> >> madness and despair. The dubious benefits of a cross-release
>>>>>>>> backwards
>>>>>>>> >> compatible plugin binary are not worth the code and infra
>>>>>>>> technical debt
>>>>>>>> >> that such approach would accrue over time.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Supporting multiple fuel releases will likely result in madness as
>>>>>>>> > discussed, however as we look to support multiple OpenStack
>>>>>>>> releases from
>>>>>>>> > the same version of fuel, this methodology becomes much more
>>>>>>>> important.
>>>>>>>> >
>>>>>>>> >>
>>>>>>>> >> On Wed, Feb 10, 2016 at 07:36:30PM +0300, Stanislaw Bogatkin
>>>>>>>> wrote:
>>>>>>>> >> > It changes mostly nothing for case of furious plugin
>>>>>>>> development when
>>>>>>>> >> > big
>>>>>>>> >> > parts of code changed from one release to another.
>>>>>>>> >> >
>>>>>>>> >> > You will have 6 different deployment_tasks directories and 30
>>>>>>>> a little
>>>>>>>> >> > bit
>>>>>>>> >> > different files in root directory of plugin. Also you forgot
>>>>>>>> about
>>>>>>>> >> > repositories directory (+6 at least), pre_build hooks (also 6)
>>>>>>>> and so
>>>>>>>> >> > on.
>>>>>>>> >> > It will look as hell after just 3 years of development.
>>>>>>>> >> >
>>>>>>>> >> > Also I can't imagine how to deal with plugin licensing if you
>>>>>>>> have
>>>>>>>> >> > Apache
>>>>>>>> >> > for liberty but BSD for mitaka release, for example.
>>>>>>>> >> >
>>>>>>>> >> > Much easier way to develop a plugin is to keep it's source in
>>>>>>>> VCS like
>>>>>>>> >> > Git
>>>>>>>> >> > and just make a branches for every fuel release. It will give
>>>>>>>> us
>>>>>>>> >> > opportunity to not store a bunch of similar but a little bit
>>>>>>>> different
>>>>>>>> >> > files in repo. There is no reason to drag all different
>>>>>>>> versions of code
>>>>>>>> >> > for specific release.
>>>>>>>> >> >
>>>>>>>> >> >
>>>>>>>> >> > On other hand there is a pros - your plugin can survive after
>>>>>>>> upgrade if
>>>>>>>> >> > it
>>>>>>>> >> > supports new release, no changes needed here.
>>>>>>>> >> >
>>>>>>>> >> > On Wed, Feb 10, 2016 at 4:04 PM, Alexey Shtokolov
>>>>>>>> >> > <ashtokolov at mirantis.com>
>>>>>>>> >> > wrote:
>>>>>>>> >> >
>>>>>>>> >> > > Fuelers,
>>>>>>>> >> > >
>>>>>>>> >> > > We are discussing the idea to extend the multi release
>>>>>>>> packages for
>>>>>>>> >> > > plugins.
>>>>>>>> >> > >
>>>>>>>> >> > > Fuel plugin builder (FPB) can create one rpm-package for all
>>>>>>>> supported
>>>>>>>> >> > > releases (from metadata.yaml) but we can specify only
>>>>>>>> deployment
>>>>>>>> >> > > scripts
>>>>>>>> >> > > and repositories per release.
>>>>>>>> >> > >
>>>>>>>> >> > > Current release definition (in metadata.yaml):
>>>>>>>> >> > >     - os: ubuntu
>>>>>>>> >> > >       version: liberty-8.0
>>>>>>>> >> > >       mode: ['ha']
>>>>>>>> >> > >       deployment_scripts_path: deployment_scripts/
>>>>>>>> >> > >       repository_path: repositories/ubuntu
>>>>>>>> >> > >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > This will result in far too much clutter.
>>>>>>>> > For starters we should support nested over rides. for example the
>>>>>>>> author may
>>>>>>>> > have already taken account for the changes between one openstack
>>>>>>>> version to
>>>>>>>> > another. In this case they only should need to define the
>>>>>>>> releases they
>>>>>>>> > support and not specify any additional locations. Later they may
>>>>>>>> determine
>>>>>>>> > that they only need to replace packages, or one other file they
>>>>>>>> should not
>>>>>>>> > be required to code every location for each release
>>>>>>>> >
>>>>>>>> > Also, at the same time we MUST clean up importing various yaml
>>>>>>>> files.
>>>>>>>> > Specifically, tasks, volumes, node roles, and network roles.
>>>>>>>> Requiring that
>>>>>>>> > they all be maintained in a single file doesn't scale, we don't
>>>>>>>> require it
>>>>>>>> > for tasks.yaml in fuel library, and we should not require it in
>>>>>>>> plugins. We
>>>>>>>> > should simply do the same thing as tasks.yaml in library, scan
>>>>>>>> the subtree
>>>>>>>> > for specific file names and just merge them all together. (This
>>>>>>>> has been
>>>>>>>> > expressed multiple times by people with larger plugins)
>>>>>>>> >
>>>>>>>> >> > > So the idea [0] is to make releases fully configurable.
>>>>>>>> >> > > Suggested changes for release definition (in metadata.yaml):
>>>>>>>> >> > >       components_path: components_liberty.yaml
>>>>>>>> >> > >       deployment_tasks_path: deployment_tasks_liberty/ # <-
>>>>>>>> folder
>>>>>>>> >>
>>>>>>>> >> > >       environment_config_path:
>>>>>>>> environment_config_liberty.yaml
>>>>>>>> >> > >       network_roles_path: network_roles_liberty.yaml
>>>>>>>> >> > >       node_roles_path: node_roles_liberty.yaml
>>>>>>>> >> > >       volumes_path: volumes_liberty.yaml
>>>>>>>> >> > >
>>>>>>>> >> > > I see the issue: if we change anything for one release (e.g.
>>>>>>>> >> > > deployment_task typo) revalidation is needed for all
>>>>>>>> releases.
>>>>>>>> >> > >
>>>>>>>> >> > > Your Pros and cons please?
>>>>>>>> >> > >
>>>>>>>> >> > > [0] https://review.openstack.org/#/c/271417/
>>>>>>>> >> > > ---
>>>>>>>> >> > > WBR, Alexey Shtokolov
>>>>>>>> >> > >
>>>>>>>> >> > >
>>>>>>>> >> > >
>>>>>>>> __________________________________________________________________________
>>>>>>>> >> > > OpenStack Development Mailing List (not for usage questions)
>>>>>>>> >> > > Unsubscribe:
>>>>>>>> >> > >
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> >> > >
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>> >> > >
>>>>>>>> >> > >
>>>>>>>> >> >
>>>>>>>> >> >
>>>>>>>> >> > --
>>>>>>>> >> > with best regards,
>>>>>>>> >> > Stan.
>>>>>>>> >>
>>>>>>>> >> >
>>>>>>>> >> >
>>>>>>>> __________________________________________________________________________
>>>>>>>> >> > OpenStack Development Mailing List (not for usage questions)
>>>>>>>> >> > Unsubscribe:
>>>>>>>> >> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> >> >
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>> __________________________________________________________________________
>>>>>>>> >> OpenStack Development Mailing List (not for usage questions)
>>>>>>>> >> Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> >>
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>> >
>>>>>>>> > --
>>>>>>>> >
>>>>>>>> > --
>>>>>>>> >
>>>>>>>> > Andrew Woodward
>>>>>>>> >
>>>>>>>> > Mirantis
>>>>>>>> >
>>>>>>>> > Fuel Community Ambassador
>>>>>>>> >
>>>>>>>> > Ceph Community
>>>>>>>> >
>>>>>>>> >
>>>>>>>> __________________________________________________________________________
>>>>>>>> > OpenStack Development Mailing List (not for usage questions)
>>>>>>>> > Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> >
>>>>>>>> __________________________________________________________________________
>>>>>>>> > OpenStack Development Mailing List (not for usage questions)
>>>>>>>> > Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>> __________________________________________________________________________
>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>> Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> __________________________________________________________________________
>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>> Unsubscribe:
>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> __________________________________________________________________________
>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>> Unsubscribe:
>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160212/67590c8a/attachment.html>


More information about the OpenStack-dev mailing list