[openstack-dev] [Murano] Versioning of Murano packages and MuranoPL classes
Georgy Okrokvertskhov
gokrokvertskhov at mirantis.com
Mon Jul 13 14:54:14 UTC 2015
On Mon, Jul 13, 2015 at 2:19 AM, Alexander Tivelkov <ativelkov at mirantis.com>
wrote:
> Hi Gosha,
>
> Supporting versioning in existing backend will require us to re-implement
> the significant part of Artifact Repository in Murano API: we'll need to
> add versions and dependencies concepts into our model (which is already
> complicated and dirty enough), extend appropriate API calls etc. And all
> the efforts will be a waste of time once we finally migrate to Artifacts.
>
> Also, what do you mean by "set by default"? V3 API is experimental, but it
> is already merged into upstream Glance, so there is no problem with using
> it in Murano right away.
>
> This is exactly why I have these concerns. I wonder how much customers
will use experimental API in production. I just don't want to add extra
block on Murano adoption way.
> --
> Regards,
> Alexander Tivelkov
>
> On Fri, Jul 10, 2015 at 5:56 PM, Georgy Okrokvertskhov <
> gokrokvertskhov at mirantis.com> wrote:
>
>> Hi Alex,
>>
>> Thank you for the great summary.
>>
>> I have a concern about item #8. Can we add an option to Murano to use
>> previous storage engine rather then Glance v3? We need to make sure that v3
>> API in Glance is set by default before we do a hard dependency on it in
>> Murano.
>>
>> Thanks
>> Gosha
>>
>> On Fri, Jul 10, 2015 at 4:53 AM, Alexander Tivelkov <
>> ativelkov at mirantis.com> wrote:
>>
>>> Hi folks,
>>>
>>> Ability to manage multiple versions of application packages and their
>>> dependencies was always an important item in Murano roadmap, however we
>>> still don't have a clear spec for this feature.
>>> Yesterday we hosted a small design session to come up with a plan on
>>> what can be done in Liberty timeframe to have proper versioning for
>>> MuranoPL classes and packages. Stan Lagun, Kirill Zaitsev and myself
>>> participated offline, some other muranoers joined remotely. Thanks to
>>> everybody who joined us.
>>>
>>> TL;DR: it turns out that now we have a clear plan which will allow us to
>>> achieve proper versioning of the packages and classes, and we'll try to
>>> implement the most important parts of it in Liberty.
>>>
>>> Here is the detailed outcome of the session:
>>>
>>>
>>> 1. We'll use the standard Semantic Versioning format
>>> ('Major.Minor.Patch[-dev-build.label[+metadata.label]]') to version our
>>> packages: changes which break backwards-compatibility should increment the
>>> major segment, non-breaking new features increment the minor segment and
>>> all non-breaking bugfixes increment the patch segment. The developers
>>> should be carefull with the "new features" part: if you add a new method to
>>> a class, it may be considered a breaking change if the existing subclasses
>>> of this class have the same method already declared. We still assume that
>>> such changes should lead to increment of 'minor' segment, however it is up
>>> to best judgement of developers in particular case: if the risk of such
>>> method override is really high it may worth to increment the 'major'
>>> segment. Proper guideline on the versioning rules will be published closer
>>> to L release.
>>>
>>> 2. A new field 'Version' is introduced into package manifest file
>>> which should define package version in complete semver format. The field
>>> itself is optional (so existing apps are not broken), if it is not
>>> specified the package is assumed to have version '0.0.0'
>>>
>>> 3. The existing 'Require' block of Application manifest will be used
>>> to specify the package dependencies. Currently it is a yaml-based
>>> dictionary, with the keys set to fully-qualified names of the dependency
>>> packages and the values set to the version of those dependencies. Currently
>>> this block is used only for integration with apps stored at
>>> apps.openstack.org. It is suggested to use this block in the
>>> deployment process as well, and extend its semantics.
>>> The version of the dependency specified there should also follow the
>>> semver notation, however it may be specified in the shortened format, i.e.
>>> without specifying the 'patch' or 'patch' and 'minior' components. In this
>>> case the dependency will be specified as a range of allowed versions. For
>>> example, a dependency version 1.2 will mean a (1.2.0 >= version < 1.3)
>>> range.
>>> If the version of a dependency is not specified (like in this
>>> existing app - [1]) then we assume the version "0" - i.e. the last
>>> available pre-release version of a package.
>>>
>>> 4. Murano core library is also a package which has its own version.
>>> The current one is assumed to have a version 0.1.0, the one which is going
>>> to be released in L will be probably called 0.2.0. The lib is still quickly
>>> evolving, so we are not releasing a 1.0.0 until we are sure that we are not
>>> going to have any breaking changes anytime soon.
>>> As with any other package it will be possible to have several
>>> versions of the Core Library installed in Murano at the same moment of time.
>>>
>>> 5. There is no mandatory need to add the the dependency on the core
>>> library to the "Requires" block of each application, as it is added there
>>> implicitly. However, this implicit dependency will have a version "0" -
>>> i.e. will reference the latest pre-release version of the Core Library
>>> available. So it is still better to pin the core library requirement to a
>>> particular version to make sure that your app does not break if we
>>> introduce any breaking change into the core lib.
>>>
>>> 6. All classes defined in a package are assumed to have a version
>>> identical to the version of the package.
>>>
>>> 7. Murano Extension Plugins (i.e python packages which declare
>>> setuptools-entrypoints in 'io.murano.extensions' namespace) also will have
>>> similar versioning semantics: they will have a fully qualified name
>>> (defined by the setuptools' package name) and a version (also defined by
>>> setuptools), an will get an ability to specify their own dependencies if
>>> needed. From the class-loader perspective the MuranoPL classes defined in
>>> the plugins are no difference from the classes defined in a regular package.
>>>
>>> 8. We are going to store murano packages as Glance V3 Artifacts
>>> Repository, naturally mapping package's FQN and version to artifact's name
>>> and version.
>>> The package dependencies will be stored in Glance as cross-artifact
>>> dynamic dependencies (i.e. dependencies not on a particular artifact but on
>>> the last artifact matching the given name and the version range query) as
>>> soon as that feature is implemented in Glance (currently only static
>>> dependencies are implemented there). Until that, the dependencies will be
>>> stored as a regular list of strings, and the Murano engine will process it
>>> and query Glance to fetch the packages.
>>>
>>> 9. In L cycle we are not going to show multiple versions of the same
>>> app in Murano dashboard: only the last one will be shown if the multiple
>>> versions are present. This is to minimize the changes at Dashboard side: in
>>> future releases we'll add the ability to select the proper version.
>>> The generation of the object model by dynamic UI also remains intact.
>>>
>>> 10. However, the structure of the object model isself gets changed:
>>> in the "?" block of each object two new fields appear: "package" and
>>> "version", which correspond to the FQN and the version of the package which
>>> contain the class of the given object. UI leaves these fields as Nones when
>>> it generates the OM, and the engine computes them in a regular way: queries
>>> the package repository for the most recent version of a package which
>>> contains the class with a given name, and saves information about its name
>>> and version. This values get persisted in an Object Model when it gets
>>> serialized after the deployment. As a result, the versions of the
>>> components are fixed once the environment is deployed, so if some packages
>>> get updated afterwards, the existing components remain pinned to their
>>> initial version. As a result, the environment may get several components of
>>> the same type but different versions.
>>>
>>> 11. When the Object Model is validated after the deserialization,
>>> the behavior of "$.class()" contract is changed. During its validation the
>>> value passed to the appropriate property or argument should be of a type
>>> which is declared either in a current package (or in the another version of
>>> the current package, given that the major component of the versions is the
>>> same) or in one of the packages satisfying the requirements of the current
>>> one. I.e. it becomes impossible to reference any class from the
>>> unreferenced package.
>>>
>>> 12. When inheriting some other class using the 'Extends' attribute,
>>> the ancestor class should be defined either in the current package or in
>>> one of the packages satisfying the requirements of the current one.
>>>
>>> 13. (creepy advanced stuff) It may turn out that in case of the
>>> multiple inheritance a single class will attempt to inherit from two
>>> different versions of a same class. An exception should be thrown in this
>>> case, unless there is a possibility to find a version of this class which
>>> satisfies all parties.
>>>
>>> *For example: classA inherits classB from packageX and classC from
>>> packageY. Both classB and classC inherit from classD from packageZ, however
>>> packageX depends on the version 1.2.0 of packageZ, while packageY depends
>>> on the version 1.3.0. This leads to a situation when classA transitively
>>> inherits classD of both versions 1.2 and 1.3. So, an exception will be
>>> thrown. However, if packageY's dependency would be just "1" (which means
>>> any of the 1.x.x family) the conflict would be resolved and a 1.2 would be
>>> used as it satisfies both inheritance chains.*
>>>
>>>
>>>
>>> So, all the above cover most of our present needs for MuranoPL package
>>> and class versioning.
>>> Also, we already have a way which allows us to properly version the
>>> format of MuranoPL language (a "Format" key in application manifests) and
>>> UI-definition files ("Version" key in that files). This basically allows us
>>> to target the packages for a minimum version of Murano / Murano Dashboard.
>>>
>>>
>>> I hope this rather lengthly email is useful. Stan Lagun has taken an
>>> action item to frame all the above into a more formal spec.
>>>
>>>
>>>
>>> [1]
>>> https://github.com/openstack/murano-apps/blob/master/MySQL/package/manifest.yaml#L26
>>>
>>>
>>> --
>>> Regards,
>>> Alexander Tivelkov
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Georgy Okrokvertskhov
>> Architect,
>> OpenStack Platform Products,
>> Mirantis
>> http://www.mirantis.com
>> Tel. +1 650 963 9828
>> Mob. +1 650 996 3284
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
--
Georgy Okrokvertskhov
Architect,
OpenStack Platform Products,
Mirantis
http://www.mirantis.com
Tel. +1 650 963 9828
Mob. +1 650 996 3284
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150713/d6cbb210/attachment.html>
More information about the OpenStack-dev
mailing list