Open Stack

On Mon, Nov 24, 2014 at 4:40 AM, Evgeniy L <eli at mirantis.com> wrote:
> Hi Andrew,
>
> Comments inline.
> Also could you please provide a link on OpenStack upgrade feature?
> It's not clear why do you need it as a plugin and how you are going
> to deliver this feature.
>
> On Sat, Nov 22, 2014 at 4:23 AM, Andrew Woodward <xarses at gmail.com> wrote:
>>
>> So as part of the pumphouse integration, I've started poking around
>> the Plugin Arch implementation as an attempt to plug it into the fuel
>> master.
>>
>> This would require that the plugin install a container, and some
>> scripts into the master node.
>>
>> First look:
>> I've looked over the fuel plugins spec [1] and see that the install
>> script was removed from rev 15 ->16 (line 134) This creates problems
>> do to the need of installing the container, and scripts so I've
>> created a bug [2] for this so that we can allow for an install script
>> to be executed prior to HCF for 6.0.
>
>
> Yes, it was removed, but nothing stops you from creating the install
> script and putting it in tarball, you don't need any changes in the
> current implementation.

how would it be executed? the plugin loading done by fuel-client
doesn't cover this.

>
> The reasons why it was done this way, see in separate mailing thread [1].
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2014-October/049073.html
>
>>
>>
>> Looking into the implementation of the install routine [3] to
>> implement [2], I see that the fuelclient is extracting the tar blindly
>> (more on that at #3) on the executor system that fuelclient is being
>> executed from. Problems with this include 1) the fuelclient may not
>> root be privileged (like in Mirantis OpenStack Express) 2) the
>> fuelclient may not be running on the same system as nailgun 3) we are
>> just calling .extractall on the tarball, this means that we haven't
>> done any validation on the files coming out of the tarball. We need to
>> validate that 3.a) the tarball was actually encoded with the right
>> base path 3.b) that the tasks.yaml file is validated and all the noted
>> scripts are found. Really, the install of the plugin should be handled
>> by the nailgun side to help with 1,2.
>
>
> 1. if you have custom installation you have to provide custom permissions
>     for /var/www/nailgun/plugins directory
> 2. you are absolutely right, see the thread from above why we decided to add
>     this feature even if it was a wrong decision from architecture point of
> view
> 3. "haven't done any validation" - not exactly, validation is done on plugin
>     building stage, also we have simple validation on plugin installation
> stage on
>     Nailgun side (that data are consistent from nailgun point of view).
> There are
>     several reasons why it was done mainly on fuel-plugin-builder side:
>       a. plugin is validated before it's installed (it dramatically
> simplifies development)
>       b. also you can check that plugin is valid without plugin building,
>           use 'fpb --check fuel_plugin_name' parameter
>       c. faster fixes delivery, if there is a bug in validation (we had
> several of them
>           during the development in fuel-plugin-builder), we cannot just
> release new
>           version of fuel, but we can do it with fuel-plugin-builder, we had
> 2 releases [1].
>           For more complicated structures you will have bugs in validation
> for sure.
>       d. if we decide to support validations on both sides, we will come up
> with a lot of bugs
>           which are related to desynchronization of validators between
> Nailgun and fuel-plugin-builder

the main validation points that should be done by nailgun is to verify
that the paths are correct. i.e.
* the tar ./<folder> == metadata.yaml['name']
* tasks.yaml + metadata.yaml refer to valid paths for "cmd",
"deployment_scripts_path", "repository_path"

Rright now there is no contract between the user building the plugin
with fpb, vs adding all the files to a tarball. if fpb is supposed to
be doing this, then there should be some form of signature that can be
parsed to ensure that these items have been pre-validated and the
package wasn't modified, or built by hand. Something that would be
easy, and cheap would be something like 'cat metdata.yaml tasks.yaml |
md5sum >md5sum' and validate this when we load the package. It also
gives us a starting point for other signers.

Alternatly, we would use fpb to validate the package prior to
installing it into nailgun.

>
> [1]
> https://github.com/stackforge/fuel-plugins/blob/master/fuel_plugin_builder/CHANGELOG.md
>
>>
>>
>> Whats next?
>> There are many parts of PA that need to be extended, I think that
>> these are the ones that we must tackle next to cover the most cases
>> a) plugin packaging: it appears that non of the "core plugins" (those
>> in fuel-plugins) are bundled into the iso.
>> b) plugin signing: we cant have "core plugins" with out some method of
>> testing, certifying, and signing them so that we can know that they
>> are trusted.
>>
>> with the help of granular roles:
>> c) the ability to replace or add new granular roles
>> d) the ability to add or modify real roles
>>
>> with the help of advanced networks:
>> e) add new network roles
>>
>> At some point soon, we also need to discuss making it easier to find a
>> catalog of modules and pull them from it, but this is less important
>> than the above
>>
>> [1]
>> https://review.openstack.org/#/c/125608/15..16/specs/6.0/cinder-neutron-plugins-in-fuel.rst
>> [2] https://bugs.launchpad.net/fuel/+bug/1395228
>> [3]
>> https://github.com/stackforge/fuel-web/blob/master/fuelclient/fuelclient/objects/plugins.py#L49
>>
>> --
>> Andrew
>> Mirantis
>> Ceph community
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Andrew
Mirantis
Ceph community