[openstack-dev] [sahara] PTG Summary

Jeremy Freudberg jeremyfreudberg at gmail.com
Mon Mar 12 15:56:49 UTC 2018


Thanks Luigi and Telles for striving to work hard, even in my absence. :)
Some notes inline.

On Mon, Mar 12, 2018 at 10:39 AM, Telles Nobrega <tenobreg at redhat.com>
wrote:

> Hello Saharans and interested folks,
>
> This PTG was a very interesting experience, for those not familiar with
> what happened there, here goes a quick summary.
>
> The event became known as SnowOpenStack PTG due to some snow that got in
> the way of event. But that didn't get in the way of determined people that
> wanted to do good work anyways.
> This PTG for Sahara was different, we only had 2 people present so a few
> topics couldn't be fully discussed.
>
> We started our week as Sahara joining the Storyboard room in order to
> understand better their status and what are the needed steps for us to
> migrate from launchpad to storyboard.
> The outcome of that meeting was better than expected. There are only a few
> things needed from our side to fully migrate. Tosky got ahead on this and
> already updated the migration script to support sahara and other projects
> needs and tested the migration with success. Now we only need to wait on
> the storyboard team to test it as well and then we need to document that we
> are migrating to storyboard.
> Action item on this for all involved in the project: Take some time to
> read on Storyboard and get familiar with it.
>
> Than we started Sahara meetings:
>
> We started with the traditional retrospective, we looked over the work
> list from Queens, marking them with Done, Partially Done and Unstarted.
> Please all take a look at that and let me know if anything not prioritized
> from that list is necessary.
>
> Now to specific discussion topics.
>
> Plugins upgrades, deprecation and removals
> ------------------------------------------------------------
>
> CDH:
>
>    - Upgrade to 5.12, 5.13 and 5.14
>    - Deprecate 5.7, 5.9 if we are able to upgrade all three version or
>    two of them
>    - Remove 5.5.0
>
> MapR:
>
>    - Upgrade to 6.0 and update latest packages for 5.2
>
> Looks like MapR 6 offers some new management services which we should
acknowledge and add.

>
>    -
>    - Remove all references to 5.1
>
> Ambari:
>
>    - Upgrade Ambari to 2.6 and HDP to 2.6 as well
>    - Use the Ambari 2.6 image with both HDP 2.6 and 2.5 and 2.4
>    - Deprecate HDP version 2.3 and work the removal of ambari 2.4 for S
>
> Spark:
>
>    - Upgrade to 2.3.0 and check 2.2.1 as subversion as 2.2.0
>    - Deprecate 1.6, 2.1 (see move out of trusty discussion)
>    - Remove 1.3
>
> Storm:
>
>    - Upgrade to 1.2.1 and 1.2.0 (under the same tag 1.2)
>    - Deprecate 1.0.1
>    - Remove 0.9.2
>
> Vanilla:
>
>    - Upgrade to vanilla 3.0.0 and check if we can add 2.7.5
>
> 2.7.5 seems much lower priority to me. Even 2.8.3 or the eventual 2.8.4
seems more important. But I'm not sure.


>
> Sahara CI
> -------------
> We are still facing issues with our third-party CI. We plan to have
> nightly jobs running each day for a plugin so we won't need too much
> resources, that seems possible with zuulv3.
> Also there is an issue with experimental queue, now we run all
> experimental jobs even when not necessary, we plan to split this in
> separate queues so we can run specific tests.
>

It looks like the wheels of progress are starting to turn on my end...


>
> Python 3 support
> -----------------------
> Python 2 support is closing and we need to fully migrate to Python 3.
> Right now we have unit tests in place, tempest should be easily resolved;
> Scenarios jobs are there but failing with swift issues. Once we have all of
> those we will have a better grasp of what we need to do and how much work
> will be. In any case, we need to get a jump on these soon.
>
> SSL/TLS everywhere
> ----------------------------
> We need to check our status of secure communication with other projects as
> well as between Sahara and its clusters. Right now Sahara should be able to
> communicate with CDH using SSL but it is hard coded and we need to change
> that and test how it goes. Also we need to check certificate management.
> For Ambari, CA is generated but not reconized on RHEL/CentOS>=7.4, we need
> to make sure this works.
>
> Move out of trusty
> ------------------------
> Trusty support will be dropped soon and we need to make sure we don't have
> any dependencies on our side. For that we need to work on removing plugins
> versions that are only supported on trusty.
>
> Plugins outside Sahara
> -------------------------------
> After discussion with Doug Hellman, seems like we have a good plan to have
> this finally done. The goal is the have a new project (sahara-plugins) that
> will require stuff from sahara and sahara itself will load plugins from
> sahara-plugins project dinamically with stevedore, this way we don't have a
> circular dependency.
> The bulk of the work is done, spec should be on its way soon.
>

+1


>

> APIv2
> --------
> We have it available as experimental, but we miss CLI, tempest client,
> tempest tests and scenarios tests.
>

And a few other notes, which I'll bring up at the virtual PTG.


>
> Boot from volume
> -----------------------
> This spec has been standing there for a long time and we need to get this
> done.
>
> Bugs
> -------
> Spark amount of resources:
>
>    - We need to check if we can define on config file
>    - Set the default to 50% of flavor capacity
>    - If we can get this info from args we won't need to change much, just
>    add some conditions to make sure we get this info and it makes into command
>    line
>
> File copy time out when file is too big:
>
>    - Paramiko PUT is not the ideal solution because it will need to write
>    file locally
>    - Pipelined + buffer also proved an inefficient solution
>    - We will test now putfo + StringIO for file-like object
>    - And if that doesn't work either we will fall back to scp
>
> This email ran longer than expected, more details can be found at
> etherpad[1] along side with our priority list for Rocky.
>
> Let me know if you need help understanding anything at the etherpad.
>
> Thanks for all present, and those who tried their best to help even though
> weren't there.
>
> [1] https://etherpad.openstack.org/p/sahara-rocky-ptg
> --
>
> TELLES NOBREGA
>
> SOFTWARE ENGINEER
>
> Red Hat Brasil  <https://www.redhat.com/>
>
> Av. Brg. Faria Lima, 3900 - 8º andar - Itaim Bibi, São Paulo
>
> tenobreg at redhat.com
> <https://red.ht/sig>
> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>  Red Hat é reconhecida entre as melhores empresas para trabalhar no Brasil
> pelo Great Place to Work.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180312/a4cd9a3d/attachment.html>


More information about the OpenStack-dev mailing list