Open Stack

Hello Saharans and interested folks,

This PTG was a very interesting experience, for those not familiar with
what happened there, here goes a quick summary.

The event became known as SnowOpenStack PTG due to some snow that got in
the way of event. But that didn't get in the way of determined people that
wanted to do good work anyways.
This PTG for Sahara was different, we only had 2 people present so a few
topics couldn't be fully discussed.

We started our week as Sahara joining the Storyboard room in order to
understand better their status and what are the needed steps for us to
migrate from launchpad to storyboard.
The outcome of that meeting was better than expected. There are only a few
things needed from our side to fully migrate. Tosky got ahead on this and
already updated the migration script to support sahara and other projects
needs and tested the migration with success. Now we only need to wait on
the storyboard team to test it as well and then we need to document that we
are migrating to storyboard.
Action item on this for all involved in the project: Take some time to read
on Storyboard and get familiar with it.

Than we started Sahara meetings:

We started with the traditional retrospective, we looked over the work list
from Queens, marking them with Done, Partially Done and Unstarted.
Please all take a look at that and let me know if anything not prioritized
from that list is necessary.

Now to specific discussion topics.

Plugins upgrades, deprecation and removals
------------------------------------------------------------

CDH:

   - Upgrade to 5.12, 5.13 and 5.14
   - Deprecate 5.7, 5.9 if we are able to upgrade all three version or two
   of them
   - Remove 5.5.0

MapR:

   - Upgrade to 6.0 and update latest packages for 5.2
   - Remove all references to 5.1

Ambari:

   - Upgrade Ambari to 2.6 and HDP to 2.6 as well
   - Use the Ambari 2.6 image with both HDP 2.6 and 2.5 and 2.4
   - Deprecate HDP version 2.3 and work the removal of ambari 2.4 for S

Spark:

   - Upgrade to 2.3.0 and check 2.2.1 as subversion as 2.2.0
   - Deprecate 1.6, 2.1 (see move out of trusty discussion)
   - Remove 1.3

Storm:

   - Upgrade to 1.2.1 and 1.2.0 (under the same tag 1.2)
   - Deprecate 1.0.1
   - Remove 0.9.2

Vanilla:

   - Upgrade to vanilla 3.0.0 and check if we can add 2.7.5


Sahara CI
-------------
We are still facing issues with our third-party CI. We plan to have nightly
jobs running each day for a plugin so we won't need too much resources,
that seems possible with zuulv3.
Also there is an issue with experimental queue, now we run all experimental
jobs even when not necessary, we plan to split this in separate queues so
we can run specific tests.

Python 3 support
-----------------------
Python 2 support is closing and we need to fully migrate to Python 3. Right
now we have unit tests in place, tempest should be easily resolved;
Scenarios jobs are there but failing with swift issues. Once we have all of
those we will have a better grasp of what we need to do and how much work
will be. In any case, we need to get a jump on these soon.

SSL/TLS everywhere
----------------------------
We need to check our status of secure communication with other projects as
well as between Sahara and its clusters. Right now Sahara should be able to
communicate with CDH using SSL but it is hard coded and we need to change
that and test how it goes. Also we need to check certificate management.
For Ambari, CA is generated but not reconized on RHEL/CentOS>=7.4, we need
to make sure this works.

Move out of trusty
------------------------
Trusty support will be dropped soon and we need to make sure we don't have
any dependencies on our side. For that we need to work on removing plugins
versions that are only supported on trusty.

Plugins outside Sahara
-------------------------------
After discussion with Doug Hellman, seems like we have a good plan to have
this finally done. The goal is the have a new project (sahara-plugins) that
will require stuff from sahara and sahara itself will load plugins from
sahara-plugins project dinamically with stevedore, this way we don't have a
circular dependency.
The bulk of the work is done, spec should be on its way soon.

APIv2
--------
We have it available as experimental, but we miss CLI, tempest client,
tempest tests and scenarios tests.

Boot from volume
-----------------------
This spec has been standing there for a long time and we need to get this
done.

Bugs
-------
Spark amount of resources:

   - We need to check if we can define on config file
   - Set the default to 50% of flavor capacity
   - If we can get this info from args we won't need to change much, just
   add some conditions to make sure we get this info and it makes into command
   line

File copy time out when file is too big:

   - Paramiko PUT is not the ideal solution because it will need to write
   file locally
   - Pipelined + buffer also proved an inefficient solution
   - We will test now putfo + StringIO for file-like object
   - And if that doesn't work either we will fall back to scp

This email ran longer than expected, more details can be found at
etherpad[1] along side with our priority list for Rocky.

Let me know if you need help understanding anything at the etherpad.

Thanks for all present, and those who tried their best to help even though
weren't there.

[1] https://etherpad.openstack.org/p/sahara-rocky-ptg
-- 

TELLES NOBREGA

SOFTWARE ENGINEER

Red Hat Brasil  <https://www.redhat.com/>

Av. Brg. Faria Lima, 3900 - 8º andar - Itaim Bibi, São Paulo

tenobreg at redhat.com
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
 Red Hat é reconhecida entre as melhores empresas para trabalhar no Brasil
pelo Great Place to Work.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180312/f0d3fb36/attachment.html>