Open Stack

On Wed, Jan 13, 2016 at 1:15 PM, Steven Dake (stdake) <stdake at cisco.com> wrote:
> Hey folks,
>
> I'd like to have a mailing list discussion about logistics of the ELKSTACK
> solution that Alicja has sorted out vs the Heka implementation that Eric is
> proposing.
>
> My take on that is Eric wants to replace rsyslog and logstash with Heka.


See my other email on this point.  At this point, given the
requirements we have (get logs from services that only speak syslog
and write logs to local files), we cannot guarantee that Heka will
replace Rsyslog.  We are going to test the use of Heka's UdpInput
(with "net" set to "unixgram") et FileOutput plugins for that.  Stay
tuned!


> That seems fine, but I want to make certain this doesn't happen in a way
> that leaves Kolla completely non-functional as we finish up Mitaka.  Liberty
> is the first version of Kolla people will deploy, and Mitaka is the first
> version of Kolla people will upgrade to, so making sure that we don't
> completely bust diagnostics (and I recognize diags as is are a little weak
> is critical).
>
> It sounds like from my reading of the previous thread on this topic, unless
> there is some intractable problem, our goal is to use Heka to replace
> resyslog and logstash.  I'd ask inc0 (who did the rsyslog work) and Alicja
> (who did the elkstack work) to understand that replacement often happens on
> work that has already been done, and its not a "waste of time" so to speak
> as an evolution of the system.
>
> Here are the deadlines:
> http://docs.openstack.org/releases/schedules/mitaka.html
>
> Let me help decode that for folks. March 4th is the final deadline to have a
> completely working solution based upon Heka if its to enter Mitaka.


Understood.


>
> Unlike previous releases of Kolla, I want to hand off release management of
> Kolla to the release management team, and to do that, we need to show a
> track record of hitting our deadlines and not adding features past feature
> freeze (the m3 milestone on March 4th).  In the past releases of Kolla we as
> a team were super loose on this requirement – going forward I prefer us
> being super strict.  Handing off to release management is a sign of maturity
> and would have an overall positive impact, assuming we can get the software
> written in time :)
>
> Eric,
>
> I'd like a plan and commitment to either hit Mitaka 3, or the N cycle.  It
> must work well first on Ansible, and second on Mesos.  If it doesn't work at
> all on Mesos, I could live with that -  I think the Mesos implementation
> will really not be ready for prime time until the middle or completion of
> the N cycle.  We lead with Ansible, and I don't see that changing any time
> soon – as a result, I want our Ansible deployment to be rock solid and
> usable out of the gate.  I don't expect to "Market" Mitaka Mesos (with the
> OpenStack foundation's help) as "production ready" but rather as "tech
> preview" and something for folks to evaluate.


It is our intent to meet the March 4th deadline.



>
> Alicja,
>
> I think a parallel development effort with the ELKSTACK that your working on
> makes sense.  In case the Heka development fails entirely, or misses Mitaka
> 3, I don't want us left lacking a diagnostics solution for Mitaka.
> Diagnostics is my priority #2 for Kolla (#1 is upgrades).  Unfortunately
> what this means is you may end up wasting your time doing development that
> is replaced at the last minute in Mitaka 3, or later in the N cycle.  This
> is very common in software development (all the code I wrote for Magnum has
> been sadly replaced).  I know you can be a good team player here and take
> one for the team so to speak, but I'm asking you if you would take offense
> to this approach.


I'd like to moderate this a bit.  We want to build on Alicja's work,
and we will reuse everything that Alicja has done/will do on
Elasticsearch and Kibana, as this part of the stack will be the same.



>
> I'd like comments/questions/concerns on the above logistics approach
> discussed, and a commitment from Eric as to when he thinks all the code
> would land as one patch stream unit.
>
> I'd also like to see the code come in as one super big patch stream (think
> 30 patches in the stream) so the work can be evaluated and merged as one
> unit.  I could also live with 2-3 different patch streams with 10-15 patches
> per stream, just so we can eval as a unit.  This means lots of rebasing on
> your part Eric ;-)  It also means a commitment from the core reviewer team
> to test and review this critical change.  If there isn't a core reviewer on
> board with this approach, please speak up now.


Makes total sense to me.


Thanks.