From kennelson11 at gmail.com  Mon Oct  1 23:57:30 2018
From: kennelson11 at gmail.com (Kendall Nelson)
Date: Mon, 1 Oct 2018 16:57:30 -0700
Subject: [OpenStack-Infra] Berlin Community Contributor Awards
In-Reply-To: <CAJ6yrQgAgYPobNwZiTSTLDJQEcoMWo1C8ERg3mx36ZMWgqhb4Q@mail.gmail.com>
References: <CAJ6yrQgAgYPobNwZiTSTLDJQEcoMWo1C8ERg3mx36ZMWgqhb4Q@mail.gmail.com>
Message-ID: <CAJ6yrQiKzhk4qZi1RoRhb8Ff2BzOgFz6c9yxRLpX5Odqyuyepw@mail.gmail.com>

Hello :)

I wanted to bring this to the top of people's inboxes as we have three
weeks left to submit community members[1].

I can think of a dozen people right now that deserve an award and I am sure
you all could do the same. It only takes a few minutes and its an easy way
to make sure they get the recognition they deserve. Show your appreciation
and nominate one person.

-Kendall (diablo_rojo)

[1] https://openstackfoundation.formstack.com/forms/berlin_stein_ccas

On Fri, Aug 24, 2018 at 11:15 AM Kendall Nelson <kennelson11 at gmail.com>
wrote:

> Hello Everyone!
>
> As we approach the Summit (still a ways away thankfully), I thought I
> would kick off the Community Contributor Award nominations early this
> round.
>
> For those of you that already know what they are, here is the form[1].
>
> For those of you that have never heard of the CCA, I'll briefly explain
> what they are :) We all know people in the community that do the dirty
> jobs, we all know people that will bend over backwards trying to help
> someone new, we all know someone that is a savant in some area of the code
> we could never hope to understand. These people rarely get the thanks they
> deserve and the Community Contributor Awards are a chance to make sure they
> know that they are appreciated for the amazing work they do and skills they
> have.
>
> So go forth and nominate these amazing community members[1]! Nominations
> will close on October 21st at 7:00 UTC and winners will be announced at the
> OpenStack Summit in Berlin.
>
> -Kendall (diablo_rojo)
>
> [1] https://openstackfoundation.formstack.com/forms/berlin_stein_ccas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20181001/66836372/attachment.html>

From graham.whaley at intel.com  Wed Oct  3 10:16:42 2018
From: graham.whaley at intel.com (Whaley, Graham)
Date: Wed, 3 Oct 2018 10:16:42 +0000
Subject: [OpenStack-Infra] Adding index and views/dashboards for Kata to ELK
	stack
Message-ID: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>

Hi Infra team.

First, a brief overview for folks who have not been in this loop.
Kata Containers has a CI that runs some metrics, and spits out JSON results. We'd like to store those results in the OSF logstash ELK infra (http://logstash.openstack.org/#/dashboard/file/logstash.json), and set up some Kibana views and dashboards so we can monitor historical trends and project health (and longer term maybe some advanced trend regression triggers).

There is a relevant github Issue/thread showing a working PoC here:
https://github.com/kata-containers/ci/issues/60#issuecomment-426579084

I believe we are at the point where we should set up a trial index and keys (for the filebeat/logstash/elastic injection) etc. so we can start to tie the parts together.
What do I need to make that next step happen? Do I need to open a formal request ticket/Issue somewhere, or can we just thrash it out here?

This email is part aimed at ClarkB :-), but obviously may involve others etc. Input most welcome.

Thanks!

 Graham

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.



From sekharvajjula at gmail.com  Fri Oct  5 13:34:36 2018
From: sekharvajjula at gmail.com (chandra sekhar)
Date: Fri, 5 Oct 2018 16:34:36 +0300
Subject: [OpenStack-Infra] Fwd: Support needed for setting up ThirdParty CI
	testing Ironic
In-Reply-To: <CAFiN7xovVfWkRVgCSGXagp63cOFdqoPj1t5oo1kisSeCSjz7Gg@mail.gmail.com>
References: <CAFiN7xovVfWkRVgCSGXagp63cOFdqoPj1t5oo1kisSeCSjz7Gg@mail.gmail.com>
Message-ID: <CAFiN7xpT85DaUpvknsM=+FQuNe_w0OAidCo8ZqGr0A-pPpgB9g@mail.gmail.com>

Hi,

I am Chandra, from Nokia. I am trying to setup Third party CI to run Ironic
testcases on Nokia Hardware (
https://wiki.openstack.org/wiki/ThirdPartySystems/Nokia_Airframe_CI).

During PTG Denver, I was instructed by openstack Infra and Ironic teams to
use zuul jobs from ironic as base (
https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L2).

When I do so, I get hundreds of project dependency failures. I took help
from #sambetts, in sorting dependent openstack projects into 2 groups (some
excluding jobs and some including them).

But for some reason, when I test this with a review in openstack/sandbox,
my change appears in zuul pipeline momentarily and vanishes. In logs I get
the below line



2018-10-05 12:19:54,222 DEBUG zuul.layout: Variant <Job pctp-base branches:
None source:
sekharvajjula/project-config-third-party-base.git/zuul.d/base-jobs.yaml at master#2>
matched <Change 0x7f0eec13abe0 607851,7>

2018-10-05 12:19:54,222 DEBUG zuul.layout: Collecting job variants for
ironic-base

2018-10-05 12:19:54,223 DEBUG zuul.layout: Variant <Job ironic-base
branches: {MatchAny:{BranchMatcher:master}} source:
openstack/ironic/zuul.d/ironic-jobs.yaml at master#1> matched <Change
0x7f0eec13abe0 607851,7>

2018-10-05 12:19:54,223 DEBUG zuul.layout: Collecting job variants for
devstack-tempest

2018-10-05 12:19:54,223 DEBUG zuul.layout: *No matching parents for job
devstack-tempest and change* <Change 0x7f0eec13abe0 607851,7>

2018-10-05 12:19:54,223 DEBUG zuul.layout: Collected jobs pctp-base for
<Change 0x7f0eec13abe0 607851,7>

2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: Reporting
change <Change 0x7f0eec13abe0 607851,7>

2018-10-05 12:19:54,223 DEBUG zuul.layout: Project <ProjectConfig
review.openstack.org/openstack-dev/sandbox source:
sekharvajjula/project-config-third-party-base.git/zuul.d/projects.yaml at master
None> matched item <QueueItem 0x7f0f3007cc50 for <Change 0x7f0eec13abe0
607851,7> in check>

2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: No jobs for
change <Change 0x7f0eec13abe0 607851,7>

2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: Removing
change <Change 0x7f0eec13abe0 607851,7> from queue



Zuul is not able to load devstack-tempest job from “openstack/tempest”,
even though I have declared it in /etc/zuul/mainl.yaml.

My base project is
https://github.com/sekharvajjula/project-config-third-party-base



PFA, logs and config files.



Can someone help me in troubleshooting this problem?



Regards,

Chandra R
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20181005/3fc159e3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: thirdpartyci.tgz
Type: application/gzip
Size: 21914601 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20181005/3fc159e3/attachment-0001.bin>

From cboylan at sapwetik.org  Fri Oct  5 20:14:21 2018
From: cboylan at sapwetik.org (Clark Boylan)
Date: Fri, 05 Oct 2018 13:14:21 -0700
Subject: [OpenStack-Infra] Adding index and views/dashboards for Kata to
 ELK stack
In-Reply-To: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
References: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
Message-ID: <1538770461.3962611.1532336704.65E4C9BC@webmail.messagingengine.com>

On Wed, Oct 3, 2018, at 3:16 AM, Whaley, Graham wrote:
> Hi Infra team.
> 
> First, a brief overview for folks who have not been in this loop.
> Kata Containers has a CI that runs some metrics, and spits out JSON 
> results. We'd like to store those results in the OSF logstash ELK infra 
> (http://logstash.openstack.org/#/dashboard/file/logstash.json), and set 
> up some Kibana views and dashboards so we can monitor historical trends 
> and project health (and longer term maybe some advanced trend regression 
> triggers).
> 
> There is a relevant github Issue/thread showing a working PoC here:
> https://github.com/kata-containers/ci/issues/60#issuecomment-426579084
> 
> I believe we are at the point where we should set up a trial index and 
> keys (for the filebeat/logstash/elastic injection) etc. so we can start 
> to tie the parts together.
> What do I need to make that next step happen? Do I need to open a formal 
> request ticket/Issue somewhere, or can we just thrash it out here?

Here and in code review is fine.

> 
> This email is part aimed at ClarkB :-), but obviously may involve others 
> etc. Input most welcome.

Currently all of our jobs submit jobs to the logstash processing system via the ansible role here [0]. That is then fed through our logstash configuration [1]. The first step in this is probably to update the logstash configuration and update the existing Kata jobs in zuul to submit jobs to index that information as appropriate?

As for Kibana views I'm not sure we ever sorted that out with the in browser version of kibana we are running. I think we can embed dashboard information in the kibana "source" or inject it into elasticsearch? This piece would take some investigation as I don't know what needs to be done off the top of my head. Note that our elasticsearch can't be written to via the public interface to it as there are no authentication and authorization controls for elasticsearch.

[0] https://git.openstack.org/cgit/openstack-infra/project-config/tree/roles/submit-logstash-jobs/defaults/main.yaml
[1] https://git.openstack.org/cgit/openstack-infra/logstash-filters/tree/filters/openstack-filters.conf

Hope this helps,
Clark


From cboylan at sapwetik.org  Fri Oct  5 20:19:10 2018
From: cboylan at sapwetik.org (Clark Boylan)
Date: Fri, 05 Oct 2018 13:19:10 -0700
Subject: [OpenStack-Infra] Fwd: Support needed for setting up ThirdParty
 CI testing Ironic
In-Reply-To: <CAFiN7xpT85DaUpvknsM=+FQuNe_w0OAidCo8ZqGr0A-pPpgB9g@mail.gmail.com>
References: <CAFiN7xovVfWkRVgCSGXagp63cOFdqoPj1t5oo1kisSeCSjz7Gg@mail.gmail.com>
 <CAFiN7xpT85DaUpvknsM=+FQuNe_w0OAidCo8ZqGr0A-pPpgB9g@mail.gmail.com>
Message-ID: <1538770750.3963880.1532340840.0B2B0491@webmail.messagingengine.com>

On Fri, Oct 5, 2018, at 6:34 AM, chandra sekhar wrote:
> Hi,
> 
> I am Chandra, from Nokia. I am trying to setup Third party CI to run Ironic
> testcases on Nokia Hardware (
> https://wiki.openstack.org/wiki/ThirdPartySystems/Nokia_Airframe_CI).
> 
> During PTG Denver, I was instructed by openstack Infra and Ironic teams to
> use zuul jobs from ironic as base (
> https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L2).
> 
> When I do so, I get hundreds of project dependency failures. I took help
> from #sambetts, in sorting dependent openstack projects into 2 groups (some
> excluding jobs and some including them).
> 
> But for some reason, when I test this with a review in openstack/sandbox,
> my change appears in zuul pipeline momentarily and vanishes. In logs I get
> the below line
> 
> 
> 
> 2018-10-05 12:19:54,222 DEBUG zuul.layout: Variant <Job pctp-base 
> branches:
> None source:
> sekharvajjula/project-config-third-party-base.git/zuul.d/base-
> jobs.yaml at master#2>
> matched <Change 0x7f0eec13abe0 607851,7>
> 
> 2018-10-05 12:19:54,222 DEBUG zuul.layout: Collecting job variants for
> ironic-base
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.layout: Variant <Job ironic-base
> branches: {MatchAny:{BranchMatcher:master}} source:
> openstack/ironic/zuul.d/ironic-jobs.yaml at master#1> matched <Change
> 0x7f0eec13abe0 607851,7>
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.layout: Collecting job variants for
> devstack-tempest
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.layout: *No matching parents for job
> devstack-tempest and change* <Change 0x7f0eec13abe0 607851,7>
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.layout: Collected jobs pctp-base for
> <Change 0x7f0eec13abe0 607851,7>
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: Reporting
> change <Change 0x7f0eec13abe0 607851,7>
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.layout: Project <ProjectConfig
> review.openstack.org/openstack-dev/sandbox source:
> sekharvajjula/project-config-third-party-base.git/zuul.d/projects.yaml at master
> None> matched item <QueueItem 0x7f0f3007cc50 for <Change 0x7f0eec13abe0
> 607851,7> in check>
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: No jobs for
> change <Change 0x7f0eec13abe0 607851,7>
> 
> 2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: Removing
> change <Change 0x7f0eec13abe0 607851,7> from queue
> 
> 
> 
> Zuul is not able to load devstack-tempest job from “openstack/tempest”,
> even though I have declared it in /etc/zuul/mainl.yaml.

Zuul is looking for the parent job of devstack-tempest which is the devstack and is defined in the openstack-dev/devstack project. You will need to include the repos that define these jobs in your main.yaml or override and define the jobs locally.

> 
> My base project is
> https://github.com/sekharvajjula/project-config-third-party-base
> 
> 
> 
> PFA, logs and config files.
> 
> 
> 
> Can someone help me in troubleshooting this problem?
> 
> 
> 
> Regards,
> 
> Chandra R



From pabelanger at redhat.com  Fri Oct  5 20:28:33 2018
From: pabelanger at redhat.com (Paul Belanger)
Date: Fri, 5 Oct 2018 16:28:33 -0400
Subject: [OpenStack-Infra] Adding index and views/dashboards for Kata to
 ELK stack
In-Reply-To: <1538770461.3962611.1532336704.65E4C9BC@webmail.messagingengine.com>
References: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
 <1538770461.3962611.1532336704.65E4C9BC@webmail.messagingengine.com>
Message-ID: <20181005202833.GA20284@localhost.localdomain>

On Fri, Oct 05, 2018 at 01:14:21PM -0700, Clark Boylan wrote:
> On Wed, Oct 3, 2018, at 3:16 AM, Whaley, Graham wrote:
> > Hi Infra team.
> > 
> > First, a brief overview for folks who have not been in this loop.
> > Kata Containers has a CI that runs some metrics, and spits out JSON 
> > results. We'd like to store those results in the OSF logstash ELK infra 
> > (http://logstash.openstack.org/#/dashboard/file/logstash.json), and set 
> > up some Kibana views and dashboards so we can monitor historical trends 
> > and project health (and longer term maybe some advanced trend regression 
> > triggers).
> > 
> > There is a relevant github Issue/thread showing a working PoC here:
> > https://github.com/kata-containers/ci/issues/60#issuecomment-426579084
> > 
> > I believe we are at the point where we should set up a trial index and 
> > keys (for the filebeat/logstash/elastic injection) etc. so we can start 
> > to tie the parts together.
> > What do I need to make that next step happen? Do I need to open a formal 
> > request ticket/Issue somewhere, or can we just thrash it out here?
> 
> Here and in code review is fine.
> 
> > 
> > This email is part aimed at ClarkB :-), but obviously may involve others 
> > etc. Input most welcome.
> 
> Currently all of our jobs submit jobs to the logstash processing system via the ansible role here [0]. That is then fed through our logstash configuration [1]. The first step in this is probably to update the logstash configuration and update the existing Kata jobs in zuul to submit jobs to index that information as appropriate?
> 
> As for Kibana views I'm not sure we ever sorted that out with the in browser version of kibana we are running. I think we can embed dashboard information in the kibana "source" or inject it into elasticsearch? This piece would take some investigation as I don't know what needs to be done off the top of my head. Note that our elasticsearch can't be written to via the public interface to it as there are no authentication and authorization controls for elasticsearch.
> 
> [0] https://git.openstack.org/cgit/openstack-infra/project-config/tree/roles/submit-logstash-jobs/defaults/main.yaml
> [1] https://git.openstack.org/cgit/openstack-infra/logstash-filters/tree/filters/openstack-filters.conf
> 
Yah, i think it would be great is we could use the JJB / grafyaml
workflow here where we store the view / dashboard in yaml then write a
job to push them into the application.  Then we don't need to deal with
public authentication.

-Paul


From sekharvajjula at gmail.com  Mon Oct  8 10:54:05 2018
From: sekharvajjula at gmail.com (chandra sekhar)
Date: Mon, 8 Oct 2018 13:54:05 +0300
Subject: [OpenStack-Infra] Fwd: Support needed for setting up ThirdParty
 CI testing Ironic
In-Reply-To: <1538770750.3963880.1532340840.0B2B0491@webmail.messagingengine.com>
References: <CAFiN7xovVfWkRVgCSGXagp63cOFdqoPj1t5oo1kisSeCSjz7Gg@mail.gmail.com>
 <CAFiN7xpT85DaUpvknsM=+FQuNe_w0OAidCo8ZqGr0A-pPpgB9g@mail.gmail.com>
 <1538770750.3963880.1532340840.0B2B0491@webmail.messagingengine.com>
Message-ID: <CAFiN7xp-xRqc55fOPmG7BqiL-TMieU+Z7KXxmZN2u5-cAbrfmg@mail.gmail.com>

Hi Clark Boylan,
Thanks for your response.

I had openstack-dev/devstack  already added to main.yaml. Still I got these
errors.
You can check it from my main.yaml, which is attached to issue's log.

Regards,
Chandra R

On Fri, Oct 5, 2018 at 11:19 PM Clark Boylan <cboylan at sapwetik.org> wrote:

> On Fri, Oct 5, 2018, at 6:34 AM, chandra sekhar wrote:
> > Hi,
> >
> > I am Chandra, from Nokia. I am trying to setup Third party CI to run
> Ironic
> > testcases on Nokia Hardware (
> > https://wiki.openstack.org/wiki/ThirdPartySystems/Nokia_Airframe_CI).
> >
> > During PTG Denver, I was instructed by openstack Infra and Ironic teams
> to
> > use zuul jobs from ironic as base (
> >
> https://github.com/openstack/ironic/blob/master/zuul.d/ironic-jobs.yaml#L2
> ).
> >
> > When I do so, I get hundreds of project dependency failures. I took help
> > from #sambetts, in sorting dependent openstack projects into 2 groups
> (some
> > excluding jobs and some including them).
> >
> > But for some reason, when I test this with a review in openstack/sandbox,
> > my change appears in zuul pipeline momentarily and vanishes. In logs I
> get
> > the below line
> >
> >
> >
> > 2018-10-05 12:19:54,222 DEBUG zuul.layout: Variant <Job pctp-base
> > branches:
> > None source:
> > sekharvajjula/project-config-third-party-base.git/zuul.d/base-
> > jobs.yaml at master#2>
> > matched <Change 0x7f0eec13abe0 607851,7>
> >
> > 2018-10-05 12:19:54,222 DEBUG zuul.layout: Collecting job variants for
> > ironic-base
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.layout: Variant <Job ironic-base
> > branches: {MatchAny:{BranchMatcher:master}} source:
> > openstack/ironic/zuul.d/ironic-jobs.yaml at master#1> matched <Change
> > 0x7f0eec13abe0 607851,7>
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.layout: Collecting job variants for
> > devstack-tempest
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.layout: *No matching parents for job
> > devstack-tempest and change* <Change 0x7f0eec13abe0 607851,7>
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.layout: Collected jobs pctp-base for
> > <Change 0x7f0eec13abe0 607851,7>
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: Reporting
> > change <Change 0x7f0eec13abe0 607851,7>
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.layout: Project <ProjectConfig
> > review.openstack.org/openstack-dev/sandbox source:
> >
> sekharvajjula/project-config-third-party-base.git/zuul.d/projects.yaml at master
> > None> matched item <QueueItem 0x7f0f3007cc50 for <Change 0x7f0eec13abe0
> > 607851,7> in check>
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: No jobs for
> > change <Change 0x7f0eec13abe0 607851,7>
> >
> > 2018-10-05 12:19:54,223 DEBUG zuul.Pipeline.openstack.check: Removing
> > change <Change 0x7f0eec13abe0 607851,7> from queue
> >
> >
> >
> > Zuul is not able to load devstack-tempest job from “openstack/tempest”,
> > even though I have declared it in /etc/zuul/mainl.yaml.
>
> Zuul is looking for the parent job of devstack-tempest which is the
> devstack and is defined in the openstack-dev/devstack project. You will
> need to include the repos that define these jobs in your main.yaml or
> override and define the jobs locally.
>
> >
> > My base project is
> > https://github.com/sekharvajjula/project-config-third-party-base
> >
> >
> >
> > PFA, logs and config files.
> >
> >
> >
> > Can someone help me in troubleshooting this problem?
> >
> >
> >
> > Regards,
> >
> > Chandra R
>
>
> _______________________________________________
> OpenStack-Infra mailing list
> OpenStack-Infra at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20181008/e3c4518d/attachment-0001.html>

From cboylan at sapwetik.org  Wed Oct 10 17:45:58 2018
From: cboylan at sapwetik.org (Clark Boylan)
Date: Wed, 10 Oct 2018 10:45:58 -0700
Subject: [OpenStack-Infra] OpenDev messaging meeting October 22 at 1800UTC
Message-ID: <1539193558.1563477.1537538640.66620387@webmail.messagingengine.com>

Hello everyone,

I spoke with the marketing and design folks at the foundation about having a meeting to talk about opendev messaging. Due to travel (we all seem to travel a lot), the earliest time that seems to work is 1800UTC Monday the 22nd of October. We've gone ahead and penciled that in and I think we will have the meeting in #openstack-infra as that worked well for the name process brainstorming.

Hopefully this time works well enough for everyone? I know its a bit early as you head west of the US, but it wasn't easy to find a time that worked well for foundation staff.

See you there,
Clark


From fungi at yuggoth.org  Wed Oct 10 18:18:00 2018
From: fungi at yuggoth.org (Jeremy Stanley)
Date: Wed, 10 Oct 2018 18:18:00 +0000
Subject: [OpenStack-Infra] OpenDev messaging meeting October 22 at
	1800UTC
In-Reply-To: <1539193558.1563477.1537538640.66620387@webmail.messagingengine.com>
References: <1539193558.1563477.1537538640.66620387@webmail.messagingengine.com>
Message-ID: <20181010181759.apttg3efxewiteld@yuggoth.org>

On 2018-10-10 10:45:58 -0700 (-0700), Clark Boylan wrote:
[...]
> Due to travel (we all seem to travel a lot), the earliest time
> that seems to work is 1800UTC Monday the 22nd of October.
[...]

I'll be attending All Things Open then, but should hopefully be able
to find a corner somewhere to jump on IRC for this. Thanks for
arranging!
-- 
Jeremy Stanley
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20181010/3e04ea97/attachment.sig>

From cboylan at sapwetik.org  Mon Oct 15 17:43:59 2018
From: cboylan at sapwetik.org (Clark Boylan)
Date: Mon, 15 Oct 2018 10:43:59 -0700
Subject: [OpenStack-Infra] Control Plane Server Upgrade Sprint Planning
In-Reply-To: <1537225743.63161.1511445720.6BC69C13@webmail.messagingengine.com>
References: <1537225743.63161.1511445720.6BC69C13@webmail.messagingengine.com>
Message-ID: <1539625439.42379.1542790472.5B63013E@webmail.messagingengine.com>

On Mon, Sep 17, 2018, at 4:09 PM, Clark Boylan wrote:
> Hello everyone,
> 
> We still have a few Trusty servers hanging around. We should work to 
> upgrade them either to Xenial (using puppet deployment) or Bionic (using 
> ansible and containers?). We've made good progress in the past using a 
> sprint type setup where the majority of us are focused on making 
> progress on this sort of work. Can we plan another sprint to complete 
> this task?
> 
> Looking at a calendar I think many of us are still recovering from PTG 
> travel this week, week after next is Ansiblefest, then I have a local 
> Portland conference the week after that. Long story short October 15-19 
> may be our best week for this. Does that week work? Any other 
> suggestions?
> 
> I think we can (and should) work on this outside of the sprint as well 
> and I will be trying to upgrade the etherpad servers in the near future. 
> Let me know if you are working on upgrading any servers/services and I 
> will do what I can to help review changes and make that happen as well.

I've started on this and all the changes to get etherpad servers running on xenial appear to be in place. Hieradata/ansible group vars have been updated and a new etherpad-dev01.openstack.org has been created.

In the process of doing this I discovered that ansible and puppet were recently (like in the last 8 hours) no longer running. The root cause seems to be citycloud has removed the Lon1 region which broken inventory generation. I've pushed https://review.openstack.org/#/c/610668/ to fix that and will work on getting ansible and puppet running again so that I can finish up with etherpad-dev.openstack.org upgrades.

If you'd like to help out please grab a server or three at https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup and we'll work through the list together.

Thank you,
Clark


From graham.whaley at intel.com  Wed Oct 17 15:50:30 2018
From: graham.whaley at intel.com (Whaley, Graham)
Date: Wed, 17 Oct 2018 15:50:30 +0000
Subject: [OpenStack-Infra] Adding index and views/dashboards for Kata to
	ELK stack
In-Reply-To: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
References: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
Message-ID: <01D2D3EEA433C0419A12152B3A36557E204F18E7@IRSMSX101.ger.corp.intel.com>

Hi Infra folks, et. al.

Maybe this fell through the cracks due to conferences, travel etc., or maybe it was too broad spectrum...
/me picks a pinpoint person or two...

Thierry, Clark - is this something you can help with or facilitate a discussion around?

 Graham

> -----Original Message-----
> From: Whaley, Graham
> Sent: Wednesday, October 3, 2018 11:17 AM
> To: openstack-infra at lists.openstack.org
> Cc: Anne Bertucio <anne at openstack.org>; Ernst, Eric <eric.ernst at intel.com>
> Subject: Adding index and views/dashboards for Kata to ELK stack
> 
> Hi Infra team.
> 
> First, a brief overview for folks who have not been in this loop.
> Kata Containers has a CI that runs some metrics, and spits out JSON results.
> We'd like to store those results in the OSF logstash ELK infra
> (http://logstash.openstack.org/#/dashboard/file/logstash.json), and set up
> some Kibana views and dashboards so we can monitor historical trends and
> project health (and longer term maybe some advanced trend regression
> triggers).
> 
> There is a relevant github Issue/thread showing a working PoC here:
> https://github.com/kata-containers/ci/issues/60#issuecomment-426579084
> 
> I believe we are at the point where we should set up a trial index and keys (for
> the filebeat/logstash/elastic injection) etc. so we can start to tie the parts
> together.
> What do I need to make that next step happen? Do I need to open a formal
> request ticket/Issue somewhere, or can we just thrash it out here?
> 
> This email is part aimed at ClarkB :-), but obviously may involve others etc. Input
> most welcome.
> 
> Thanks!
> 
>  Graham

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.



From cboylan at sapwetik.org  Thu Oct 18 16:10:07 2018
From: cboylan at sapwetik.org (Clark Boylan)
Date: Thu, 18 Oct 2018 09:10:07 -0700
Subject: [OpenStack-Infra] Adding index and views/dashboards for Kata to
	ELK stack
In-Reply-To: <01D2D3EEA433C0419A12152B3A36557E204F18E7@IRSMSX101.ger.corp.intel.com>
References: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
 <01D2D3EEA433C0419A12152B3A36557E204F18E7@IRSMSX101.ger.corp.intel.com>
Message-ID: <1539879007.3697557.1546669152.55C51D64@webmail.messagingengine.com>

On Wed, Oct 17, 2018, at 8:50 AM, Whaley, Graham wrote:
> Hi Infra folks, et. al.
> 
> Maybe this fell through the cracks due to conferences, travel etc., or 
> maybe it was too broad spectrum...
> /me picks a pinpoint person or two...
> 
> Thierry, Clark - is this something you can help with or facilitate a 
> discussion around?

Paul [0] and I [1] had both responded a while back with some thoughts on next steps to make this possible, but those only went to the infra mailing list (I didn't check if you were subscribed). Hopefully the steps there make sense and we can move forward with something along those lines?

[0] http://lists.openstack.org/pipermail/openstack-infra/2018-October/006206.html
[1] http://lists.openstack.org/pipermail/openstack-infra/2018-October/006204.html

Clark


From graham.whaley at intel.com  Fri Oct 19 10:36:21 2018
From: graham.whaley at intel.com (Whaley, Graham)
Date: Fri, 19 Oct 2018 10:36:21 +0000
Subject: [OpenStack-Infra] Adding index and views/dashboards for Kata to
	ELK stack
In-Reply-To: <1539879007.3697557.1546669152.55C51D64@webmail.messagingengine.com>
References: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
 <01D2D3EEA433C0419A12152B3A36557E204F18E7@IRSMSX101.ger.corp.intel.com>
 <1539879007.3697557.1546669152.55C51D64@webmail.messagingengine.com>
Message-ID: <01D2D3EEA433C0419A12152B3A36557E204F3B6D@IRSMSX101.ger.corp.intel.com>

> Paul [0] and I [1] had both responded a while back with some thoughts on next
> steps to make this possible, but those only went to the infra mailing list (I didn't
> check if you were subscribed). Hopefully the steps there make sense and we can
> move forward with something along those lines?

Ah, mea culpa! - I should have made it clearer I was not subscribed to the list, and pushed myself onto the CC. Apologies for that.

Let's tackle what looks like the easier one first:

> --- copied from the list ---
> Yah, i think it would be great is we could use the JJB / grafyaml
> workflow here where we store the view / dashboard in yaml then write a
> job to push them into the application.  Then we don't need to deal with
> public authentication.

If I understand then, we'd set up a kata git repo (or somewhere in our existing kata CI repo) to store the grafyaml files that define our views, and upon merges, a JJB defined Zuul job would take those, convert them to JSON and inject them into the grafana index of the elastic DB - that sounds nice to me, and I think should be workable. I like that flow.

For the data injection itself then:
> --- copied from the list ---
> Currently all of our jobs submit jobs to the logstash processing system via the ansible role here [0]. That is then fed through our logstash configuration [1]. The first step > in this is probably to update the logstash configuration and update the existing Kata jobs in zuul to submit jobs to index that information as appropriate?

Right, I think the logstash config will need updating to understand there is metrics JSON being injected, and to pass it on. We'll have to see how we get the filter for the JSON into that setup, and if we need (or can) have the filebeat inject a specific tag for instance.

I don't think the Zuul Ansible role will be applicable - the metrics run on bare metal machines running Jenkins, and export their JSON results via a filebeat socket. My theory was we'd then add the socket input to the logstash server to receive from that filebeat - as in my gist at https://gist.github.com/grahamwhaley/aa730e6bbd6a8ceab82129042b186467

One crux here is that the metrics have to run on a machine with guaranteed performance (so not a shared/virtual cloud instance), and hence currently run under Jenkins and not on the OSF/Zuul CI infra.

Let me know you see any issues with that Jenkins/filebeat/socket/JSON flow.

I need to deploy a new machine to process master branch merges to generate the data (currently we have a machine that is processing PRs at submission, not merge, which is not the data we want to track long term). I'll let you know when I have that up and running. If we wanted to move on this earlier, then I could inject data to a test index from my local test setup - all it would need I believe is the valid keys for the filebeat->logstash connection.

> Clark
Thanks!
  Graham (now on copy ;-)
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

From cboylan at sapwetik.org  Tue Oct 23 17:02:37 2018
From: cboylan at sapwetik.org (Clark Boylan)
Date: Tue, 23 Oct 2018 10:02:37 -0700
Subject: [OpenStack-Infra] Adding index and views/dashboards for Kata to
	ELK stack
In-Reply-To: <01D2D3EEA433C0419A12152B3A36557E204F3B6D@IRSMSX101.ger.corp.intel.com>
References: <01D2D3EEA433C0419A12152B3A36557E204E9663@IRSMSX101.ger.corp.intel.com>
 <01D2D3EEA433C0419A12152B3A36557E204F18E7@IRSMSX101.ger.corp.intel.com>
 <1539879007.3697557.1546669152.55C51D64@webmail.messagingengine.com>
 <01D2D3EEA433C0419A12152B3A36557E204F3B6D@IRSMSX101.ger.corp.intel.com>
Message-ID: <1540314157.1272773.1551970776.0C80340F@webmail.messagingengine.com>

On Fri, Oct 19, 2018, at 3:36 AM, Whaley, Graham wrote:
> > Paul [0] and I [1] had both responded a while back with some thoughts on next
> > steps to make this possible, but those only went to the infra mailing list (I didn't
> > check if you were subscribed). Hopefully the steps there make sense and we can
> > move forward with something along those lines?
> 
> Ah, mea culpa! - I should have made it clearer I was not subscribed to 
> the list, and pushed myself onto the CC. Apologies for that.
> 
> Let's tackle what looks like the easier one first:
> 
> > --- copied from the list ---
> > Yah, i think it would be great is we could use the JJB / grafyaml
> > workflow here where we store the view / dashboard in yaml then write a
> > job to push them into the application.  Then we don't need to deal with
> > public authentication.
> 
> If I understand then, we'd set up a kata git repo (or somewhere in our 
> existing kata CI repo) to store the grafyaml files that define our 
> views, and upon merges, a JJB defined Zuul job would take those, convert 
> them to JSON and inject them into the grafana index of the elastic DB - 
> that sounds nice to me, and I think should be workable. I like that 
> flow.
> 

You could put them in our existing grafyaml config, https://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana, though this grafana is only configured to talk to our graphite server, http://graphite.openstack.org. This gets rendered at http://grafana.openstack.org. We can probably update that grafana instance to also look at an elasticsearch index (I'm not super familiar with how grafana is configured and if you can have multiple datastores in use at the same time).

> For the data injection itself then:
> > --- copied from the list ---
> > Currently all of our jobs submit jobs to the logstash processing system via the ansible role here [0]. That is then fed through our logstash configuration [1]. The first step > in this is probably to update the logstash configuration and update the existing Kata jobs in zuul to submit jobs to index that information as appropriate?
> 
> Right, I think the logstash config will need updating to understand 
> there is metrics JSON being injected, and to pass it on. We'll have to 
> see how we get the filter for the JSON into that setup, and if we need 
> (or can) have the filebeat inject a specific tag for instance.
> 
> I don't think the Zuul Ansible role will be applicable - the metrics run 
> on bare metal machines running Jenkins, and export their JSON results 
> via a filebeat socket. My theory was we'd then add the socket input to 
> the logstash server to receive from that filebeat - as in my gist at 
> https://gist.github.com/grahamwhaley/aa730e6bbd6a8ceab82129042b186467

I don't think we would want to expose write access to the unauthenticated logstash and elasticsearch system to external systems. The thing that makes this secure today is we (community infrastructure team) control the existing writers. The existing writers are available for your use (see below) should you decide to use them.

> 
> One crux here is that the metrics have to run on a machine with 
> guaranteed performance (so not a shared/virtual cloud instance), and 
> hence currently run under Jenkins and not on the OSF/Zuul CI infra.

Zuul (by way of Nodepool) can speak to arbitrary machines as long as they speak an ansible connection protocol. In this case the default of ssh would probably work when tied to nodepool's static instance driver. The community infrastructure happens to only talk to cloud VMs today because that is what we have been given access to, but should be able to talk to other resources if people show up with them.

> 
> Let me know you see any issues with that Jenkins/filebeat/socket/JSON flow.
> 
> I need to deploy a new machine to process master branch merges to 
> generate the data (currently we have a machine that is processing PRs at 
> submission, not merge, which is not the data we want to track long 
> term). I'll let you know when I have that up and running. If we wanted 
> to move on this earlier, then I could inject data to a test index from 
> my local test setup - all it would need I believe is the valid keys for 
> the filebeat->logstash connection.
> 
> > Clark
> Thanks!
>   Graham (now on copy ;-)

Ideally we'd make use of the existing community infrastructure as much as possible to make this sustainable and secure. We are happy to modify our existing tooling as necessary to do this. Update the logstash configuration, add Nodepool resources, have grafana talk to elasticsearch, and so on.

Clark