Open Stack

On 07/03/2014 10:31 AM, Sullivan, Jon Paul wrote:
>> -----Original Message-----
>> From: Anita Kuno [mailto:anteaya at anteaya.info]
>> Sent: 03 July 2014 15:06
>> To: openstack-dev at lists.openstack.org
>> Subject: Re: [openstack-dev] [third-party-ci][neutron] What is "Success"
>> exactly?
> 
> I guess you missed this last time - the mail had gotten quite long :D
> 
I had yes, thanks for drawing my attention to it.
>>>> Hi Jon Paul: (Is it Jon Paul or Jon?)
>>>
>>> Hi Anita - it's Jon-Paul or JP.
>>>
Ah, thanks JP.
>>>>
>>>
>>> But there is a second side to what you were saying which was the
>> developer feedback.  I guess I am suggesting that if you are putting a
>> system in place for developers to vote on the 3rd party CI, should that
>> same system be in effect for the Openstack check/gate jobs?
>>>
>> It already is, it is called #openstack-infra. All day long (the 24 hour
>> day) developers drop in and tell us exactly how they feel about any
>> aspect of OpenStack Infrastructure. They let us know when documentation
>> is confusing, when things are broken, when a patch should have been
>> merged and failed to be, when Zuul is caught in a retest loop and
>> occasionally when we get something right.
> 
> I had presumed this to be the case, and I guess this is the first port of call when developers have questions on 3rd-party CI?  If so, then a very interesting metric that would speak to the reliability of the 3rd CI might be responsiveness to irc questions?
> 
Yes, developers ask questions about what specific 3rd party accounts are
doing when commenting on their patches all the time. Often some version
of "Why is systemx-ci commenting on my patch?" Many of them ask in infra
and many of them ping me directly.

Then we move into some variation of "Systemx-ci is {some behaviour that
does not meet requirements}. {What do I do? | Can someone do something
to fix this? | Can we disable this system?}
Requirements: http://ci.openstack.org/third_party.html#requirements
Open Patches:
https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:third-party,n,z
and
https://review.openstack.org/#/c/104565/

Sure responsiveness to irc questions would be an interesting metric. Now
how to collect data. I suppose you could scrape irc logs - I don't want
to see the regex to parse what is considered to be irc responsiveness.
You could ask the infra team if you like, but then that is a subset of
what I have already suggested for all developers plus puts more work on
infra which I will not voluntarily do, not if we can avoid it. You could
ask me, but my response will be based on an aggregation of my gut
responses based on personal experience with individual admins for
different accounts, it doesn't scale and while I feel it has some
credence should not be the sole source of information for any metric
given the scope of the issue. We currently have 70 gerrit ci accounts,
I'm not going to offer an opinion on accounts I have never interacted
with if everything has been running fine and they have had no reason to
interact with me.

By allowing the developers affected by the third party systems offer
their feedback, a more diverse source of data is collected. Keep in mind
that as a developer I have never had to splunk logs from third party ci
on my patches since the majority of my patches are for infra, which has
very little testing by third party ci. I'd like to have input from
developer who do interact with third party ci artifacts.

>>
>> OpenStack Infra logs can be found here:
>> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/
>>
>> I don't think having an irc channel for third party is practical because
>> it simply will split infra resources and I have my doubts about how
>> responsive folks would be in it. Hence my suggestion of the pages to
>> allow developers to share the kind of information they share in
>> openstack-infra all the time.
> 
> Yes - I can understand your viewpoint on this, and it makes sense to have a forum where developers can raise commetns or concerns and those responsible for the 3rd party CI can respond.
Thanks and hopefully they will respond, and at the very least it will be
a quick way of seeing how many developers have attempted to give
feedback and the speed or lack thereof of a response.

There are some system admins that are very responsive and some are even
beginning to be proactive, by sending an email to the ml (dev and/or
infra) and informing us when their system is failing to build (we have
to get faster at disabling systems in those circumstances, but I
appreciate the proactiveness here) as well as posting when they move
their logs to a url with a dns rather than a hard coded ip address and
that breaks backward compatibility. Thank you for being proactive.
http://lists.openstack.org/pipermail/openstack-infra/2014-July/001473.html
http://lists.openstack.org/pipermail/openstack-dev/2014-July/039270.html

Thanks JP,
Anita.
> 
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> Thanks, 
> Jon-Paul Sullivan ☺ Cloud Services - @hpcloud
> 	
> Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway.
> Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2. 
> Registered Number: 361933
>  
> The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender.
> 
> To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as "HP CONFIDENTIAL".
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>