Hello all,
<tl;dr> The Charms projects are increasingly heavy users of external GitHub dependencies, and we are facing intermittent issues with this at the gate.
Does anyone have ideas as to how we should handle this from the point of view of Charm teams?
Anyone from Zuul have ideas/pointers on how we could help improve the external GitHub dependency support? </tl;dr>
As many of you know the OpenStack Charms project is in the process of replacing the framework for performing functional deployment testing of Charms with ``Zaza`` [0].
Two of the key features of the Zaza framework is reusability of tests simply by referencing already written tests with a Python module path in a test definition in a YAML file, and general applicability across other Charms, not just OpenStack specific ones.
Because of this the Zaza project, which also contains the individual functional test modules, is hosted on GitHub and not on the OpenStack Infrastructure.
Whenever a change is proposed to a charm that require new or changes to existing functional tests, we need a effective way for the individual contributor to have their Charm change (which is proposed on OpenStack Infrastructure) tested with code from their Zaza change (which is proposed as a PR on GitHub).
We have had some success with adding ``Depends-On:`` and the full URL to the GitHub PR in the commit message. There is experimental support for using that as a gate check in Zuul, and Canonical's third party Charm CI is configured to pull the correct version of Zaza based on Depends-On referencing GitHub PRs.
However, we often have to go through extra hoops to land things as the gate code appears to not always successfully handle GitHub PR references in Depends-On.
For reference, after discussion in #openstack-infra I got a log excerpt [1] and a reference to a WIP PR [2] that might be relevant.
0: https://zaza.readthedocs.io/en/latest/ 1: http://paste.openstack.org/show/744664/ 2: https://review.openstack.org/#/c/613143/