[OpenStack Infra] Current documentations of OpenStack CI/CD architecture
Hi, Is there any documents somewhere describing the current architecture of the CI/CD system that OpenStack Infrastructure is running? Best regards, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*
On 2019-08-06 16:49:15 +0900 (+0900), Trinh Nguyen wrote:
Is there any documents somewhere describing the current architecture of the CI/CD system that OpenStack Infrastructure is running?
OpenStack uses the OpenDev deployment of the Zuul project gating system for CI/CD. The OpenDev infrastructure sysadmins maintain some operational Zuul deployment documentation for their own purposes at https://docs.openstack.org/infra/system-config/zuul.html and also some information which was written separately during v3 migration at https://docs.openstack.org/infra/system-config/zuulv3.html which is due to get rolled into the first. Zuul itself already has excellent documentation (written by many of the same people) and its main architecture can be found described in the Admin Guide, with the diagram at https://zuul-ci.org/docs/zuul/admin/components.html providing a nice component overview. What specifically are you looking for? -- Jeremy Stanley
Hi Jeremy, Thanks for pointing that out. They're pretty helpful. Sorry for not clarifying the purpose of my question in the first email. Right now my company is using Jenkins for CI/CD which is not scalable and for me it's hard to define job pipeline because of XML. I'm about to build a demonstration for my company using Zuul with Github as a replacement and trying to make sense of the OpenStack deployment of Zuul. I have been working with OpenStack projects for a couple of cycles in which Zuul has shown me its greatness and I think I can bring that power to the company. Bests, On Tue, Aug 6, 2019 at 11:42 PM Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2019-08-06 16:49:15 +0900 (+0900), Trinh Nguyen wrote:
Is there any documents somewhere describing the current architecture of the CI/CD system that OpenStack Infrastructure is running?
OpenStack uses the OpenDev deployment of the Zuul project gating system for CI/CD. The OpenDev infrastructure sysadmins maintain some operational Zuul deployment documentation for their own purposes at https://docs.openstack.org/infra/system-config/zuul.html and also some information which was written separately during v3 migration at https://docs.openstack.org/infra/system-config/zuulv3.html which is due to get rolled into the first. Zuul itself already has excellent documentation (written by many of the same people) and its main architecture can be found described in the Admin Guide, with the diagram at https://zuul-ci.org/docs/zuul/admin/components.html providing a nice component overview.
What specifically are you looking for? -- Jeremy Stanley
-- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*
On 2019-08-07 09:48:25 +0900 (+0900), Trinh Nguyen wrote: [...]
Right now my company is using Jenkins for CI/CD which is not scalable and for me it's hard to define job pipeline because of XML.
Not steer you away from Zuul, but back when we originally used Jenkins we noticed the same things. We conquered the XML problem by inventing jenkins-job-builder, which allows you to define Jenkins jobs via templated YAML and then use that to generate the XML it expects. The scalability issue was worked around by creating the jenkins-gearman plugin and having (an earlier incarnation of) Zuul distribute jobs across multiple Jenkins masters via the Gearman protocol. You'll notice that current Zuul versions retain some of this heritage by continuing to use YAML (much of which is for Ansible) and multiple executors (which are no longer Jenkins masters, just servers invoking Ansible) communicating with the scheduler via Gearman. For us it's been a natural evolution.
I'm about to build a demonstration for my company using Zuul with Github as a replacement and trying to make sense of the OpenStack deployment of Zuul. I have been working with OpenStack projects for a couple of cycles in which Zuul has shown me its greatness and I think I can bring that power to the company. [...]
If you're looking for inspiration, check out some of the user stories at https://zuul-ci.org/users.html and visit the Zuul community in the #zuul channel on the Freenode IRC network or maybe subscribe to the mailing lists here: http://lists.zuul-ci.org/cgi-bin/mailman/listinfo Zuul has some remarkably thorough documentation, and helpful folks who are always happy to answer your questions. Good luck with your demo and let us know if you need any help! -- Jeremy Stanley
Trinh Nguyen <dangtrinhnt@gmail.com> writes:
Hi Jeremy,
Thanks for pointing that out. They're pretty helpful.
Sorry for not clarifying the purpose of my question in the first email. Right now my company is using Jenkins for CI/CD which is not scalable and for me it's hard to define job pipeline because of XML. I'm about to build a demonstration for my company using Zuul with Github as a replacement and trying to make sense of the OpenStack deployment of Zuul. I have been working with OpenStack projects for a couple of cycles in which Zuul has shown me its greatness and I think I can bring that power to the company.
Bests,
In addition to the excellent information that Jeremy provided, since you're talking about setting up a proof of concept, you may find it simpler to start with the Zuul Quick-Start: https://zuul-ci.org/start That's a container-based tutorial that will set you up with a complete Zuul system running on a single host, along with a private Gerrit instance. Once you have that running, it's fairly straightforward to take that and update the configuration to use GitHub instead of Gerrit. -Jim
Thanks, Jeremy and James for the super helpful information. I will try. On Wed, Aug 7, 2019 at 11:06 AM James E. Blair <corvus@inaugust.com> wrote:
Trinh Nguyen <dangtrinhnt@gmail.com> writes:
Hi Jeremy,
Thanks for pointing that out. They're pretty helpful.
Sorry for not clarifying the purpose of my question in the first email. Right now my company is using Jenkins for CI/CD which is not scalable and for me it's hard to define job pipeline because of XML. I'm about to build a demonstration for my company using Zuul with Github as a replacement and trying to make sense of the OpenStack deployment of Zuul. I have been working with OpenStack projects for a couple of cycles in which Zuul has shown me its greatness and I think I can bring that power to the company.
Bests,
In addition to the excellent information that Jeremy provided, since you're talking about setting up a proof of concept, you may find it simpler to start with the Zuul Quick-Start:
That's a container-based tutorial that will set you up with a complete Zuul system running on a single host, along with a private Gerrit instance. Once you have that running, it's fairly straightforward to take that and update the configuration to use GitHub instead of Gerrit.
-Jim
-- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*
participants (3)
-
corvus＠inaugust.com
-
Jeremy Stanley
-
Trinh Nguyen