[infra] CentOS support for mirror role in system-config
Hi all, TL;DR: I have proposed a set of changes to add CentOS support to the mirror role in system-config with [1] and would appreciate reviews. Long version: the RDO project maintains a set of mirrors that mimic those provided by the OpenDev Infra team to jobs running in review.opendev.org. The reason for this is to provide the same environment for TripleO jobs to run on both OpenDev's and RDO's Gerrit platforms. Previously, we used the Puppet modules from the system-config repo, together with some unmerged changes to move that support to puppet-openstackci, as it was suggested during the review process [2]. Once those modules were obsoleted, we have proposed a set of changes to the mirror ansible role [1] to add that support. I would appreciate reviews on those changes (thanks Ian for the first reviews!). Some of them are small bugfixes to fix the already existing CentOS support, while [3] is the one targeting the mirror role. Thanks, Javier [1] - https://review.opendev.org/#/q/status:open+project:opendev/system-config+bra... [2] - https://review.opendev.org/#/q/status:open+project:opendev/puppet-openstackc... [3] - https://review.opendev.org/736996
On Tue, Jul 21, 2020, at 1:25 AM, Javier Pena wrote:
Hi all,
TL;DR: I have proposed a set of changes to add CentOS support to the mirror role in system-config with [1] and would appreciate reviews.
Long version: the RDO project maintains a set of mirrors that mimic those provided by the OpenDev Infra team to jobs running in review.opendev.org. The reason for this is to provide the same environment for TripleO jobs to run on both OpenDev's and RDO's Gerrit platforms.
Previously, we used the Puppet modules from the system-config repo, together with some unmerged changes to move that support to puppet-openstackci, as it was suggested during the review process [2]. Once those modules were obsoleted, we have proposed a set of changes to the mirror ansible role [1] to add that support.
I would appreciate reviews on those changes (thanks Ian for the first reviews!). Some of them are small bugfixes to fix the already existing CentOS support, while [3] is the one targeting the mirror role.
One thing we've done as part of the shift from puppet to Ansible is intentionally "hide" these implementation details. Specifically all of the roles under system-config/playbooks/roles aren't really intended to be consumed externally. We've done this because back with the Puppet modules we invested extra effort to make the modules re-consumable, but then never got much help in making that viable. With the shift to Ansible we've taken that as an opportunity to make it clear we don't really intend to make these roles re-consumable (that is why they are in playbooks/roles). This has allowed us to reduce the number of platforms we care about as well as make changes assuming we are the only users. One specific concern along these lines is we've added https support to the mirrors. Our Xenial jobs are the last remaining place where https support isn't always available; once Xenial jobs are retired we'd like to force https. Doing that may break downstream users if they have consumers of their mirrors that cannot do https. This case may not apply to RDO, but I'm sure there are others that would. That said, the use case you describe is a reasonable one. I think several of the changes are relatively minor and don't present major concern (particularly those to roles/ not playbooks/roles/), and we have much better ability to test things now. I'm not sure what the best option is at this point. I'd like to selfishly retain the simplicity we've gained through the switch to Ansible. Would it make sense for RDO to use a copy of the role where centos support can be added? I guess the issue with this is the role has several other dependencies and isn't necessarily usable in isolation. Would RDO expect us to coordinate upstream changes to the mirrors with them? Curious to hear what others think.
Thanks, Javier
[1] - https://review.opendev.org/#/q/status:open+project:opendev/system-config+bra... [2] - https://review.opendev.org/#/q/status:open+project:opendev/puppet-openstackc... [3] - https://review.opendev.org/736996
Thanks for your reply Clark, some comments inline
On Tue, Jul 21, 2020, at 1:25 AM, Javier Pena wrote:
Hi all,
TL;DR: I have proposed a set of changes to add CentOS support to the mirror role in system-config with [1] and would appreciate reviews.
Long version: the RDO project maintains a set of mirrors that mimic those provided by the OpenDev Infra team to jobs running in review.opendev.org. The reason for this is to provide the same environment for TripleO jobs to run on both OpenDev's and RDO's Gerrit platforms.
Previously, we used the Puppet modules from the system-config repo, together with some unmerged changes to move that support to puppet-openstackci, as it was suggested during the review process [2]. Once those modules were obsoleted, we have proposed a set of changes to the mirror ansible role [1] to add that support.
I would appreciate reviews on those changes (thanks Ian for the first reviews!). Some of them are small bugfixes to fix the already existing CentOS support, while [3] is the one targeting the mirror role.
One thing we've done as part of the shift from puppet to Ansible is intentionally "hide" these implementation details. Specifically all of the roles under system-config/playbooks/roles aren't really intended to be consumed externally. We've done this because back with the Puppet modules we invested extra effort to make the modules re-consumable, but then never got much help in making that viable. With the shift to Ansible we've taken that as an opportunity to make it clear we don't really intend to make these roles re-consumable (that is why they are in playbooks/roles). This has allowed us to reduce the number of platforms we care about as well as make changes assuming we are the only users.
One specific concern along these lines is we've added https support to the mirrors. Our Xenial jobs are the last remaining place where https support isn't always available; once Xenial jobs are retired we'd like to force https. Doing that may break downstream users if they have consumers of their mirrors that cannot do https. This case may not apply to RDO, but I'm sure there are others that would.
In the RDO case, https is a welcome enhancement, actually. The only potential issue would be the hardcoded paths to the SSL certficates, but nothing we cannot work with.
That said, the use case you describe is a reasonable one. I think several of the changes are relatively minor and don't present major concern (particularly those to roles/ not playbooks/roles/), and we have much better ability to test things now. I'm not sure what the best option is at this point. I'd like to selfishly retain the simplicity we've gained through the switch to Ansible.
Would it make sense for RDO to use a copy of the role where centos support can be added? I guess the issue with this is the role has several other dependencies and isn't necessarily usable in isolation.
I'd prefer to avoid forking the role if possible, so we can automatically get updates when new mirrors are added (for example, https://review.opendev.org/738942, even if not relevant to our use case). If this copy of the role is actively maintained, though, it would be feasible. About role dependencies, I have been able to make it work with just the openafs-client and kerberos-client roles plus some glue code to provide the SSL certificate. To get CI jobs to pass I also had to update the base role [4], but this could be optional depending on the final setup.
Would RDO expect us to coordinate upstream changes to the mirrors with them?
Not really. Most of the issues we have had so far have been related to new mirrors being missing from our config because we had not updated the under-review patches, so I do not expect a tight coordination to be required. Regards, Javier [4] - https://review.opendev.org/737043
Curious to hear what others think.
Thanks, Javier
[1] - https://review.opendev.org/#/q/status:open+project:opendev/system-config+bra... [2] - https://review.opendev.org/#/q/status:open+project:opendev/puppet-openstackc... [3] - https://review.opendev.org/736996
On Tue, Jul 21, 2020 at 09:30:19AM -0700, Clark Boylan wrote:
One specific concern along these lines is we've added https support to the mirrors.
Another thing I can see coming is kafs support; which requires recent kernels but is becoming available in Debian. Just another area we'll probably want to play in that is distro specific.
Would RDO expect us to coordinate upstream changes to the mirrors with them?
Perhaps we should quantify what the bits are we need? As I mentioned, I've been shy to move the openafs roles outside system-config because they rely on debs/rpms built specifically by us to work around no-packages (rpm) or out of date packages (deb). I don't want anyone to think they're generic then we break something when we update the packages for our own purposes. There isn't really any magic in the the apache setup done in the mirror role; it's more or less a straight "install packages put config in" role. That argument cuts both ways -- it's not much for system-config to maintain but it's not really much to duplicate outside. The mirror config I can see us wanting to be in sync with. I'd be happy to move that into a separate role, with a few paramaters to make it write out in different locations, etc. instead of lumping it all in with the server setup? Is that a compromise position between keeping centos servers in system-config and making things reusable? Are there other roles of concern? -i
On Tue, Jul 21, 2020 at 09:30:19AM -0700, Clark Boylan wrote:
One specific concern along these lines is we've added https support to the mirrors.
Another thing I can see coming is kafs support; which requires recent kernels but is becoming available in Debian. Just another area we'll probably want to play in that is distro specific.
Would RDO expect us to coordinate upstream changes to the mirrors with them?
Perhaps we should quantify what the bits are we need?
As I mentioned, I've been shy to move the openafs roles outside system-config because they rely on debs/rpms built specifically by us to work around no-packages (rpm) or out of date packages (deb). I don't want anyone to think they're generic then we break something when we update the packages for our own purposes.
There isn't really any magic in the the apache setup done in the mirror role; it's more or less a straight "install packages put config in" role. That argument cuts both ways -- it's not much for system-config to maintain but it's not really much to duplicate outside.
The mirror config I can see us wanting to be in sync with. I'd be happy to move that into a separate role, with a few paramaters to make it write out in different locations, etc. instead of lumping it all in with the server setup?
Is that a compromise position between keeping centos servers in system-config and making things reusable? Are there other roles of concern?
This could work. Besides the kerberos-client and openafs-client roles (which should be relatively straightforward to replicate if needed), that would be all we need to keep the configuration in sync. Regards, Javier
-i
participants (3)
-
Clark Boylan
-
Ian Wienand
-
Javier Pena