[infra][dev] Options for upcoming git:// to https:// transition
Hi, In moving repos to OpenDev and a gitea instance [1] we will need to change any git://git.openstack.org references to https://, as gitea does not currently support the native git protocol. First up, from a "gate on fire" point of view, the sky is not falling. Most jobs referencing git repos should be using the local Zuul repositories during their gate CI, and not pulling directly from git. This should minimise issues. Still, we need to fix things up. Some raw numbers: to simply perform a "s|git://git.openstack.org|https://git.openstack.org|" over every openstack/* repository on master and every stable/* branch inside results in ~1200 changes [2]. If we limit that just to projects with a devstack plugin (I would say the main user of git:// urls as every plugin has more or less copied it this way) that's about 420 changes. If you'd like to try it yourself, scripts are at [3]. I can't really categorise this; there's a lot of documentation updates, a lot of devstack plugin changes, a bunch of -1/+1 changes and some weirder ones with manifest files, etc. There was discussion over this in the infra meeting [5]. I think we came up with 3 options: 1. Propose changes now ---------------------- We can propose all these changes to the gate now (slowly, but ultimately in one hit) and let teams review and merge them. Pros: * changes are tested * we get this done _before_ the transition; meaning one less thing * teams get a heads up, can call out anything odd Cons: * sure to be a lot of unrelated gate failures * sure to be a lot of unmerged changes * experience has shown that mass changes can cause confusion. Things happen like others then proposing similar changes or otherwise getting the wrong idea about what is happening. * Ultimately will still be using redirects from openstack.org to opendev.org after transition 2. Merge changes at transition time ----------------------------------- We can have the same changes ready to go, and merge them manually during the transition maintenance period to gitea; bypassing the gate. Pros: * avoids cons of 1 Cons: * one more thing to do during a tricky transition * changes aren't tested. This is a very low risk change; but we all know the xkcd about breaking people's workflows [4] * Ultimately will still be using redirects from openstack.org to opendev.org after transition 3. Merge changes at transition time, with full rename ----------------------------------------------------- Similar to 2, but by timing such that OpenDev gitea will be fully operational we can modify the changes such that they go directly to the OpenDev namespace. This is a more tricky change, essentially (https|git)://git.openstack.org/openstack/project -> https://git.opendev.org/openstack/project Pros: * Pros of 2. * This moves directly to the "final state" Cons: * Cons of 2. * Adds another match, which increases again the risk that things break with the new URLs (internal regex matches, etc). * The "openstack/project" part of the name is, I believe, still under discussion over exactly how we will lay that out, and if projects may have their own namespaces, etc. Adds a synchronization point with this; possible complexity. My personal feeling here is that we have enough to do with the transition, and we would be better to sort this out as much as possible first. Thus I'd suggest we take approach 1 of submitting changes to the gate now. I imagine active projects will appreciate the heads up and merge things quickly. However, given the low risk, I'd suggest we agree on a date to force-merge any outstanding reviews before the cut over. This leaves us relying on the redirects from openstack.org -> opendev.org, but practically there is no way we can avoid having those in place anyway (due to external links), so cleanup of that can be handled by projects asynchronously. Of course, opinions welcome :) I'd suggest we make a concrete decision in the next infra meeting. -i [1] http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.htm... [2] http://paste.openstack.org/show/747762/ [3] https://review.openstack.org/642314 [4] https://xkcd.com/1172/ [5] http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-12-19.01.lo...
On Thu, Mar 14, 2019 at 09:45:50PM +1100, Ian Wienand wrote:
In moving repos to OpenDev and a gitea instance [1] we will need to change any git://git.openstack.org references to https://, as gitea does not currently support the native git protocol.
1. Propose changes now ----------------------
We can propose all these changes to the gate now (slowly, but ultimately in one hit) and let teams review and merge them.
In the last infra meeting [1] we decided to take this approach. You can expect to see reviews for this shortly, and merging them is appreciated. They will created under the topic "opendev-gerrit-git". Our current plan is to merge any outstanding changes on the 19th April. The related Storyboard is #2004627, task #29701. These are not linked from the change to avoid overwhelming the story with links. The scripts creating the changes are at [2] Thanks, -i [1] http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-19-19.01.lo... [2] https://review.openstack.org/#/c/642314/
On Fri, Mar 22, 2019 at 10:39 AM Ian Wienand <iwienand@redhat.com> wrote:
On Thu, Mar 14, 2019 at 09:45:50PM +1100, Ian Wienand wrote:
In moving repos to OpenDev and a gitea instance [1] we will need to change any git://git.openstack.org references to https://, as gitea does not currently support the native git protocol.
1. Propose changes now ----------------------
We can propose all these changes to the gate now (slowly, but ultimately in one hit) and let teams review and merge them.
In the last infra meeting [1] we decided to take this approach. You can expect to see reviews for this shortly, and merging them is appreciated.
Just wondering if thought was given to doing #1 with also renaming to opendev? Granted we'd have to make sure opendev was ready for it, but since the reviews will be happening manually (at least for a few weeks) it might be a good opportunity to help automate it while checking it's correct. Cheers, Josh
They will created under the topic "opendev-gerrit-git".
Our current plan is to merge any outstanding changes on the 19th April.
The related Storyboard is #2004627, task #29701. These are not linked from the change to avoid overwhelming the story with links.
The scripts creating the changes are at [2]
Thanks,
-i
[1] http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-19-19.01.lo... [2] https://review.openstack.org/#/c/642314/
On Fri, Mar 22, 2019 at 12:49:10PM +1100, Joshua Hesketh wrote:
Just wondering if thought was given to doing #1 with also renaming to opendev? Granted we'd have to make sure opendev was ready for it, but since the reviews will be happening manually (at least for a few weeks) it might be a good opportunity to help automate it while checking it's correct.
So that falls under the original option 3, I'd say. The issue is that, let's take nova, we're not 100% sure if it will be "git.opendev.org/openstack/nova" or "git.opendev.org/nova/nova" or something else entirely in the end. Either way, "https://git.openstack.org/openstack/nova" (i.e. the existing URL) will be redirected to the correct location at transition time. So we can make sure everything is pointing at that well before the transition, and that avoids breakage. Fixing the URLs to point at the new canonical OpenDev home can be done at any time after the transition. Thanks, -i
Yep, that makes sense (as per IRC). Thanks! On Fri, Mar 22, 2019 at 6:15 PM Ian Wienand <iwienand@redhat.com> wrote:
On Fri, Mar 22, 2019 at 12:49:10PM +1100, Joshua Hesketh wrote:
Just wondering if thought was given to doing #1 with also renaming to opendev? Granted we'd have to make sure opendev was ready for it, but since the reviews will be happening manually (at least for a few weeks) it might be a good opportunity to help automate it while checking it's correct.
So that falls under the original option 3, I'd say.
The issue is that, let's take nova, we're not 100% sure if it will be "git.opendev.org/openstack/nova" or "git.opendev.org/nova/nova" or something else entirely in the end.
Either way, "https://git.openstack.org/openstack/nova" (i.e. the existing URL) will be redirected to the correct location at transition time.
So we can make sure everything is pointing at that well before the transition, and that avoids breakage. Fixing the URLs to point at the new canonical OpenDev home can be done at any time after the transition.
Thanks,
-i
---- On Thu, 21 Mar 2019 18:37:10 -0500 Ian Wienand <iwienand@redhat.com> wrote ----
On Thu, Mar 14, 2019 at 09:45:50PM +1100, Ian Wienand wrote:
In moving repos to OpenDev and a gitea instance [1] we will need to change any git://git.openstack.org references to https://, as gitea does not currently support the native git protocol.
1. Propose changes now ----------------------
We can propose all these changes to the gate now (slowly, but ultimately in one hit) and let teams review and merge them.
In the last infra meeting [1] we decided to take this approach. You can expect to see reviews for this shortly, and merging them is appreciated.
They will created under the topic "opendev-gerrit-git".
Our current plan is to merge any outstanding changes on the 19th April.
The related Storyboard is #2004627, task #29701. These are not linked from the change to avoid overwhelming the story with links.
One question: I saw you are changing the links on all stable branches etc which are involved in our gate testing. +1 on that. My question is for libraries, if links modification is in any library (something on code side) then, should not we release that lib and bump the min version(low constraint) ? otherwise, it fails when project gate jobs/installation install that lib with an older version which use old url? I was reviewing openstack-dev/hacking changes [1] when this question strikes me but hacking change is just in tests so we are ok there. [1] https://review.openstack.org/#/c/646379/ -gmann
The scripts creating the changes are at [2]
Thanks,
-i
[1] http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-03-19-19.01.lo... [2] https://review.openstack.org/#/c/642314/
On 2019-03-26 10:01:25 -0500 (-0500), Ghanshyam Mann wrote: [...]
if links modification is in any library (something on code side) then, should not we release that lib and bump the min version(low constraint)? otherwise, it fails when project gate jobs/installation install that lib with an older version which use old url?
About the only places I can imagine this could be necessary is deployment projects which perform deployment from source and default to retrieving source code from our Git service by default. If any of those projects are using classic git protocol over 9418/tcp instead of HTTP/HTTPS then they will likely want to perform some point releases prior to April 19. I strongly doubt any of these are tracked in global requirements or constraints files, so there shouldn't be knock-on effects.
I was reviewing openstack-dev/hacking changes [1] when this question strikes me but hacking change is just in tests so we are ok there.
[1] https://review.openstack.org/#/c/646379/ [...]
Yeah, as you say, this one isn't really an example of the possible class of problem you're describing. -- Jeremy Stanley
participants (4)
-
Ghanshyam Mann
-
Ian Wienand
-
Jeremy Stanley
-
Joshua Hesketh