[infra][qe][nova] Would it be possible to host a custom cirros image somewhere for the nova-next job?
Hello all, I recently landed a fix in Cirros [1] to allow it to be used by the nova-next job when it is configured to use the q35 machine type [2], something that we would like to make the default sometime in the future. With no release of Cirros on the horizon I was wondering if I could instead host a build of the image somewhere for use solely by the nova-next job? If this isn't possible I'll go back to the Cirros folks upstream and ask for a release but given my change is the only one to land there in almost a year I wanted to ask about this approach first. Many thanks in advance, Lee [1] https://github.com/cirros-dev/cirros/pull/65 [2] https://review.opendev.org/c/openstack/nova/+/708701
On Wed, 2021-03-03 at 10:46 +0000, Lee Yarwood wrote:
Hello all,
I recently landed a fix in Cirros [1] to allow it to be used by the nova-next job when it is configured to use the q35 machine type [2], something that we would like to make the default sometime in the future.
With no release of Cirros on the horizon I was wondering if I could instead host a build of the image somewhere for use solely by the nova-next job?
If this isn't possible I'll go back to the Cirros folks upstream and ask for a release but given my change is the only one to land there in almost a year I wanted to ask about this approach first.
am im not sure where would be the best location but when i have tought about it in the past i came up with a few options that may or may not work. i know that the nodepool images are hosted publiclaly but i can never remberer where. if we wraped the cirror build in a dib element that might be an approch we coudl take. build it woith nodepool build then mirror that to the cloud and use it. the other idea i had for this in the past was to put the image on tarballs.openstack.org we have the ironic ipa image there already so if we did a one of build we coud maybe see if infra were ok with hosting it there and the mirroing that to our cloud providres to avoid the need to pull it. the other option might be to embed it in the nodepool images. while not all jobs will need it most will so if we had a static copy of it we could inject it into the image in the devstack data dir or anogher whle know cache dir in the image and just have devstack use that instead of downloading it.
Many thanks in advance,
Lee
[1] https://github.com/cirros-dev/cirros/pull/65 [2] https://review.opendev.org/c/openstack/nova/+/708701
On 2021-03-03 12:52:40 +0000 (+0000), Sean Mooney wrote: [...]
i know that the nodepool images are hosted publiclaly but i can never remberer where. if we wraped the cirror build in a dib element that might be an approch we coudl take. build it woith nodepool build then mirror that to the cloud and use it.
The images we build from Nodepool 1. are created with diskimage-builder (does DIB support creating Cirros images?), and are uploaded to Glance in all our donor-providers (are we likely to actually boot it directly in them?).
the other idea i had for this in the past was to put the image on tarballs.openstack.org we have the ironic ipa image there already so if we did a one of build we coud maybe see if infra were ok with hosting it there and the mirroing that to our cloud providres to avoid the need to pull it.
The files hosted on the tarballs site (including the IPA images) are built by Zuul jobs. If you can figure out what project that job should reside in and a cadence to run it, that might be an option.
the other option might be to embed it in the nodepool images. while not all jobs will need it most will so if we had a static copy of it we could inject it into the image in the devstack data dir or anogher whle know cache dir in the image and just have devstack use that instead of downloading it.
We already do that for official Cirros images: https://opendev.org/openstack/project-config/src/branch/master/nodepool/elem... If you look in the build logs of a DevStack-based job, you'll see it check, e.g., /opt/stack/devstack/files/cirros-0.4.0-x86_64-disk.img and then not wget that image over the network because it exists on the filesystem. -- Jeremy Stanley
On Wed, 3 Mar 2021 at 13:45, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2021-03-03 12:52:40 +0000 (+0000), Sean Mooney wrote: [...]
i know that the nodepool images are hosted publiclaly but i can never remberer where. if we wraped the cirror build in a dib element that might be an approch we coudl take. build it woith nodepool build then mirror that to the cloud and use it.
The images we build from Nodepool 1. are created with diskimage-builder (does DIB support creating Cirros images?), and are uploaded to Glance in all our donor-providers (are we likely to actually boot it directly in them?).
the other idea i had for this in the past was to put the image on tarballs.openstack.org we have the ironic ipa image there already so if we did a one of build we coud maybe see if infra were ok with hosting it there and the mirroing that to our cloud providres to avoid the need to pull it.
The files hosted on the tarballs site (including the IPA images) are built by Zuul jobs. If you can figure out what project that job should reside in and a cadence to run it, that might be an option.
the other option might be to embed it in the nodepool images. while not all jobs will need it most will so if we had a static copy of it we could inject it into the image in the devstack data dir or anogher whle know cache dir in the image and just have devstack use that instead of downloading it.
We already do that for official Cirros images:
https://opendev.org/openstack/project-config/src/branch/master/nodepool/elem...
If you look in the build logs of a DevStack-based job, you'll see it check, e.g., /opt/stack/devstack/files/cirros-0.4.0-x86_64-disk.img and then not wget that image over the network because it exists on the filesystem.
Thanks both, So this final option looks like the most promising. It would not however involve any independent build steps in nodepool or zuul. If infra are okay with this I can host the dev build I have with my change applied and post a change to cache it alongside the official images. Thanks again, Lee
On 2021-03-03 17:33:59 +0000 (+0000), Lee Yarwood wrote:
On Wed, 3 Mar 2021 at 13:45, Jeremy Stanley <fungi@yuggoth.org> wrote: [...]
https://opendev.org/openstack/project-config/src/branch/master/nodepool/elem...
If you look in the build logs of a DevStack-based job, you'll see it check, e.g., /opt/stack/devstack/files/cirros-0.4.0-x86_64-disk.img and then not wget that image over the network because it exists on the filesystem.
Thanks both,
So this final option looks like the most promising. It would not however involve any independent build steps in nodepool or zuul. If infra are okay with this I can host the dev build I have with my change applied and post a change to cache it alongside the official images.
Yeah, I don't see that being much different from hitting download.cirros-cloud.net during out image builds. If you're forking or otherwise making unofficial builds of Cirros though, you might want to name the files something which makes that clear so people seeing it in job build logs know it's not an official Cirros provided version. I believe our Nodepool builders also cache such downloads locally, so you'll probably see them downloaded once each from handful of builders and, not again until the next time you update the filename in that DIB element. -- Jeremy Stanley
On Wed, Mar 03, 2021 at 07:27:46PM +0000, Jeremy Stanley wrote:
Yeah, I don't see that being much different from hitting download.cirros-cloud.net during out image builds. If you're forking or otherwise making unofficial builds of Cirros though, you might want to name the files something which makes that clear so people seeing it in job build logs know it's not an official Cirros provided version.
I'd personally prefer to build it in a job and publish it so that you can see what's been done. Feels like a similar situation to the RPMs we build for openafs (see ~[1]). If you put your patch somewhere, you can use a file-matcher to just rebuild/upload when something changes. I see the argument that one binary blob from upstream is about as trustworthy as any other, though. -i [1] https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/playbook...
On 2021-03-04 09:31:08 +1100 (+1100), Ian Wienand wrote: [...]
I'd personally prefer to build it in a job and publish it so that you can see what's been done. [...]
Sure, all things being equal I'd also prefer a transparent automated build with a periodic refresh, but that seems like something we can make incremental improvement on. Of course, you're a more active reviewer on DevStack than I am, so I'm happy to follow your lead if it's something you feel strongly about. -- Jeremy Stanley
On Wed, 3 Mar 2021 at 22:41, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2021-03-04 09:31:08 +1100 (+1100), Ian Wienand wrote: [...]
I'd personally prefer to build it in a job and publish it so that you can see what's been done. [...]
Sure, all things being equal I'd also prefer a transparent automated build with a periodic refresh, but that seems like something we can make incremental improvement on. Of course, you're a more active reviewer on DevStack than I am, so I'm happy to follow your lead if it's something you feel strongly about.
Agreed, if there's a need to build and cache another unreleased cirros fix then I would be happy to look into automating the build somewhere but for a one off I think this is just about acceptable. FWIW nova-next is passing with the image below: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/c/openstack/nova/+/708701 I'll clean that change up later today. Thanks again, Lee
On Thu, 4 Mar 2021 at 11:07, Lee Yarwood <lyarwood@redhat.com> wrote:
On Wed, 3 Mar 2021 at 22:41, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2021-03-04 09:31:08 +1100 (+1100), Ian Wienand wrote: [...]
I'd personally prefer to build it in a job and publish it so that you can see what's been done. [...]
Sure, all things being equal I'd also prefer a transparent automated build with a periodic refresh, but that seems like something we can make incremental improvement on. Of course, you're a more active reviewer on DevStack than I am, so I'm happy to follow your lead if it's something you feel strongly about.
Agreed, if there's a need to build and cache another unreleased cirros fix then I would be happy to look into automating the build somewhere but for a one off I think this is just about acceptable.
FWIW nova-next is passing with the image below:
WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/c/openstack/nova/+/708701
I'll clean that change up later today.
After all of this Cirros 0.5.2 was released on Friday after another colleague asked. I've reverted my original change to cache the dev build, introduced a change to cache 0.5.2 and switched devstack over to 0.5.2 below: Revert "Add custom cirros image with ahci module enabled to cache" https://review.opendev.org/c/openstack/project-config/+/779140 Add Cirros 0.5.2 to cache https://review.opendev.org/c/openstack/project-config/+/779178 Update Cirros to 0.5.2 https://review.opendev.org/c/openstack/devstack/+/779179 Apologies for all of the noise with this, hopefully that's it for now. Thanks again, Lee
On Wed, 3 Mar 2021 at 19:33, Jeremy Stanley <fungi@yuggoth.org> wrote:
On 2021-03-03 17:33:59 +0000 (+0000), Lee Yarwood wrote:
On Wed, 3 Mar 2021 at 13:45, Jeremy Stanley <fungi@yuggoth.org> wrote: [...]
https://opendev.org/openstack/project-config/src/branch/master/nodepool/elem...
If you look in the build logs of a DevStack-based job, you'll see it check, e.g., /opt/stack/devstack/files/cirros-0.4.0-x86_64-disk.img and then not wget that image over the network because it exists on the filesystem.
Thanks both,
So this final option looks like the most promising. It would not however involve any independent build steps in nodepool or zuul. If infra are okay with this I can host the dev build I have with my change applied and post a change to cache it alongside the official images.
Yeah, I don't see that being much different from hitting download.cirros-cloud.net during out image builds. If you're forking or otherwise making unofficial builds of Cirros though, you might want to name the files something which makes that clear so people seeing it in job build logs know it's not an official Cirros provided version. I believe our Nodepool builders also cache such downloads locally, so you'll probably see them downloaded once each from handful of builders and, not again until the next time you update the filename in that DIB element.
Okay I've pushed the following for this: Add custom cirros image with ahci module enabled to cache https://review.opendev.org/c/openstack/project-config/+/778590 I've gone for cirros-0.5.1-dev-ahci-x86_64-disk.img as the image name but we can hash that out in the review. Many thanks again! Lee
participants (4)
-
Ian Wienand
-
Jeremy Stanley
-
Lee Yarwood
-
Sean Mooney