Open Stack

On 02/08/16 03:11, Luigi Toscano wrote:
> On Monday, 1 August 2016 10:56:21 CEST Zane Bitter wrote:
>> On 29/07/16 13:12, Luigi Toscano wrote:
>>> Hi all,
>>> the Sahara jobs on the gate run the scenario tests (from sahara-tests)
>>> using the fake plugin, so no real Hadoop/Spark/BigData operations are
>>> performed, but other the other expected operations are executed on the
>>> image. In order to do this we used for long time this image:
>>> http://tarballs.openstack.org/heat-test-image/fedora-heat-test-image.qcow2
>>>
>>> which was updated early on this Friday (July 29th) from Fedora 22 to
>>> Fedora 24 breaking our jobs with some cryptic error, maybe something
>>> related to the repositories:
>>> http://logs.openstack.org/46/335946/12/check/gate-sahara-tests-dsvm-scenar
>>> io-nova-heat/5eeff52/logs/screen-sahara-eng.txt.gz?level=WARNING
>> So AFAICT from the log:
>>
>> "rpm -q xfsprogs" prints "package xfsprogs is not installed" which is
>> expected if xfsprogs is not installed.
>>
>> "yum install -y xfsprogs" redirects to "/usr/bin/dnf install -y
>> xfsprogs" which is expected on F24.
>>
>> dnf fails with "Error: Failed to synchronize cache for repo 'fedora'"
>> which means it couldn't download the Fedora repository data.
>>
>> "sudo mount -o data=writeback,noatime,nodiratime /dev/vdb
>> /volumes/disk1" then fails, doubtlessly because xfsprogs in not installed.
>>
>> The absence of "sudo" in the yum command (when it does appear in the
>> mount command) is suspicious, but unlikely to be the reason it can't
>> sync the cache.
> This is why I mentioned the repositories, yes.
>
>> It's not obvious why this change of image would suddenly result in not
>> being able to install packages. It seems more likely that you've never
>> been able to install packages, but the previous image had xfsprogs
>> preinstalled and the new one doesn't. I don't know the specifics of how
>> that image is built, but certainly Fedora has been making an ongoing
>> effort to strip the cloud image back to basics.
> But this is not a normal Fedora image. If I read project-config correctly,
> this is generated by this job:
>
> http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/
> jobs/heat.yaml#n34
>
>  From a brief chat on #heat on Friday it seems that the image is not gated or
> checked or even used right now. Is it the case? The image is almost a simple
> Fedora with few extra packages:
> http://git.openstack.org/cgit/openstack/heat-templates/tree/hot/software-config/test-image/build-heat-test-image.sh
>
We've stopped using this image recently because the download failure 
rate from tarballs.openstack.org was impacting heat's gate job 
reliability. We've switched to a vanilla fedora for now because none of 
our tests actually require a customized image. When we do have such 
tests we'll likely do boot-time install of packages from an AFS infra 
mirror.

We had no idea that Sahara was using this image in their gate, and it 
was certainly never intended for broader consumption.

Sahara would have a few options for an alternative:

- changing the test to work on a vanilla image

- do boot-time installation of the required packages

- work with infra on creating and hosting a custom image

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160802/430397ea/attachment.html>