Open Stack

Hi, all

I'd like to discuss the topic about how do we configure tempest in CI jobs
for TripleO.
I have currently two patches:
support for tempest: https://review.openstack.org/#/c/295844/
actually run of tests: https://review.openstack.org/#/c/297038/

Right now there is no upstream tool to configure tempest, so everybody use
their own tools. However it's planned and David Mellado is working on it
AFAIK.
Till then everybody use their own tools for tempest configuration.
I'd review two of them:
1) Puppet configurations that is used in puppet modules CI
2) Using configure_tempest.py script from
https://github.com/redhat-openstack/tempest/blob/master/tools/config_tempest.py

Unfortunately there is no ready puppet module or script, that configures
tempest, you need to create your own.

On other hand the config_tempest.py script provides full configuration,
support for tempest-deployer-input.conf and possibility to add any config
options in the command line when running it:

python config_tempest.py \
        --out etc/tempest.conf \
        --debug \
        --create \
        --deployer-input ~/tempest-deployer-input.conf \
        identity.uri $OS_AUTH_URL \
        compute.allow_tenant_isolation true \
        identity.admin_password $OS_PASSWORD \
        compute.build_timeout 500 \
        compute.image_ssh_user cirros

Also it uploads images, creates necessary roles, etc. The only thing it
requires - existence of public network.
So finally all tempest configuration from scratch will be like:

CONFIGURE_TEMPEST_DIR="$(ls
/usr/share/openstack-tempest-*/tools/configure-tempest-directory)"
$CONFIGURE_TEMPEST_DIR
neutron net-create nova --shared --router:external=True
--provider:network_type flat --provider:physical_network datacentre;
neutron subnet-create --name ext-subnet --allocation-pool
start=$FLOATING_IP_START,end=$FLOATING_IP_END --disable-dhcp --gateway
$EXTERNAL_NETWORK_GATEWAY nova $FLOATING_IP_CIDR;
python tempest/tools/install_venv.py
python config_tempest.py \
        --out etc/tempest.conf \
        --debug \
        --create \
        --deployer-input ~/tempest-deployer-input.conf \
        identity.uri $OS_AUTH_URL \
        compute.allow_tenant_isolation true \
        identity.admin_password $OS_PASSWORD \
        compute.build_timeout 500 \
        compute.image_ssh_user cirros
testr init; testr run

In my patch [1] I have proposed it with little changes to TripleO CI repo
[2]

Like I wrote before there is an option to use puppet for this, I spent a
time to investigate how to do it and would like share the results with your
in order to compare it with config_tempest.py approach.

First of all it's surprising that puppet-tempest actually doesn't know to
do almost anything. All it knows - it's to set IDs of public network (but
not router) and images. That's all. All the rest you need to configure
manually.
Then comes another problem - you can use it only on overcloud controller
node, where are all service configurations and hiera data. Most of values
are taken directly from /etc/{service}/service.conf files, so doing it on
undercloud you will configure undercloud itself (instead of overcloud)

So first of all you need to upload this manifest to controller node of
overcloud.
Let's write this puppet manifest, I wrote everything in one file for saving
a time, but of course it should be a module with usual puppet module
structure: module_name/manifests/init.pp with module_name class.

Manual configurations:

class testt::config {
  $os_username = 'admin'
  $os_tenant_name = hiera(keystone::roles::admin::admin_tenant)
  $os_password = hiera(admin_password)
  $os_auth_url = hiera(keystone::endpoint::public_url)
  $keystone_auth_uri = regsubst($os_auth_url, '/v2.0', '')
  $floating_range       = "192.0.2.0/24"
  $gateway_ip           = "192.0.2.1"
  $floating_pool        = 'start=192.0.2.50,end=192.0.2.99'
  $fixed_range          = '10.0.0.0/24'
  $router_name          = 'router1'
  $ca_bundle_cert_path = '/etc/ssl/certs/ca-bundle.crt'
  $cert_path           =
'/etc/pki/ca-trust/source/anchors/puppet_openstack.pem'
  $update_ca_certs_cmd = '/usr/bin/update-ca-trust force-enable &&
/usr/bin/update-ca-trust extract'
  $host_url = regsubst($keystone_auth_uri, ':5000', '')
}

Most of data is taken from hiera on the controller host. (/etc/hieradata)
Then we start actually the tempest configuration. Surprisingly it doesn't
have resource type to work with flavors, so all its configuration is done
by "exec"s. We run puppet with bash to run bash within a puppet, what gives
pretty big overhead.

class testt::provision {
  include testt::config

  $os_auth_options = "--os-username ${config::os_username} --os-password
${config::os_password} --os-tenant-name ${config::os_tenant_name}
--os-auth-url ${config::os_auth_url}/v2.0"

  exec { 'manage_m1.nano_nova_flavor':
    path     => '/usr/bin:/bin:/usr/sbin:/sbin',
    provider => shell,
    command  => "nova ${os_auth_options} flavor-delete m1.nano ||: ; nova
${os_auth_options} flavor-create m1.nano pup_tempest_custom_nano 128 0 1",
    unless   => "nova ${os_auth_options} flavor-list | grep
pup_tempest_custom_nano",
  }

  exec { 'manage_m1.micro_nova_flavor':
    path     => '/usr/bin:/bin:/usr/sbin:/sbin',
    provider => shell,
    command  => "nova ${os_auth_options} flavor-delete m1.micro ||: ;nova
${os_auth_options} flavor-create m1.micro pup_tempest_custom_micro 128 0 1",
    unless   => "nova ${os_auth_options} flavor-list | grep
pup_tempest_custom_micro",
  }

Then we create public and private networks, and router between them:

$neutron_deps = [Neutron_network['nova']]

neutron_network { 'nova':
    ensure          => 'present',
    router_external => true,
    tenant_name     => "${config::os_tenant_name}",
  }

neutron_subnet { 'ext-subnet':
  ensure           => 'present',
  cidr             => "${config::floating_range}",
  enable_dhcp      => false,
  allocation_pools => ["${config::floating_pool}"],
  gateway_ip       => "${config::gateway_ip}",
  network_name     => 'nova',
  tenant_name      => "${config::os_tenant_name}",
}

neutron_network { 'private':
    ensure      => 'present',
    tenant_name => "${config::os_tenant_name}",
  }

neutron_subnet { 'private_subnet':
  ensure       => 'present',
  cidr         => "${config::fixed_range}",
  network_name => 'private',
  tenant_name  => "${config::os_tenant_name}",
}

neutron_router { "${config::router_name}":
  ensure               => 'present',
  tenant_name          => "${config::os_tenant_name}",
  gateway_network_name => 'nova',
  require              => Neutron_subnet['ext-subnet'],
}
neutron_router_interface { "${config::router_name}:private_subnet":
  ensure => 'present',
}

After this it's a time to upload images:

  glance_image { 'cirros':
    ensure           => present,
    container_format => 'bare',
    disk_format      => 'qcow2',
    is_public        => 'yes',
    source           => '
http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img',
  }
  glance_image { 'cirros_alt':
    ensure           => present,
    container_format => 'bare',
    disk_format      => 'qcow2',
    is_public        => 'yes',
    source           => '
http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img',
  }

And then you run actually puppet tempest with configuration values, most of
them you set by yourself:

class { '::tempest':


debug => true,
use_stderr => false,
log_file => 'tempest.log',
tempest_clone_owner => $::id,
git_clone => true,
setup_venv => true,
tempest_clone_path => '/tmp/openstack/tempest',
lock_path => '/tmp/openstack/tempest',
tempest_config_file => '/tmp/openstack/tempest/etc/tempest.conf',
configure_images => true,
configure_networks => true,
allow_tenant_isolation => true,
identity_uri => "${testt::config::keystone_auth_uri}/v2.0",
identity_uri_v3 => "${testt::config::keystone_auth_uri}/v3",
admin_username => "${testt::config::os_username}",
admin_tenant_name => "${testt::config::os_tenant_name}",
admin_password => "${testt::config::os_password}",
admin_domain_name => 'Default',
auth_version => 'v3',
image_name => 'cirros',
image_name_alt => 'cirros_alt',
cinder_available => true,
glance_available => true,
horizon_available => $horizon,
nova_available => true,
neutron_available => true,
ceilometer_available => $ceilometer,
aodh_available => $aodh,
trove_available => $trove,
sahara_available => $sahara,
heat_available => $heat,
swift_available => true,
ironic_available => $ironic,
public_network_name => 'nova',
dashboard_url => "${testt::config::host_url}",
flavor_ref => 'pup_tempest_custom_nano',
flavor_ref_alt => 'pup_tempest_custom_micro',
image_ssh_user => 'cirros',
image_alt_ssh_user => 'cirros',
img_file => 'cirros-0.3.4-x86_64-disk.img',
compute_build_interval => 10,
ca_certificates_file => "${testt::config::ca_bundle_cert_path}",
img_dir => '/tmp/openstack/tempest',
}

But it's not enough, you need also to make some workarounds and additional
configurations, for example:

tempest_config { 'object-storage/operator_role':
  value => 'SwiftOperator',
  path  => "${tempest_clone_path}/etc/tempest.conf",
}
}

After this run puppet on controller node:

sudo puppet apply --verbose --debug --detailed-exitcodes -e "include
::testt" | tee ~/puppet_run.log

After everything is finished, you need to copy the folder with tempest to
your node:
scp -r -heat-admin@${CONTROLLER}:/tmp/openstack /tmp/

After this run within this directory testr init and run tests:
/tmp/tempest/tools/with_venv.sh testr init
/tmp/tempest/tools/with_venv.sh testr run

There are still holes in this configuration and most likely you'd fix it by
another workarounds and tempest_config runs, because it's still a few of
skipped tests, so configuration is not full as it would be done with
config_tempest.py.
You don't have also any possibility to add custom configuration in running
the manifest, for each config change you need to change the manifest itself
which makes it maintenance harder and more complex.

I would say that conclusion is quite obvious for me and it's much easier
even to write tempest.conf manually from scratch or simple template and use
5 bash lines, then use puppet for things it's completely not fit to.

P.S. In this script I used ideas from puppet-openstack-integration and
packstack projects.

[1] https://review.openstack.org/#/c/295844/
[2] https://git.openstack.org/openstack-infra/tripleo-ci

-- 
Best regards
Sagi Shnaidman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160407/17b3dc55/attachment.html>