Open Stack

Hi Kris,

Thanks for the feedback, I think everyone involved in kolla-ansible 
should take the time to read through it, as it definitely highlights 
some areas that we need to improve.

There's a lot of questions here, so I haven't gone into too much detail 
on any specific one; my hope is that I can clear up the majority of it 
and then we can follow up on some of the topics that require more 
discussion.

Hope it helps,
-Paul

 >      * I need to define a number of servers in my inventory outside of
 >     the specific servers that I want to perform actions against.  I need
 >     to define groups baremetal, rabbitmq, memcached, and control (IN
 >     addition to the glance specific groups) most of these seem to be
 >     gathering information for config? (Baremetal was needed soley to try
 >     to run the bootstrap play)

You only need to define the top level groups, i.e. control, network, 
storage, monitoring, etc. If you don't want or have dedicated nodes for 
each of these groups it's fine to put the same node into multiple 
groups. So for example, if you're not interested in monitoring right 
now, you can just put your control node(s) under this and forget about 
it. The groups marked with [*:children] (e.g. bootstrap) are "groups of 
groups" and you shouldn't need to modify these at all.

 > Running a change specifically against
 >     "glance" causes fact gathering on a number of other servers not
 >     specifically where glance is running?  My concern here is that I
 >     want to be able to run kola-ansible against a specific service and
 >     know that only those servers are being logged into.

The fact gathering on every server is a compromise taken by Kolla to 
work around limitations in Ansible. It works well for the majority of 
situations; for more detail and potential improvements on this please 
have a read of this post: 
http://lists.openstack.org/pipermail/openstack-dev/2016-November/107833.html

 >     * I want to run a dry-run only, being able to see what will happen
 >     before it happens, not during; during makes it really hard to see
 >     what will happen until it happens. Also supporting  `ansible --diff`
 >     would really help in understanding what will be changed (before it
 >     happens).

Agree a dry run would be useful, I believe it came up during the 
Barcelona design summit but has not yet been looked at. The ansible 
--diff sounds like something we could easily do, if you could log a 
blueprint at blueprints.launchpad.net/kolla-ansible I think that would help.

 >     * Database task are ran on every deploy and status of change DB
 >     permissions always reports as changed? Even when nothing happens,
 >     which makes you wonder "what changed"?

This shouldn't be the case, I just double checked taking Glance as an 
example, it reports "ok" (no change) for all runs after the initial 
deploy. Perhaps you've come across a bug, if you think this is the case 
please log one.

 > Also, Can someone tell me why the DB stuff is done on a
 >     deployment task?  Seems like the db checks/migration work should
 >     only be done on a upgrade or a bootstrap?

Deploy includes bootstrap, but bootstrap is only done if the database is 
not found (or on upgrade). Again it sounds like you're coming across 
some unusual behavior here, suggest checking in with us on 
#openstack-kolla or filing a bug.

 >     * Database services (that at least we have) our not managed by our
 >     team, so don't want kolla-ansible touching those (since it won't be
 >     able to). No way to mark the DB as "externally managed"?  IE we dont
 >     have permissions to create databases or add users.  But we got all
 >     other permissions on the databases that are created, so normal
 >     db-manage tooling works.

This is definitely something we need - I'm pretty sure I saw something 
around this in the review queue very recently. I can't find it off hand 
so hopefully someone can chip in here on the status of this work.

 >     * Maintenance level operations; doesn't seem to be any built-in to
 >     say 'take a server out  of a production state, deploy to it, test
 >     it, put it back into production'  Seems like if kola-ansible is
 >     doing haproxy for API's, it should be managing this?  Or an
 >     extension point to allow us to run our own maintenance/testing 
scripts?

Again, discussed, needs to happen, but not there as of yet.

 >     * Config must come from kolla-ansible and generated templates.  I
 >     know we have a patch up for externally managed service
 >     configuration.  But if we aren't suppose to use kolla-ansible for
 >     generating configs (see below), why cant we override this piece?

I'm not quite following you here, the config templates from 
kolla-ansible are one of it's stronger pieces imo, they're reasonably 
well tested and maintained. What leads you to believe they shouldn't be 
used?

 >     * Certain parts of it are 'reference only' (the config tasks),
 >     are not recommended

This is untrue - kolla-ansible is designed to stand up a stable and 
usable OpenStack 'out of the box'. There are definitely gaps in the 
operator type tasks as you've highlighted, but I would not call it 
'reference only'.

 >     Is kolla-ansibles design philosophy that every deployment is an
 >     upgrade?  Or every deployment should include all the base level
 >     boostrap tests?

Every deployment should be idempotent. This is key to kolla-ansible's 
philosophy. It sounds like you've come across some tasks that are 
performing this way which is confusing things. Upgrades are only 
performed if explicitly requested (i.e. kolla-ansible upgrade).

On 23/01/17 00:06, Steven Dake (stdake) wrote:
> Kris,
>
>
>
> Thanks for adding the kolla tag.  As we reach milestone 3, every kolla
> dev is knee deep in development and probably not reading mails not
> directly tagged with [kolla].
>
>
>
> IOutlook 2016 has a bug where I cannot respond inline.  I am replying
> here so my gmail account picks up the email so that I can respond inline.
>
>
>
> It will take me several hours to process your email and I’m not certain
> I can answer all of the questions to your satisfaction as kolla is a
> pretty large project and it is difficult for any one person to know all
> of the details.  I will hopefully have a response sometime this evening
> for the things I can answer.  Hopefully others in the community can fill
> in the gaps.
>
>
>
> Regards
>
> -steve
>
>
>
>
>
>
>
>     *From: *"Kris G. Lindgren" <klindgren at godaddy.com>
>     *Reply-To: *"OpenStack Development Mailing List (not for usage
>     questions)" <openstack-dev at lists.openstack.org>
>     *Date: *Friday, January 20, 2017 at 6:03 PM
>     *To: *"openstack-dev at lists.openstack.org"
>     <openstack-dev at lists.openstack.org>
>     *Cc: *"openstack-operators at lists.openstack.org"
>     <openstack-operators at lists.openstack.org>
>     *Subject: *Re: [openstack-dev] [kolla-ansible] [kolla] Am I doing
>     this wrong?
>
>
>
>     Adding [kolla] tag.
>
>
>
>
>
>     ___________________________________________________________________
>
>     Kris Lindgren
>
>     Senior Linux Systems Engineer
>
>     GoDaddy
>
>
>
>     *From: *"Kris G. Lindgren" <klindgren at godaddy.com>
>     *Date: *Friday, January 20, 2017 at 4:54 PM
>     *To: *"openstack-dev at lists.openstack.org"
>     <openstack-dev at lists.openstack.org>
>     *Cc: *"openstack-operators at lists.openstack.org"
>     <openstack-operators at lists.openstack.org>
>     *Subject: *Re: [kolla-ansible] Am I doing this wrong?
>
>
>
>     Poke.  Bueller?
>
>
>
>
>
>     ___________________________________________________________________
>
>     Kris Lindgren
>
>     Senior Linux Systems Engineer
>
>     GoDaddy
>
>
>
>     *From: *"Kris G. Lindgren" <klindgren at godaddy.com>
>     *Date: *Tuesday, January 10, 2017 at 5:34 PM
>     *To: *"openstack-dev at lists.openstack.org"
>     <openstack-dev at lists.openstack.org>
>     *Subject: *[kolla-ansible] Am I doing this wrong?
>
>
>
>     Hello Kolla/Kolla-ansible peoples.
>
>
>
>     I have been trying to take kolla/kolla-ansible and use it to start
>     moving our existing openstack deployment into containers.  At the
>     same time also trying to fix some of the problems that we created
>     with our previous deployment work (everything was in puppet).  Where
>     we had puppet doing **everything** which eventually created a system
>     that effectively performed actions at a distance.  As we were never
>     really 100% what puppet was going to do when we ran it.  Even with
>     NOOP mode enabled.  So taking an example of building and deploying
>     glance via kolla-ansible. I am running into some problems/concerns
>     and wanted to reach out to make sure that I am not missing something.
>
>
>
>     Things that I am noticing:
>
>      * I need to define a number of servers in my inventory outside of
>     the specific servers that I want to perform actions against.  I need
>     to define groups baremetal, rabbitmq, memcached, and control (IN
>     addition to the glance specific groups) most of these seem to be
>     gathering information for config? (Baremetal was needed soley to try
>     to run the bootstrap play).  Running a change specifically against
>     "glance" causes fact gathering on a number of other servers not
>     specifically where glance is running?  My concern here is that I
>     want to be able to run kola-ansible against a specific service and
>     know that only those servers are being logged into.
>
>
>
>     * I want to run a dry-run only, being able to see what will happen
>     before it happens, not during; during makes it really hard to see
>     what will happen until it happens. Also supporting  `ansible --diff`
>     would really help in understanding what will be changed (before it
>     happens).  Ideally, this wouldn’t be 100% needed.  But the ability
>     to figure out what a run would **ACTUALLY** do on a box is what I
>     was hoping to see.
>
>
>
>     * Database task are ran on every deploy and status of change DB
>     permissions always reports as changed? Even when nothing happens,
>     which makes you wonder "what changed"?  Seems like this is because
>     the task either reports a 0 or a 1, where it seems like there is 3
>     states, did nothing, updated something, failed to do what was
>     required.  Also, Can someone tell me why the DB stuff is done on a
>     deployment task?  Seems like the db checks/migration work should
>     only be done on a upgrade or a bootstrap?
>
>
>
>     * Database services (that at least we have) our not managed by our
>     team, so don't want kolla-ansible touching those (since it won't be
>     able to). No way to mark the DB as "externally managed"?  IE we dont
>     have permissions to create databases or add users.  But we got all
>     other permissions on the databases that are created, so normal
>     db-manage tooling works.
>
>
>
>     * Maintenance level operations; doesn't seem to be any built-in to
>     say 'take a server out  of a production state, deploy to it, test
>     it, put it back into production'  Seems like if kola-ansible is
>     doing haproxy for API's, it should be managing this?  Or an
>     extension point to allow us to run our own maintenance/testing scripts?
>
>
>
>     * Config must come from kolla-ansible and generated templates.  I
>     know we have a patch up for externally managed service
>     configuration.  But if we aren't suppose to use kolla-ansible for
>     generating configs (see below), why cant we override this piece?
>
>
>
>     Hard to determine what kolla-ansible *should* be used for:
>
>
>
>     * Certain parts of it are 'reference only' (the config tasks), some
>     are not recommended
>
>       to be used at all (bootstrap?); what is the expected parts of
>     kolla-ansible people are
>
>       actually using (and not just as a reference point); if parts of
>     kolla-ansible are just
>
>       *reference only* then might as well be really upfront about it and
>     tell people how to
>
>       disable/replace those reference pieces?
>
>
>
>     * Seems like this will cause everyone who needs to make tweaks to
>     fork or create an "overlay" to override playbooks/tasks with
>     specific functions?
>
>
>
>     Other questions:
>
>
>
>     Is kolla-ansibles design philosophy that every deployment is an
>     upgrade?  Or every deployment should include all the base level
>     boostrap tests?
>
>
>
>     Because it seems to me that you have a required set of tasks that
>     should only be done once (boot strap).  Another set of tasks that
>     should be done for day to day care/feeding: service restarts, config
>     changes, updates to code (new container deployments), package
>     updates (new docker container deployment).  And a final set of tasks
>     for upgrades where you will need to do things like db migrations and
>     other special upgrade things.  It also seems like the day to day
>     care and feeding tasks should be incredibly targeted/explicit. For
>     example, deploying a new glance container (not in an upgrade
>     scenario).  I would expect it to login to the glance servers one at
>     a time.  Place the server in maintenance mode to ensure that actions
>     are not performed against it.  Downloaded the new container.  Start
>     the new container.  Test the new container, if successful, place the
>     new container into rotation.  Stop the old container.  Remove the
>     server from maintenance mode.  Move on to the next server.  All of
>     that would only need to involve login into the glance servers.  In
>     testing kola-ansible it does not seem like the act of deploying is
>     that targeted?
>
>
>
>
>
>     ___________________________________________________________________
>
>     Kris Lindgren
>
>     Senior Linux Systems Engineer
>
>     GoDaddy
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>