Open Stack

Hi Steve

  I'm just suggesting the bi-frost stuff allow sufficient 'hooks' for
operators to insert site specific setup.  Not that Kolla/Bi-Frost try
to handle 'everything'.

  For instance LDAP... Horizon integration with LDAP would certainly
be within Kolla's perview.  However, operators also use LDAP for login
access to the host via PAM.  This is site-specific, and outside of
Kolla's mission.

  As an example of 'respecting existing configuration' - some sites
may use OpenVSwitch for host level networking.  Kolla currently starts
a new openvswitchdb container without checking if OpenVSwitch is
already running - this kills the host networking.

  If you'll pardon the pun, there are a 'host' of situations like
this, where operators will have to provide
(possibly many/detailed) site specific configurations to a bare metal
host.  Networking, Security, Backups, Monitoring/Logging, etc.  These
may all be subject to corporate wide policies that are non-negotiable
and have to be followed.

  Again, I realize Kolla/BiFrost can not be everything for everyone.
I just want to suggest that we provide well documented methods for
operators to insert site-specific roles/plays/whatever, and that we
take care to avoid 'stepping on' things.

  I have no idea as to what/how-many 'hooks' would be required.  I
tend to think a simple 'before-bifrost' role and 'after-bifrost' role
would be enough.  However, others may have more input on that...
I like the idea of using roles, as that would allow you to centralize
all your 'site-specific' bits.  This way operators don't have to
modify the existing kolla/BiFrost stuff.


On Sat, Jul 2, 2016 at 3:10 PM, Steven Dake (stdake) <stdake at cisco.com> wrote:
> Stephen,
>
> Responses inline.
>
> On 7/1/16, 11:35 AM, "Stephen Hindle" <shindle at llnw.com> wrote:
>
>>Maybe I missed it - but is there a way to provide site specific
>>configurations?  Things we will run into in the wild include:
>>    Configuring multiple non-openstack nics
>
> We don¹t have anything like this at present or planned.  Would you mind
> explaining the use case?  Typically we in the Kolla community expect a
> production deployment to only deploy OpenStack, and not other stacks on
> top of the bare metal hardware.  This is generally considered best
> practice at this time, unless of course your deploying something on top of
> OpenStack that may need these nics.  The reason is that OpenStack itself
> managed alongside another application doesn¹t know what it doesn't know
> and can't handle capacity management or any of a number of other things
> required to make an OpenStack cloud operate.
>
>>     IPMI configuration
>
> BiFrost includes IPMI integration - assumption being we will just use
> whatever BiFrost requires here for configuration.
>
>>     Password integration with Corporate LDAP etc.
>
> We have been asked several times for this functionality, and it will come
> naturally during either Newton or Occata.
>
>>     Integration with existing SANs
>
> Cinder integrates with SANs, and in Newton, we have integration with
> iSCSI.  Unfortunately because of some controversy around how glance should
> provide images with regards to cinder, using existing SAN gear with iSCSI
> integration as is done by Cinder may not work as expected in a HA setup.
>
>>     Integration with existing corporate IPAM
>
> No idea
>
>>     Corporate Security policy (firewall rules, sudo groups,
>>hosts.allow, ssh configs,etc)
>
> This is environmental specific and its hard to make a promise on what we
> could deliver in a generic way that would be usable by everyone.
> Therefore our generic implementation will be the "bare minimum" to get the
> system into an operational state.  The things listed above are outside the
> "bare minimum" iiuc.
>
>>
>>Thats just off the top of my head - I'm sure we'll run into others.  I
>>tend to think the best way
>>to approach this is to allow some sort of 'bootstrap' role, that could
>>be populated by the
>>operators.  This should initially be empty (Kolla specific 'bootstrap'
>
> Our bootstrap playbook is for launching BiFrost and bringing up the bare
> metal machines with an SSH credential.  It appears from this thread we
> will have another playbook to do the bare metal initialization (thiings
> like turning off firewalld, turning on chrony, I.e. Making the bare metal
> environment operational for OpenStack)
>
> I think what you want is a third playbook which really belongs in the
> domain of the operators to handle site-specific configuration as required
> by corporate rules and the like.
>
>
>>actions should be
>>in another role) to prevent confusion.
>>
>>We also have to be careful that kolla doesn't stomp on any non-kolla
>>configuration...
>
> Could you expand here.  Kolla currently expects the machines under its
> control to be only OpenStack machines, and not have other applications
> running on them.
>
> Hope that was helpful.
>
> Regards
> -steve
>
>>
>>
>>On Thu, Jun 30, 2016 at 12:43 PM, Mooney, Sean K
>><sean.k.mooney at intel.com> wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Steven Dake (stdake) [mailto:stdake at cisco.com]
>>>> Sent: Monday, June 27, 2016 9:21 PM
>>>> To: OpenStack Development Mailing List (not for usage questions)
>>>> <openstack-dev at lists.openstack.org>
>>>> Subject: Re: [openstack-dev] [kolla][ironic] My thoughts on Kolla +
>>>> BiFrost integration
>>>>
>>>>
>>>>
>>>> On 6/27/16, 11:19 AM, "Devananda van der Veen"
>>>> <devananda.vdv at gmail.com>
>>>> wrote:
>>>>
>>>> >At a quick glance, this sequence diagram matches what I
>>>> >envisioned/expected.
>>>> >
>>>> >I'd like to suggest a few additional steps be called out, however I'm
>>>> >not sure how to edit this so I'll write them here.
>>>> >
>>>> >
>>>> >As part of the installation of Ironic, and assuming this is done
>>>> >through Bifrost, the Actor should configure Bifrost for their
>>>> >particular network environment. For instance: what eth device is
>>>> >connected to the IPMI network; what IP ranges can Bifrost assign to
>>>> >physical servers; and so on.
>>>> >
>>>> >There are a lot of other options during the install that can be
>>>> >changed, but the network config is the most important. Full defaults
>>>> >for this roles' config options are here:
>>>> >
>>>> >https://github.com/openstack/bifrost/blob/master/playbooks/roles/bifro
>>>> s
>>>> >t-i
>>>> >ronic-install/defaults/main.yml
>>>> >
>>>> >and documentation is here:
>>>> >
>>>> >https://github.com/openstack/bifrost/tree/master/playbooks/roles/bifro
>>>> s
>>>> >t-i
>>>> >ronic-install
>>>> >
>>>> >
>>>> >
>>>> >Immediately before "Ironic PXE boots..." step, the Actor must perform
>>>> >an action to "enroll" hardware (the "deployment targets") in Ironic.
>>>> >This could be done in several ways: passing a YAML file to Bifrost;
>>>> >using the Ironic CLI; or something else.
>>>> >
>>>> >
>>>> >"Ironic reports success to the bootstrap operation" is ambiguous.
>>>> >Ironic does not currently support notifications, so, to learn the
>>>> >status of the deployments, you will need to poll the Ironic API (eg,
>>>> >"ironic node-list").
>>>> >
>>>>
>>>> Great,
>>>>
>>>> Thanks for the feedback.  I'll integrate your changes into the sequence
>>>> diagram when I have a free hour or so - whenever that is :)
>>>>
>>>> Regards
>>>> -steve
>>> [Mooney, Sean K] I agree with most of devananda points and had come to
>>>similar
>>> Conlcutions.
>>>
>>> At a highlevel I think the workflow from 0 to cloud would be as follow.
>>> Assuming you have one linux system.
>>> - clone http://github.com/openstack/kolla && cd kolla
>>> - tools/kolla-host build-host-deploy
>>>         This will install ansible if not installed then invoke a
>>>playbook to install
>>>         All build dependencies and generate the kolla-build.conf
>>>passwords.yml and global.yml.
>>>      Install kolla python package
>>> - configure kolla-build.conf as required
>>> - tools/build.py or kolla-build to build image
>>> - configure global.yml and or biforst specific file
>>>   This would involve specifying a file that can be used with bifrost
>>>dynamic inventory.
>>>   Configuring network interface for bifrost to use.
>>>   Enable ssh-key generate or supply one to use as the key to us when
>>>connecting to the servers post deploy.
>>>   Configure diskimage builder options or supply path to a file on the
>>>system to use as your os image.
>>> - tools/kolla-host deploy-bifrost
>>>   Deploys bifrost container.
>>>   Copies images/keys
>>>   Bootstraps bifrost and start services.
>>> - tools/kolla-host deploy-servers
>>>   Invokes bifrost enroll and deploy dynamic then polls until all
>>>   Servers are provisioned or a server fails.
>>> - tools/kolla-hosts bootstrap-servers
>>>   Installs all kolla deploy dependencies
>>>   Docker ect. This will also optionally do things such as
>>>   Configure hugepages, configure cpu isolation, firewall settings
>>>   Or any other platform level config for example apply labels to ceph
>>>   Disks .
>>>   This role will reboot the remote server at the end of the role if
>>>required
>>>   e.g. after installing The wily kernel on Ubuntu 14.04
>>> - configure global.yml as normal
>>> - tools/kolla-ansible prechecks (this should now pass)
>>> - tools/kolla-ansible deploy
>>> - profit
>>>
>>> I think this largely agrees with the diagram you proposed but has a
>>>couple of extra steps/details.
>>>
>>>>
>>>> >
>>>> >
>>>> >Cheers,
>>>> >--Devananda
>>>> >
>>>> >On 06/23/2016 06:54 PM, Steven Dake (stdake) wrote:
>>>> >> Hey folks,
>>>> >>
>>>> >> I created the following sequence diagram to show my thinking on
>>>> >>Ironic  integration.  I recognize some internals of the recently
>>>> >>merged bifrost changes  are not represented in this diagram.  I would
>>>> >>like to see a bootstrap action do  all of the necessary things to
>>>> >>bring up BiFrost in a container using Sean's WIP  Kolla patch
>>>> followed
>>>> >>by bare metal minimal OS load followed by Kolla dependency  software
>>>> >>(docker-engine, docker-py, and ntpd) loading and initialization.
>>>> >>
>>>> >> This diagram expects ssh keys to be installed on the deployment
>>>> >>targets via BiFrost.
>>>> >>
>>>> >> https://creately.com/diagram/ipt09l352/ROMDJH4QY1Avy1RYhbMUDraaQ4%3D
>>>> >>
>>>> >> Thoughts welcome, especially from folks in the Ironic community or
>>>> >>Sean who is  leading this work in Kolla.
>>>> >>
>>>> >> Regards,
>>>> >> -steve
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>_____________________________________________________________________
>>>> _
>>>> >>___
>>>> >>_
>>>> >> OpenStack Development Mailing List (not for usage questions)
>>>> >> Unsubscribe:
>>>> >>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> >>
>>>> >
>>>> >______________________________________________________________________
>>>> _
>>>> >___ OpenStack Development Mailing List (not for usage questions)
>>>> >Unsubscribe:
>>>> >OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________________________________
>>>> ___
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe: OpenStack-dev-
>>>> request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>_________________________________________________________________________
>>>_
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>--
>>Stephen Hindle - Senior Systems Engineer
>>480.807.8189 480.807.8189
>>www.limelight.com Delivering Faster Better
>>
>>Join the conversation
>>
>>at Limelight Connect
>>
>>--
>>The information in this message may be confidential.  It is intended
>>solely
>>for
>>the addressee(s).  If you are not the intended recipient, any disclosure,
>>copying or distribution of the message, or any action or omission taken
>>by
>>you
>>in reliance on it, is prohibited and may be unlawful.  Please immediately
>>contact the sender if you have received this message in error.
>>
>>
>>__________________________________________________________________________
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Stephen Hindle - Senior Systems Engineer
480.807.8189 480.807.8189
www.limelight.com Delivering Faster Better

Join the conversation

at Limelight Connect

-- 
The information in this message may be confidential.  It is intended solely 
for
the addressee(s).  If you are not the intended recipient, any disclosure,
copying or distribution of the message, or any action or omission taken by 
you
in reliance on it, is prohibited and may be unlawful.  Please immediately
contact the sender if you have received this message in error.