Open Stack

Just to followup since the required packages are finally available, the
patches have been updated and are passing CI now.

https://review.openstack.org/#/c/202763/

-Alex

On Fri, Jul 24, 2015 at 8:32 AM, Alex Schultz <aschultz at mirantis.com> wrote:

> Unfortunately we got stuck with package availability issues. It was
> successful without using librarian packages for building, but since we need
> to leverage packages the work to get packaged versions of the code has
> unfortunately blocked this. I've got the packages built, but the time to
> get them to an publicly available mirror has killed this effort. Since it's
> now past FF, I don't think this will be for 7.0 but we should be in a good
> spot to start early for 8.0.  Also I've switched to librarian-puppet-simple
> since it has far fewer dependencies and we only want to support git
> references for packages.  Once that makes it to the mirror and can pass CI,
> this change would in theory be ready.  I'm continuing to keep an eye on the
> changes so that we'll be ready to start running with them when the time
> comes.
>
> -Alex
>
> On Fri, Jul 24, 2015 at 1:41 AM, Mike Scherbakov <mscherbakov at mirantis.com
> > wrote:
>
>> Do we have any success here.. ?
>>
>> On Mon, Jul 20, 2015 at 8:32 AM Alex Schultz <aschultz at mirantis.com>
>> wrote:
>>
>>> Vladimir,
>>>
>>> Thanks.  Can you point me to the error for perestroika? I'd be happy to
>>> take a look as well. I spent most of Friday throwing various options at the
>>> CI system to try and figure out how to get the spec to work with the CI
>>> fuel-library package building so perhaps there's a different way to handle
>>> this in the spec.
>>>
>>> -Alex
>>>
>>> On Mon, Jul 20, 2015 at 10:02 AM, Vladimir Kozhukalov <
>>> vkozhukalov at mirantis.com> wrote:
>>>
>>>> Alex,
>>>>
>>>> As I've just found out this package available here [1] is not actually
>>>> build with your patch (instead it is from previous successful build). Looks
>>>> like "perestroika" can not build this package due to some environment
>>>> related issues. I've poked Dmitry Burmistrov to check it out.
>>>> However, your patch is OK, make system can build this package and ISO
>>>> passes BVT tests.
>>>>
>>>>
>>>> [1]
>>>> http://172.18.160.74/osci/review/CR-202763/mos/7.0/fuel/base/centos6/Packages/fuel-library7.0-7.0.0-1.mos6888.git.bac86fe.noarch.rpm
>>>>
>>>> Vladimir Kozhukalov
>>>>
>>>> On Mon, Jul 20, 2015 at 4:04 PM, Vladimir Kozhukalov <
>>>> vkozhukalov at mirantis.com> wrote:
>>>>
>>>>> Alex,
>>>>>
>>>>> Yes, spec is much better place to introduce this. BTW, "perestroika"
>>>>> builds new package for every commit and patch set and publishes them via
>>>>> HTTP. Please check here
>>>>> http://172.18.160.74/osci/review/CR-202763/mos/7.0/fuel/base/centos6/Packages/fuel-library7.0-7.0.0-1.mos6888.git.bac86fe.noarch.rpm
>>>>> if the package contains all necessary upstream modules.
>>>>>
>>>>> Vladimir Kozhukalov
>>>>>
>>>>> On Sat, Jul 18, 2015 at 3:28 AM, Alex Schultz <aschultz at mirantis.com>
>>>>> wrote:
>>>>>
>>>>>> Not until we start using it then any ci that tests with that module
>>>>>> will validate the modules inclusion. You can check the output of the jobs
>>>>>> as we are printing what modules are managed by librarian.
>>>>>>
>>>>>> -Alex
>>>>>> On Jul 17, 2015 6:17 PM, "Andrew Woodward" <xarses at gmail.com> wrote:
>>>>>>
>>>>>>> Fantastic, do we have some way to validate that the module was
>>>>>>> pulled in properly as part of fuel-library CI?
>>>>>>>
>>>>>>> On Fri, Jul 17, 2015 at 2:48 PM Alex Schultz <aschultz at mirantis.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hey All,
>>>>>>>>
>>>>>>>> I've figured it out without having to modify the fuel-main build
>>>>>>>> code. I've updated the fuel-library spec with a build action that invokes
>>>>>>>> the script to pull down external modules.  Please take some time to review
>>>>>>>> the two reviews out there for this change to see if there are any issues
>>>>>>>> with the way it is implemented.
>>>>>>>>
>>>>>>>> https://review.openstack.org/#/c/202763/
>>>>>>>> https://review.openstack.org/#/c/202767/
>>>>>>>>
>>>>>>>> This is a first step towards being able to pull in unmodified
>>>>>>>> external puppet modules.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> -Alex
>>>>>>>>
>>>>>>>> On Fri, Jul 17, 2015 at 4:23 PM, Andrew Woodward <
>>>>>>>> awoodward at mirantis.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jul 17, 2015 at 12:24 PM Vladimir Kozhukalov <
>>>>>>>>> vkozhukalov at mirantis.com> wrote:
>>>>>>>>>
>>>>>>>>>> Alex,
>>>>>>>>>>
>>>>>>>>>> Great that you did this. Now I think I can prepare fuel-main
>>>>>>>>>> patch to invoke this script right before building fuel-library package.
>>>>>>>>>> I'll add you to review it. Is it ok if I do this monday morning?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Keep in minde we agreeded to a deadline to get this sorted and in
>>>>>>>>> shape to land by EOD monday or we will have to retain the old, and crappy
>>>>>>>>> fork method. If possible please work out how this needs to work as early as
>>>>>>>>> possible so Alex can continue.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Vladimir Kozhukalov
>>>>>>>>>>
>>>>>>>>>> On Fri, Jul 17, 2015 at 5:51 PM, Alex Schultz <
>>>>>>>>>> aschultz at mirantis.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Vladimir,
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jul 17, 2015 at 7:33 AM, Vladimir Kozhukalov <
>>>>>>>>>>> vkozhukalov at mirantis.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Alex,
>>>>>>>>>>>>
>>>>>>>>>>>> Gathering upstream modules certainly should be implemented as a
>>>>>>>>>>>> separate script so as to make it possible to use it wherever we need this
>>>>>>>>>>>> (tests, builds, etc.) According to builds there are two things
>>>>>>>>>>>>
>>>>>>>>>>>> 1) We have so called "perestroika" package build system (Dmitry
>>>>>>>>>>>> Burmistrov is a main contributor here). By the end of next week we are
>>>>>>>>>>>> going to switch building all the packages to "perestroika". And in order to
>>>>>>>>>>>> gather upstream modules right before building fuel-library package, we need
>>>>>>>>>>>> to change perestroika build scripts.
>>>>>>>>>>>>
>>>>>>>>>>>> 2) Currently we build packages using make system and you are
>>>>>>>>>>>> right about the place where you need to make changes.
>>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82
>>>>>>>>>>>> If you create shell script, I'll help you to add it to make code.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I have updated my review[0] to extract the update logic to it's
>>>>>>>>>>> own bash script that can be invoked by the build scripts.  Let me know what
>>>>>>>>>>> would be the best way to wedge this in there.  I think for the
>>>>>>>>>>> "perestroika" this would also be needed for the fuel-library build, so if
>>>>>>>>>>> you point me at that I can see if I can help make that change as well.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> -Alex
>>>>>>>>>>>
>>>>>>>>>>> [0] https://review.openstack.org/#/c/202763/
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Vladimir Kozhukalov
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jul 17, 2015 at 2:56 PM, Aleksandr Didenko <
>>>>>>>>>>>> adidenko at mirantis.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I believe build_repo function is the best way to do this [0].
>>>>>>>>>>>>> So for fuel-library we'll need to run a shell script right from the repo
>>>>>>>>>>>>> before 'touch $$@'. We can make it either conditional ( test -f
>>>>>>>>>>>>> ./path/additional_build_script.sh && bash ./path/additional_build_script.sh
>>>>>>>>>>>>> ) or as additional parameter to function and add it in fuel-library call [1]
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Alex
>>>>>>>>>>>>>
>>>>>>>>>>>>> [0]
>>>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/repos.mk#L16-L37
>>>>>>>>>>>>> [1]
>>>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/repos.mk#L45
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jul 17, 2015 at 2:37 PM, Alex Schultz <
>>>>>>>>>>>>> aschultz at mirantis.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey Alex,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Jul 17, 2015 4:32 AM, "Aleksandr Didenko" <
>>>>>>>>>>>>>> adidenko at mirantis.com> wrote:
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Hi,
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > I think that we should provide a separate script that will
>>>>>>>>>>>>>> fetch the upstream modules into fuel-library/deployment/puppet/ directory.
>>>>>>>>>>>>>> It will allow us to have everything in a single place and use this script
>>>>>>>>>>>>>> in ISO build process and CI jobs.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Right. That is what I'm going for. The issue I need help with
>>>>>>>>>>>>>> is the best way to execute this as part of the build process.  From what i
>>>>>>>>>>>>>> understand of the build process is that we are using git archive for all
>>>>>>>>>>>>>> pieces so I'm not sure how to wedge in an extra script execution to do the
>>>>>>>>>>>>>> module fetch.  The creation of the script isn't the issue, the issue is how
>>>>>>>>>>>>>> can I properly run it as part of the build process.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > Regards,
>>>>>>>>>>>>>> > Alex
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> -Alex
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> > On Thu, Jul 16, 2015 at 11:17 PM, Alex Schultz <
>>>>>>>>>>>>>> aschultz at mirantis.com> wrote:
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> Hello everyone,
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> I have committed the initial configuration required to
>>>>>>>>>>>>>> start leveraging librarian-puppet as part of the way we pull in upstream
>>>>>>>>>>>>>> puppet modules[0]. Additionally, I have also committed a change that would
>>>>>>>>>>>>>> pull in the openstack-ironic module[1].  The one piece that is missing from
>>>>>>>>>>>>>> this being a complete solution is the ability to run librarian-puppet as
>>>>>>>>>>>>>> part of our build process for the fuel-library.  I've looked into the
>>>>>>>>>>>>>> fuel-main build scripts and I think it's over my head to figure this out
>>>>>>>>>>>>>> just by looking. Can anyone explain to me or assist me in how I could go
>>>>>>>>>>>>>> about modifying the existing build system to be able to run
>>>>>>>>>>>>>> librarian-puppet to prepare the source for the package?  In my initial
>>>>>>>>>>>>>> investigation, it looks like it would be a modification of the
>>>>>>>>>>>>>> fuel-main/packages/module.mk[3] file.  I basically need to
>>>>>>>>>>>>>> do the prepare_library[3] function from the 202763 review[0] after we've
>>>>>>>>>>>>>> pulled all the sources together to fetch the upstream modules.
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> Thanks,
>>>>>>>>>>>>>> >> -Alex
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> [0] https://review.openstack.org/202763
>>>>>>>>>>>>>> >> [1] https://review.openstack.org/202767
>>>>>>>>>>>>>> >> [2]
>>>>>>>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82
>>>>>>>>>>>>>> >> [3]
>>>>>>>>>>>>>> https://review.openstack.org/#/c/202763/1/utils/jenkins/fuel_noop_tests.rb
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> __________________________________________________________________________
>>>>>>>>>>>>>> >> OpenStack Development Mailing List (not for usage
>>>>>>>>>>>>>> questions)
>>>>>>>>>>>>>> >> Unsubscribe:
>>>>>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> __________________________________________________________________________
>>>>>>>>>>>>>> > OpenStack Development Mailing List (not for usage questions)
>>>>>>>>>>>>>> > Unsubscribe:
>>>>>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> __________________________________________________________________________
>>>>>>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>>>>>>>> Unsubscribe:
>>>>>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> __________________________________________________________________________
>>>>>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>>>>>>> Unsubscribe:
>>>>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> __________________________________________________________________________
>>>>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>>>>>> Unsubscribe:
>>>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>>>>>
>>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> __________________________________________________________________________
>>>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>>>>> Unsubscribe:
>>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> __________________________________________________________________________
>>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>>>> Unsubscribe:
>>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> --
>>>>>>>>> Andrew Woodward
>>>>>>>>> Mirantis
>>>>>>>>> Fuel Community Ambassador
>>>>>>>>> Ceph Community
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> __________________________________________________________________________
>>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>>> Unsubscribe:
>>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> __________________________________________________________________________
>>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>>> Unsubscribe:
>>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Andrew Woodward
>>>>>>>
>>>>>>> Mirantis
>>>>>>>
>>>>>>> Fuel Community Ambassador
>>>>>>>
>>>>>>> Ceph Community
>>>>>>>
>>>>>>>
>>>>>>> __________________________________________________________________________
>>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>>> Unsubscribe:
>>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> __________________________________________________________________________
>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>> Unsubscribe:
>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> --
>> Mike Scherbakov
>> #mihgen
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150724/609e21bf/attachment-0001.html>