[openstack-dev] [kolla] Tags, revisions, dockerhub
Joshua Harlow
harlowja at fastmail.com
Mon Apr 24 21:10:23 UTC 2017
Just for some insight,
This is the stuff we are putting into our kolla image(s):
{% block openstack_base_header %}
ADD additions-archive /
# Because we have a internal pip with internal packages.
RUN cp /additions/pip/pip.conf /etc/pip.conf
ENV PIP_CONFIG_FILE /etc/pip.conf
LABEL JENKINS_BUILD=${jenkins_job.build_number}
LABEL JENKINS_BUILD_TAG=${jenkins_job.build_tag}
LABEL JENKINS_BUILD_URL=${jenkins_job.build_url}
{% endblock %}
So the above shows up, and can be linked back into the jenkins job that
built the images.
Then inside the image we have the following:::
$ cat /job.json | python -mjson.tool
{
"build_id": "15",
"build_name": null,
"build_number": "15",
"build_tag": "jenkins-cinder-15",
"build_url": "https://cloud1.jenkins.int.godaddy.com/job/cinder/15/",
"kolla_customizations": {
"extra_requirements": [],
"repositories": [
{
"name": "kolla",
"ref": "stable/newton",
"url": "git://git.openstack.org/openstack/kolla"
},
{
"name": "clean",
"ref": "7.0.1",
"url": "git://git.openstack.org/openstack/cinder.git"
},
{
"name": "dirty",
"ref": "7.0.1",
"url": "git://git.openstack.org/openstack/cinder.git"
},
{
"name": "deploy",
"ref": "master",
"url":
"git at github.secureserver.net:cloudplatform/openstack-deploy.git"
},
{
"name": "requirements",
"ref": "stable/liberty",
"url": "git://git.openstack.org/openstack/requirements"
}
],
"template_overrides": "\n\n{% set
cinder_volume_packages_override = ['python-rados', 'python-rbd', 'ceph']
%}\n\n"
},
"project": {
"author": "OpenStack",
"fullname": "cinder-7.0.1",
"name": "cinder",
"url": "http://www.openstack.org/",
"version": "7.0.1"
},
"run": {
"deploy_git": null,
"deploy_ref": null,
"git_target_repo": null,
"git_target_repo_branch": null,
"kolla_git": null,
"kolla_image_namespace": null,
"kolla_ref": null,
"maintainers": null,
"project": null,
"project_git": null,
"project_ref": null,
"pushing_to_artifactory": null,
"requirement_git": null,
"requirement_ref": null,
"running_unit_tests": null,
"unit_test_path": null
}
}
Though the run dict values are `null`, gotta fix that, haha.
But the general idea is hopefully clear.
-Josh
Steve Baker wrote:
>
>
> On Thu, Apr 20, 2017 at 8:12 AM, Michał Jastrzębski <inc007 at gmail.com
> <mailto:inc007 at gmail.com>> wrote:
>
> So after discussion started here [1] we came up with something like
> that:
>
> 1. Docker build will create "fingerprint" - manifesto of versions
> saved somewhere (LABEL?)
>
>
> This would be great, especially a full package version listing in an
> image label. However I don't see an easy way of populating a label from
> data inside the image. Other options could be:
> - have a script inside the image in a known location which generates the
> package manifest on the fly, do a docker run whenever you need to get a
> manifest to compare with another image.
> - write out the package list during image build to a known location, do
> a docker run to cat out its contents when needed
>
> As for the format, taking a yum only image as an example would we need
> anything more than the output of "rpm -qa | sort"?
>
> 2. We create new CLI tool kolla-registry for easier management of
> pushing and versioning
> 3. kolla-registry will be able to query existing source docker
> registry (incl. dockerhub) for latest tag-revision and it's version
> manifesto, also dest registry for tags-revisions and manifesto
> 4. if source image manifesto != dest image manifesto -> push source
> image to dest registry and increase tag-revision by 1
> 5. kolla-registry will output latest list of images:tags-revisions
> available for kolla-k8s/ansible to consume
> 6. we keep :4.0.0 style images for every tag in kolla repository.
> These are static and will not be revised.
>
>
> Yes, this is fine, but please keep in mind that this change[1] could be
> merged without changing these published 4.0.0 style image tags, with the
> added advantage of locally built images with a git checkout of kolla
> have a less ambiguous default tag.
> [1] https://review.openstack.org/#/c/448380/
>
> Different scenerios can be handled this way
> 1. Autopushing to dockerhub will query freshest built registry
> (tarballs, source) and and dockerhub (dest), it will create
> image:branchname (nova-api:ocata) for HEAD of stable branch every run
> and image:branchname-revision with revision increase
> 2. Users will have easy time managing their local registry - dockerhub
> (source) and local (dest), if nova-api:ocata on dockerhub is newer
> than local, pull it to local and increase local tip and revision
>
> Thoughts?
>
>
> Generally positive :)
>
>
> [1]
> http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-04-19.log.html#t2017-04-19T19:10:25
> <http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-04-19.log.html#t2017-04-19T19:10:25>
>
> On 19 April 2017 at 10:45, Fox, Kevin M <Kevin.Fox at pnnl.gov
> <mailto:Kevin.Fox at pnnl.gov>> wrote:
> > That works for detecting changes in the build system.
> >
> > It does not solve the issue of how to keep containers atomic on
> end user systems.
> >
> > All images in a k8s deployment should be the same image. This is
> done by specifying the same tag. When a new update is done, the
> updated deployment should specify a new tag to distinguish it from
> the old tag so that roll forwards/roll backs work atomically and as
> designed. Otherwise, roll back can actually break or roll forward
> wont actually grab newer images.
> >
> > Thanks,
> > Kevin
> >
> > ________________________________________
> > From: Michał Jastrzębski [inc007 at gmail.com <mailto:inc007 at gmail.com>]
> > Sent: Wednesday, April 19, 2017 8:32 AM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [kolla] Tags, revisions, dockerhub
> >
> > I think LABEL is great idea for all the "informative" stuff. In fact
> > if we could somehow abuse LABEL to fill it up after we get packages
> > installed, we could use it for version manifesto. That will make
> logic
> > around "if version changed" much easier since we'll have easy access
> > to this information on both image and container.
> >
> > Our autopushing mechanism will work with tags and HEAD of stable
> > branch in this case.
> >
> > Kevin, then your use case would be done like that:
> > 1. pull container nova-compute:ocata, tag it locally to
> > nova-compute:ocata-deployed, deploy it
> > 2. every now and then pull fresh nova-compute:ocata from dockerhub
> > 3. compare versions in LABELs to see whether you want to upgrade
> or not
> > 4. if you do, retag :ocata-deployed to :ocata-old, :ocata to
> > :ocata-deployed and run upgrade
> > 5. keep ocata-old, revision it, backup it for as long as you want
> >
> > I also think that we can ship utils to do this in kolla, so people
> > won't need to write these themselves.
> >
> > Does that work?
> >
> > Cheers,
> > Michal
> >
> > On 19 April 2017 at 05:02, Flavio Percoco <flavio at redhat.com
> <mailto:flavio at redhat.com>> wrote:
> >> On 19/04/17 11:20 +0100, Paul Bourke wrote:
> >>>
> >>> I'm wondering if moving to using docker labels is a better way
> of solving
> >>> the various issue being raised here.
> >>>
> >>> We can maintain a tag for each of master/ocata/newton/etc, and
> on each
> >>> image have a LABEL with info such as 'pbr of service/pbr of
> kolla/link to CI
> >>> of build/etc'. I believe this solves all points Kevin mentioned
> except
> >>> rollback, which afaik, OpenStack doesn't support anyway. It
> also solves
> >>> people's concerns with what is actually in the images, and is a
> standard
> >>> Docker mechanism.
> >>>
> >>> Also as Michal mentioned, if users are concerned about keeping
> images,
> >>> they can tag and stash them away themselves. It is overkill to
> maintain
> >>> hundreds of (imo meaningless) tags in a registry, the majority
> of which
> >>> people don't care about - they only want the latest of the
> branch they're
> >>> deploying.
> >>>
> >>> Every detail of a running Kolla system can be easily deduced by
> scanning
> >>> across nodes and printing the labels of running containers,
> functionality
> >>> which can be shipped by Kolla. There are also methods for
> fetching labels of
> >>> remote images[0][1] for users wishing to inspect what they are
> upgrading to.
> >>>
> >>> [0] https://github.com/projectatomic/skopeo
> <https://github.com/projectatomic/skopeo>
> >>> [1] https://github.com/docker/distribution/issues/1252
> <https://github.com/docker/distribution/issues/1252>
> >>
> >>
> >>
> >> You beat me to it, Paul.
> >>
> >> I think using lables to communicate the version of each
> openstack software
> >> installed in the image is the way to go here. We're looking into
> doing this
> >> ourselves as part of the RDO pipeline and it'd be awesome to
> have it being
> >> part
> >> of kolla-build itself. Steve Baker, I believe, was working on this.
> >>
> >> The more explicit we are about the contents of the image, the
> better. People
> >> want to know what's in there, rather than assuming based on the tag.
> >>
> >> Flavio
> >>
> >>
> >>> -Paul
> >>>
> >>> On 18/04/17 22:10, Michał Jastrzębski wrote:
> >>>>
> >>>> On 18 April 2017 at 13:54, Doug Hellmann
> <doug at doughellmann.com <mailto:doug at doughellmann.com>> wrote:
> >>>>>
> >>>>> Excerpts from Michał Jastrzębski's message of 2017-04-18
> 13:37:30 -0700:
> >>>>>>
> >>>>>> On 18 April 2017 at 12:41, Doug Hellmann
> <doug at doughellmann.com <mailto:doug at doughellmann.com>> wrote:
> >>>>>>>
> >>>>>>> Excerpts from Steve Baker's message of 2017-04-18 10:46:43
> +1200:
> >>>>>>>>
> >>>>>>>> On Tue, Apr 18, 2017 at 9:57 AM, Doug Hellmann
> >>>>>>>> <doug at doughellmann.com <mailto:doug at doughellmann.com>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Excerpts from Michał Jastrzębski's message of 2017-04-12
> 15:59:34
> >>>>>>>>> -0700:
> >>>>>>>>>>
> >>>>>>>>>> My dear Kollegues,
> >>>>>>>>>>
> >>>>>>>>>> Today we had discussion about how to properly name/tag
> images being
> >>>>>>>>>> pushed to dockerhub. That moved towards general
> discussion on
> >>>>>>>>>> revision
> >>>>>>>>>> mgmt.
> >>>>>>>>>>
> >>>>>>>>>> Problem we're trying to solve is this:
> >>>>>>>>>> If you build/push images today, your tag is 4.0
> >>>>>>>>>> if you do it tomorrow, it's still 4.0, and will keep
> being 4.0
> >>>>>>>>>> until
> >>>>>>>>>> we tag new release.
> >>>>>>>>>>
> >>>>>>>>>> But image built today is not equal to image built
> tomorrow, so we
> >>>>>>>>>> would like something like 4.0.0-1, 4.0.0-2.
> >>>>>>>>>> While we can reasonably detect history of revisions in
> dockerhub,
> >>>>>>>>>> local env will be extremely hard to do.
> >>>>>>>>>>
> >>>>>>>>>> I'd like to ask you for opinions on desired behavior and
> how we
> >>>>>>>>>> want
> >>>>>>>>>> to deal with revision management in general.
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Michal
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> What's in the images, kolla? Other OpenStack components?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Yes, each image will typically contain all software
> required for one
> >>>>>>>> OpenStack service, including dependencies from OpenStack
> projects or
> >>>>>>>> the
> >>>>>>>> base OS. Installed via some combination of git, pip, rpm, deb.
> >>>>>>>>
> >>>>>>>>> Where does the
> >>>>>>>>> 4.0.0 come from?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> Its the python version string from the kolla project
> itself, so
> >>>>>>>> ultimately
> >>>>>>>> I think pbr. I'm suggesting that we switch to using the
> >>>>>>>> version.release_string[1] which will tag with the longer
> version we
> >>>>>>>> use for
> >>>>>>>> other dev packages.
> >>>>>>>>
> >>>>>>>>
> [1]https://review.openstack.org/#/c/448380/1/kolla/common/config.py
> <https://review.openstack.org/#/c/448380/1/kolla/common/config.py>
> >>>>>>>
> >>>>>>>
> >>>>>>> Why are you tagging the artifacts containing other projects
> with the
> >>>>>>> version number of kolla, instead of their own version
> numbers and some
> >>>>>>> sort of incremented build number?
> >>>>>>
> >>>>>>
> >>>>>> This is what we do in Kolla and I'd say logistics and
> simplicity of
> >>>>>> implementation. Tags are more than just information for us.
> We have to
> >>>>>
> >>>>>
> >>>>> But for a user consuming the image, they have no idea what
> version of
> >>>>> nova is in it because the version on the image is tied to a
> different
> >>>>> application entirely.
> >>>>
> >>>>
> >>>> That's easy enough to check tho (just docker exec into
> container and
> >>>> do pip freeze). On the other hand you'll have information that
> "this
> >>>> set of various versions was tested together" which is arguably
> more
> >>>> important.
> >>>>
> >>>>>> deploy these images and we have to know a tag. Combine that
> with clear
> >>>>>> separation of build phase from deployment phase (really
> build phase is
> >>>>>> entirely optional thanks to dockerhub), you'll end up with
> either
> >>>>>> automagical script that will have to somehow detect correct
> version
> >>>>>> mix of containers that works with each other, or hand
> crafted list
> >>>>>> that will have 100+ versions hardcoded.
> >>>>>>
> >>>>>> Incremental build is hard because builds are atomic and you
> never
> >>>>>> really know how many times images were rebuilt (also local
> rebuilt vs
> >>>>>> dockerhub-pushed rebuild will cause collisions in tags).
> >>>>>>
> >>>>>>> Doug
> >>>>>>>
> >>>>>>>
> >>>>>>>
> __________________________________________________________________________
> >>>>>>> OpenStack Development Mailing List (not for usage questions)
> >>>>>>> Unsubscribe:
> >>>>>>>
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> __________________________________________________________________________
> >>>>> OpenStack Development Mailing List (not for usage questions)
> >>>>> Unsubscribe:
> >>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>>>
> >>>>
> >>>>
> >>>>
> __________________________________________________________________________
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>>>
> >>>
> >>>
> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>
> >>
> >> --
> >> @flaper87
> >> Flavio Percoco
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list