Open Stack

This thread is not about format itself, but about the approach when all
repos are thought independently. I.e. no patterns like this suite,
suite-updates, suite-security, no limitations for suite and suite-updates
should be located on the same host. It should be flat list of independent
repos. There is an opinion that I am opposing to that when defining repos a
user should define only base url and base suite:

http://archive.ubuntu.com/ubuntu
trusty

and then we are supposed to derive a set of repos like this:

http://archive.ubuntu.com/ubuntu/dists/trusty main universe
multiverse restricted
http://archive.ubuntu.com/ubuntu/dists/trusty-updates main universe
multiverse restricted
http://archive.ubuntu.com/ubuntu/dists/trusty-security main universe
multiverse restricted

That is exactly what we have in fuel-menu at the moment. That is wrong
(IMO). Why I think it is wrong to hard code "-updates -security" suffixes
and "main universe multiverse restricted" sections is described in the very
first letter of the ML thread. Again, repos should be independent. There
should not be any assumptions about -updates or something like this. That
is the main topic of the thread.

Regarding to format of the repo data I think the format that is now used by
nailgun [0] (list of dicts) is very convenient and easily extendable. And
we should use this format wherever we need to serialize/de-serialize
repository data (you can think of this format as a part of Fuel internal
communication protocol). We'd better avoid using native formats like apt
sources or yum.conf for passing data between Fuel components because it
requires having complicated (and thus fragile) code to serialize and
de-serialize these data. Native formats should be used only when yum or apt
should be directly configured.

[0]
https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/fixtures/openstack.yaml#L2006-L2053


Vladimir Kozhukalov

On Fri, Dec 11, 2015 at 1:56 PM, Aleksandr Mogylchenko <
amogylchenko at mirantis.com> wrote:

> There are common practices developed by operating system developers, and
> thousands of people using them every day.
>
> Redefining that experience will bring nothing more but questions and
> misunderstanding, because if someone has questions, instead of hundreds
> already written manuals a person would be forced to go to us.
>
> There is nothing wrong in storing it *internally* in whatever format is
> suitable for Fuel. But UI, in my opinion, should follow practices that
> already exist, not define something new.
>
> And, btw, URIs are not repositories, but just one component:
> https://wiki.debian.org/RepositoryFormat
>
> --
> Aleksandr Mogylchenko
>
>
> > On Dec 11, 2015, at 12:29, Igor Kalnitsky <ikalnitsky at mirantis.com>
> wrote:
> >
> > Hey folks -
> >
> > +1 from my side on the idea of having the unified repo format. It will
> > simplify a cross-project contribution. I think we can file a blueprint
> > for 9.0.
> >
> > I have only two questions to discuss:
> >
> > * We need to declare format for RPM repos either.
> > * Shouldn't we use slightly different set of fields for flat Debian
> repos?
> >
> > - Igor
> >
> > On Fri, Dec 11, 2015 at 11:28 AM, Fedor Zhadaev <fzhadaev at mirantis.com>
> wrote:
> >> Hello Vladimir,
> >>
> >> I definitely agree that using one uri for generating number of repos is
> not
> >> the best solution.
> >> According to Fuel Menu - there was the discussion in gerrit [1] about
> >> repositories format. The first version of my patch implements actually
> your
> >> idea about equality and independence of repositories. It meets
> disagreements
> >> and no one voted for this POV. So I had to change the implementation in
> >> favor to the current version.
> >>
> >> According to this situation I would like to discuss if proposed changes
> are
> >> needed before making new patch. Guys, who took part in the previous
> patch
> >> discussion, please share your opinions.
> >>
> >> [1] https://review.openstack.org/#/c/242646/
> >>
> >> 2015-12-10 22:47 GMT+03:00 Vladimir Kozhukalov <
> vkozhukalov at mirantis.com>:
> >>>
> >>> Dear colleagues,
> >>>
> >>> At the moment we have several places where we configure multiple
> rpm/deb
> >>> repositories. Those are:
> >>>
> >>> Web UI (cluster settings tab) where we define repos for cluster
> deployment
> >>> Fuel-menu (bootstrap section) where we define repos for building ubuntu
> >>> bootstrap image
> >>> Fuel-mirror where we define repos that are to be cloned (full or
> partial
> >>> mirrors)
> >>>
> >>> I'd prefer all these places to provide the same UX. By that I mean that
> >>> these components should use the same input data structure like this
> [0],
> >>> i.e. a flat list of fully independent repositories (see an example
> below).
> >>> First repo in the list is supposed to be a base OS repo (i.e. contains
> base
> >>> packages like libc).
> >>>
> >>> [
> >>> {
> >>>    type: deb,
> >>>    url: some-url,
> >>>    section: some-section,
> >>>    suite: some-suite,
> >>>    priority: some-priority
> >>>  },
> >>>  {
> >>>    type: deb,
> >>>    url: another-url,
> >>>    section: another-section,
> >>>    suite: another-suite,
> >>>    priority: another-priority
> >>>  },
> >>> ...
> >>> ]
> >>>
> >>> I'd like to focus on the fact that these repositories should be defined
> >>> independently (no base url, no base suite, etc.) That makes little
> sense to
> >>> speculate about consistency of a particular repository. We only should
> talk
> >>> about consistency of the whole list of repositories together.
> >>>
> >>> I'll try to explain. In the real world we usually deal with sets of
> >>> repositories which look like this:
> >>>
> >>> http://archive.ubuntu.com/ubuntu/dists/trusty/
> >>> http://archive.ubuntu.com/ubuntu/dists/trusty-updates/
> >>> http://archive.ubuntu.com/ubuntu/dists/trusty-security/
> >>> http://mirror.fuel-infra.org/mos-repos/ubuntu/8.0/dists/mos8.0/
> >>>
> http://mirror.fuel-infra.org/mos-repos/ubuntu/8.0/dists/mos8.0-updates/
> >>>
> http://mirror.fuel-infra.org/mos-repos/ubuntu/8.0/dists/mos8.0-security/
> >>>
> >>> As you can see these repositories have common hosts and base suites
> and it
> >>> instills us to think that repositories should not be defined separately
> >>> which is wrong. This special case does not break the whole approach.
> It is
> >>> just a special case. Repositories are nothing more than just sets of
> >>> packages that can depend on each other forming a tree when taken
> together.
> >>> Package relation does matter, not repository location, not suite name.
> >>> Parsing package tree for a set of repositories we can easily figure out
> >>> whether this set is consistent or not (e.g. python-packetary allows to
> do
> >>> this).
> >>>
> >>> Taking into account the above, I'd say UI should allow a user to define
> >>> repositories independently not forcing to use special patterns like
> suite +
> >>> suite-updates + suite-security, not forcing repositories to be located
> on
> >>> the same host. That means we should modify fuel-menu bootstrap section
> which
> >>> currently allows a user to define a base url that is then used to form
> a
> >>> group of repos (base, base-updates, base-security). Besides, it
> contradicts
> >>> to our use case when we put mosX.Y locally on the master node while
> >>> mosX.Y-updates and mosX.Y-security are supposed to be available online.
> >>>
> >>> What do you guys think of that?
> >>>
> >>>
> >>> [0]
> >>>
> https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/fixtures/openstack.yaml#L2006-L2053
> >>>
> >>>
> >>> Vladimir Kozhukalov
> >>>
> >>>
> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>
> >>
> >>
> >> --
> >> Kind Regards,
> >> Fedor Zhadaev
> >>
> >> Skype: zhadaevfm
> >> IRC: fzhadaev
> >>
> >>
> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151211/4d734385/attachment.html>