Open Stack

Guys,

looks like you’ve started to talk about different things. As I see, original proposal was: stop treat MOS DEB repo as a special case and use the same flow for all repos. Your use case does not contradict it.

Moreover, it requires standard flow for all repos. ‘Put everything on the ISO’ use case should be implemented as a new feature. It is a matter of running fuel-createmirror script during ISO build and usage of it during master node deployment. It definitely should use mirror as a single object. And this object should be compatible with result of the fuel-createmirror script usage.

> On 10 Sep 2015, at 18:18, Vladimir Kuklin <vkuklin at mirantis.com> wrote:
> 
> Folks
> 
> I guess I need to get you on-site to deploy something at our user's datacenter. I do want to be able to download an ISO which contains all packages. This may not be the primary artifact of our software suite, but we need to have this opportunity to build full ISO with ALL components. Please, do not narrow down our feature set just by thinking that users do not need something because we are reluctant to implement this. Just believe me - users need this opportunity in a lot of deployment cases. It is not hard to implement it. We do not need to set this as a default option, but we need to have it. That is my point.
> 
> On Thu, Sep 10, 2015 at 5:40 PM, Vladimir Kozhukalov <vkozhukalov at mirantis.com <mailto:vkozhukalov at mirantis.com>> wrote:
> Vladimir,
> 
> * We don't have full ISO anyway
> * We don't require to create mirror. When you launch your browser, do you mean to have mirror of the Internet locally? Probably, no. The same is here. Internet connection is the common requirement nowadays, but if you don't have one, you definitely need to have a kind of local copy.
> 
> Vladimir Kozhukalov
> 
> On Thu, Sep 10, 2015 at 4:17 PM, Vladimir Kuklin <vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>> wrote:
> Igor
> 
> Having poor access to the internet is a regular use case which we must support. This is not a crazy requirement. Not having full ISO makes cloud setup harder to complete. Even more, not having hard requirement to create a mirror will detract newcomers. I can say that if I were a user and saw a requirement to create mirror I would not try the product with comparison to the case when I can get a full ISO with all the stuff I need.
> 
> On Thu, Sep 10, 2015 at 4:06 PM, Vladimir Kozhukalov <vkozhukalov at mirantis.com <mailto:vkozhukalov at mirantis.com>> wrote:
> Guys, 
> 
> I really appreciate your opinions on whether Fuel should be all inclusive or not. But the original topic of this thread is different. I personally think that in 2015 it is not a big deal to make the master node able to access any online host (even taking into account paranoid security policies). It is just a matter of network engineering. But it is completely out of the scope. What I am suggesting is to align the way how we treat different repos, whether upstream or MOS. What I am working on right now is I am trying to make Fuel build and delivery approach really flexible. That means we need to have as little non-standard ways/hacks/approaches/options as possible. 
> 
> > Why can't we make this optional in the build system? It should be easy to implement, is not it?
> 
> That is exactly what I am trying to do (make it optional). But I don't want it to be yet another boolean variable for this particular thing (MOS repo). We have working approach for dealing with repos. Repos can either online or local mirrors. We have a tool for making local mirrors (fuel-createmirror). Even if we put MOS on the ISO, a user still can not deploy OpenStack, because he/she still needs upstream to be available. Anyway, the user is still forced to do some additional actions. Again, we have plans to improve the quality and UX of fuel-createmirror. 
> 
> Yet another thing I don't want to be on the master node is a bunch of MOS repos directories named like 
> /var/www/nailgun/2015.1-7.0 
> /var/www/nailgun/2014.4-6.1 
> with links like
> /var/www/nailgun/ubuntu -> /var/www/nailgun/2015.1-7.0
> What does this link mean? Even Fuel developers can be confused. It is scary to imagine what users think of it :-) Why should Nailgun and upgrade script manage that kind of storage in this exact kind of format? A long time ago people invented RPM/DEB repositories, tools to manage them and structure for versioning them. We have Perestoika for that and we have plans to put all package/mirror related tools in one place (github.com/stackforge/fuel-mirror <http://github.com/stackforge/fuel-mirror>) and make all these tools available out of Fuel CI. So, users will be able to easily build their own packages, clone necessary repos and manage them in the way which is standard in the industry. However, it is out of the scope of the letter. 
> 
> I also don't like the idea of putting MOS repo on the ISO by default because it encourages people thing that ISO is the way of distributing MOS. ISO should be nothing more than just a way of installing Fuel from scratch. MOS should be distributed via MOS repos. Fuel is available as RPM package in RPM MOS repo.
> 
> 
> 
> 
> 
>  
> 
> Vladimir Kozhukalov
> 
> On Thu, Sep 10, 2015 at 1:18 PM, Igor Kalnitsky <ikalnitsky at mirantis.com <mailto:ikalnitsky at mirantis.com>> wrote:
> Mike,
> 
> > still not exactly true for some large enterprises. Due to all the security, etc.,
> > there are sometimes VPNs / proxies / firewalls with very low throughput.
> 
> It's their problem, and their policies. We can't and shouldn't handle
> all possible cases. If some enterprise has "no Internet" policy, I bet
> it won't be a problem for their IT guys to create an intranet mirror
> for MOS packages. Moreover, I also bet they do have a mirror for
> Ubuntu or other Linux distributive. So it basically about approach how
> to consume our mirrors.
> 
> On Thu, Sep 10, 2015 at 12:30 PM, Vladimir Kuklin <vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>> wrote:
> > Folks
> >
> > I think, Mike is completely right here - we need an option to build
> > all-in-one ISO which can be tried-out/deployed unattendedly without internet
> > access. Let's let a user make a choice what he wants, not push him into
> > embarassing situation. We still have many parts of Fuel which make choices
> > for user that cannot be overriden. Let's not pretend that we know more than
> > user does about his environment.
> >
> > On Thu, Sep 10, 2015 at 10:33 AM, Oleg Gelbukh <ogelbukh at mirantis.com <mailto:ogelbukh at mirantis.com>>
> > wrote:
> >>
> >> The reason people want offline deployment feature is not because of poor
> >> connection, but rather the enterprise intranets where getting subnet with
> >> external access sometimes is a real pain in various body parts.
> >>
> >> --
> >> Best regards,
> >> Oleg Gelbukh
> >>
> >> On Thu, Sep 10, 2015 at 8:52 AM, Igor Kalnitsky <ikalnitsky at mirantis.com <mailto:ikalnitsky at mirantis.com>>
> >> wrote:
> >>>
> >>> Hello,
> >>>
> >>> I agree with Vladimir - the idea of online repos is a right way to
> >>> move. In 2015 I believe we can ignore this "poor Internet connection"
> >>> reason, and simplify both Fuel and UX. Moreover, take a look at Linux
> >>> distributives - most of them fetch needed packages from the Internet
> >>> during installation, not from CD/DVD. The netboot installers are
> >>> popular, I can't even remember when was the last time I install my
> >>> Debian from the DVD-1 - I use netboot installer for years.
> >>>
> >>> Thanks,
> >>> Igor
> >>>
> >>>
> >>> On Thu, Sep 10, 2015 at 3:58 AM, Yaguang Tang <ytang at mirantis.com <mailto:ytang at mirantis.com>> wrote:
> >>> >
> >>> >
> >>> > On Thu, Sep 10, 2015 at 3:29 AM, Alex Schultz <aschultz at mirantis.com <mailto:aschultz at mirantis.com>>
> >>> > wrote:
> >>> >>
> >>> >>
> >>> >> Hey Vladimir,
> >>> >>
> >>> >>>
> >>> >>>
> >>> >>>>>
> >>> >>>>> 1) There won't be such things in like [1] and [2], thus less
> >>> >>>>> complicated flow, less errors, easier to maintain, easier to
> >>> >>>>> understand,
> >>> >>>>> easier to troubleshoot
> >>> >>>>> 2) If one wants to have local mirror, the flow is the same as in
> >>> >>>>> case
> >>> >>>>> of upstream repos (fuel-createmirror), which is clrear for a user
> >>> >>>>> to
> >>> >>>>> understand.
> >>> >>>>
> >>> >>>>
> >>> >>>> From the issues I've seen,  fuel-createmirror isn't very straight
> >>> >>>> forward and has some issues making it a bad UX.
> >>> >>>
> >>> >>>
> >>> >>> I'd say the whole approach of having such tool as fuel-createmirror
> >>> >>> is a
> >>> >>> way too naive. Reliable internet connection is totally up to network
> >>> >>> engineering rather than deployment. Even using proxy is much better
> >>> >>> that
> >>> >>> creating local mirror. But this discussion is totally out of the
> >>> >>> scope of
> >>> >>> this letter. Currently,  we have fuel-createmirror and it is pretty
> >>> >>> straightforward (installed as rpm, has just a couple of command line
> >>> >>> options). The quality of this script is also out of the scope of this
> >>> >>> thread. BTW we have plans to improve it.
> >>> >>
> >>> >>
> >>> >>
> >>> >> Fair enough, I just wanted to raise the UX issues around these types
> >>> >> of
> >>> >> things as they should go into the decision making process.
> >>> >>
> >>> >>
> >>> >>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> Many people still associate ISO with MOS, but it is not true when
> >>> >>>>> using
> >>> >>>>> package based delivery approach.
> >>> >>>>>
> >>> >>>>> It is easy to define necessary repos during deployment and thus it
> >>> >>>>> is
> >>> >>>>> easy to control what exactly is going to be installed on slave
> >>> >>>>> nodes.
> >>> >>>>>
> >>> >>>>> What do you guys think of it?
> >>> >>>>>
> >>> >>>>>
> >>> >>>>
> >>> >>>> Reliance on internet connectivity has been an issue since 6.1. For
> >>> >>>> many
> >>> >>>> large users, complete access to the internet is not available or not
> >>> >>>> desired.  If we want to continue down this path, we need to improve
> >>> >>>> the
> >>> >>>> tools to setup the local mirror and properly document what
> >>> >>>> urls/ports/etc
> >>> >>>> need to be available for the installation of openstack and any
> >>> >>>> mirror
> >>> >>>> creation process.  The ideal thing is to have an all-in-one CD
> >>> >>>> similar to a
> >>> >>>> live cd that allows a user to completely try out fuel wherever they
> >>> >>>> want
> >>> >>>> with out further requirements of internet access.  If we don't want
> >>> >>>> to
> >>> >>>> continue with that, we need to do a better job around providing the
> >>> >>>> tools
> >>> >>>> for a user to get up and running in a timely fashion.  Perhaps
> >>> >>>> providing an
> >>> >>>> net-only iso and an all-included iso would be a better solution so
> >>> >>>> people
> >>> >>>> will have their expectations properly set up front?
> >>> >>>
> >>> >>>
> >>> >>> Let me explain why I think having local MOS mirror by default is bad:
> >>> >>> 1) I don't see any reason why we should treat MOS  repo other way
> >>> >>> than
> >>> >>> all other online repos. A user sees on the settings tab the list of
> >>> >>> repos
> >>> >>> one of which is local by default while others are online. It can make
> >>> >>> user a
> >>> >>> little bit confused, can't it? A user can be also confused by the
> >>> >>> fact, that
> >>> >>> some of the repos can be cloned locally by fuel-createmirror while
> >>> >>> others
> >>> >>> can't. That is not straightforward, NOT fuel-createmirror UX.
> >>> >>
> >>> >>
> >>> >>
> >>> >> I agree. The process should be the same and it should be just another
> >>> >> repo. It doesn't mean we can't include a version on an ISO as part of
> >>> >> a
> >>> >> release.  Would it be better to provide the mirror on the ISO but not
> >>> >> have
> >>> >> it enabled by default for a release so that we can gather user
> >>> >> feedback on
> >>> >> this? This would include improved documentation and possibly allowing
> >>> >> a user
> >>> >> to choose their preference so we can collect metrics?
> >>> >>
> >>> >>
> >>> >>> 2) Having local MOS mirror by default makes things much more
> >>> >>> convoluted.
> >>> >>> We are forced to have several directories with predefined names and
> >>> >>> we are
> >>> >>> forced to manage these directories in nailgun, in upgrade script,
> >>> >>> etc. Why?
> >>> >>> 3) When putting MOS mirror on ISO, we make people think that ISO is
> >>> >>> equal
> >>> >>> to MOS, which is not true. It is possible to implement really
> >>> >>> flexible
> >>> >>> delivery scheme, but we need to think of these things as they are
> >>> >>> independent.
> >>> >>
> >>> >>
> >>> >>
> >>> >> I'm not sure what you mean by this. Including a point in time copy on
> >>> >> an
> >>> >> ISO as a release is a common method of distributing software. Is this
> >>> >> a
> >>> >> messaging thing that needs to be addressed? Perhaps I'm not familiar
> >>> >> with
> >>> >> people referring to the ISO as being MOS.
> >>> >>
> >>> >>
> >>> >>> For large users it is easy to build custom ISO and put there what
> >>> >>> they
> >>> >>> need but first we need to have simple working scheme clear for
> >>> >>> everyone. I
> >>> >>> think dealing with all repos the same way is what is gonna makes
> >>> >>> things
> >>> >>> simpler.
> >>> >>>
> >>> >>
> >>> >>
> >>> >> Who is going to build a custom ISO? How does one request that? What
> >>> >> resources are consumed by custom ISO creation process/request? Does
> >>> >> this
> >>> >> scale?
> >>> >>
> >>> >>
> >>> >>>
> >>> >>> This thread is not about internet connectivity, it is about aligning
> >>> >>> things.
> >>> >>>
> >>> >>
> >>> >> You are correct in that this thread is not explicitly about internet
> >>> >> connectivity, but they are related. Any changes to remove a local
> >>> >> repository
> >>> >> and only provide an internet based solution makes internet
> >>> >> connectivity
> >>> >> something that needs to be included in the discussion.  I just want to
> >>> >> make
> >>> >> sure that we properly evaluate this decision based on end user
> >>> >> feedback not
> >>> >> because we don't want to manage this from a developer standpoint.
> >>> >
> >>> >
> >>> >
> >>> >  +1, whatever the changes is, please keep Fuel as a tool that can
> >>> > deploy
> >>> > without Internet access, this is part of reason that people like it and
> >>> > it's
> >>> > better that other tools.
> >>> >>
> >>> >>
> >>> >> -Alex
> >>> >>
> >>> >>
> >>> >>
> >>> >> __________________________________________________________________________
> >>> >> OpenStack Development Mailing List (not for usage questions)
> >>> >> Unsubscribe:
> >>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Yaguang Tang
> >>> > Technical Support, Mirantis China
> >>> >
> >>> > Phone: +86 15210946968
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > __________________________________________________________________________
> >>> > OpenStack Development Mailing List (not for usage questions)
> >>> > Unsubscribe:
> >>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>> >
> >>>
> >>>
> >>> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe:
> >>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>
> >>
> >>
> >> __________________________________________________________________________
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >>
> >
> >
> >
> > --
> > Yours Faithfully,
> > Vladimir Kuklin,
> > Fuel Library Tech Lead,
> > Mirantis, Inc.
> > +7 (495) 640-49-04
> > +7 (926) 702-39-68
> > Skype kuklinvv
> > 35bk3, Vorontsovskaya Str.
> > Moscow, Russia,
> > www.mirantis.com <http://www.mirantis.com/>
> > www.mirantis.ru <http://www.mirantis.ru/>
> > vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> 
> -- 
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com <http://www.mirantis.ru/>
> www.mirantis.ru <http://www.mirantis.ru/>
> vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> 
> -- 
> Yours Faithfully,
> Vladimir Kuklin,
> Fuel Library Tech Lead,
> Mirantis, Inc.
> +7 (495) 640-49-04
> +7 (926) 702-39-68
> Skype kuklinvv
> 35bk3, Vorontsovskaya Str.
> Moscow, Russia,
> www.mirantis.com <http://www.mirantis.ru/>
> www.mirantis.ru <http://www.mirantis.ru/>
> vkuklin at mirantis.com <mailto:vkuklin at mirantis.com>__________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150910/d9901f7b/attachment.html>