[openstack-dev] [Neutron][IPAM] Arbitrary JSON blobs in ipam db tables
Joshua Harlow
harlowja at fastmail.com
Thu Nov 5 00:48:09 UTC 2015
Shraddha Pandhe wrote:
>
>
> On Wed, Nov 4, 2015 at 3:12 PM, Kevin Benton <blak111 at gmail.com
> <mailto:blak111 at gmail.com>> wrote:
>
> >If we add our own database for internal stuff, we go back to the
> same problem of allowing bad design.
>
> I'm not sure I understand what you are saying here. A JSON blob that
> only one driver knows how to interpret is worse than a vendor table.
>
> I am only talking about the IPAM tables here. The reference
> implementation doesn't need to play with JSON blob at all. Rather I
> would say, it shouldn't. It can be left up to the vendors/users to
> manage that blob responsibly. I can create my own database and point my
> IPAM module to that, but then IPAM tables are practically useless for
> me. The only reason for suggesting the blob is flexibility, which is the
> main reason for pluggability of IPAM.
>
> They both are specific to one driver but at least with a vendor
> table you can have DB migrations, integrity, column queries, etc.
> Additionally, the vendor table with extra features exposed via an
> API extension makes it more clear to the API caller what is vendor
> specific.
>
>
> I agree that thats a huge advantage of having a db. But sometimes, it
> may not be absolutely necessary to have an extra DB.
>
> e.g. For multiple gateways support, a separate database would probably
> add more overhead than required. All I want is to be able to fetch those
> IPs.
>
> The user can take a responsible decision whether to use the blob or the
> database depending on the requirement, if they have the flexibility.
>
> Can you elaborate what you mean by bad design?
>
> When we are working on internal features, we have to follow different
> timelines. Having an arbitrary blob can sometimes make us use that by
> default, especially under pressing deadlines, instead of consulting with
> broader audience and finding the right solution.
Just my 2 cents, and I know this since I'm on the team shraddha is on,
but the above isn't really that great of an excuse for having/adding
arbitrary blob(s); thinking long-term and figuring out what is really
required (and perhaps that ends up being a structured format vs a json
blob) is usually the better way of dealing with these types of issues
(knowingly fully well that it is not always possible).
Everyone in every company has different timelines and that (IMHO)
shouldn't be a 'way out' of consulting with a broader audience and
finding the right solution...
>
> On Nov 4, 2015 3:58 PM, "Shraddha Pandhe"
> <spandhe.openstack at gmail.com <mailto:spandhe.openstack at gmail.com>>
> wrote:
>
>
>
> On Wed, Nov 4, 2015 at 1:38 PM, Armando M. <armamig at gmail.com
> <mailto:armamig at gmail.com>> wrote:
>
>
>
> On 4 November 2015 at 13:21, Shraddha Pandhe
> <spandhe.openstack at gmail.com
> <mailto:spandhe.openstack at gmail.com>> wrote:
>
> Hi Salvatore,
>
> Thanks for the feedback. I agree with you that arbitrary
> JSON blobs will make IPAM much more powerful. Some other
> projects already do things like this.
>
> e.g. In Ironic, node has driver_info, which is JSON. it
> also has an 'extras' arbitrary JSON field. This allows
> us to put any information in there that we think is
> important for us.
>
>
> I personally feel that relying on json blobs is not only
> dangerously affecting portability, but it causes us to bloat
> the business logic, and forcing us to be doing less
> efficient when querying/filtering data
>
>
> Most importantly though, I feel it's like abdicating our
> responsibility to do a good design job.
>
> How does it affect portability?
>
> I don't think it forces us to do anything. 'Allows'? Maybe. But
> that can be solved. Before making any design decisions for
> internal feature-requests, we should first check with the
> community if its a wider use-case. If it is a wider use-case, we
> should collaborate and fix it upstream the right way.
>
> I feel that, its impossible for the community to know all the
> use-cases. Even if they knew, it would be impossible to
> incorporate all of them. I filed a bug few months ago about
> multiple gateway support for subnets.
>
> https://bugs.launchpad.net/neutron/+bug/1464361
> It was marked as 'Wont fix' because nobody else had this
> use-case. Adding and maintaining a patch to support this is
> super risky as it breaks the APIs. A JSON blob would have helped
> me here.
>
> I have another use-case. For multi-ip support for Ironic, we
> want to divide the IP allocation ranges into two: Static IPs and
> extra IPs. The static IPs are pre-configured IPs for Ironic
> inventory whereas extra IPs are the multi-ips. Nobody else in
> the community has this use-case.
>
> If we add our own database for internal stuff, we go back to the
> same problem of allowing bad design.
>
> Ultimately, we should be able to identify how to model these
> extensions you're thinking of both conceptually and logically.
>
>
> I would agree with that. If theres an effort going on in this
> direction, ill be happy to join. Without this, people like us
> with unique use-cases are stuck with having patches.
>
>
> I couldn't care less if other projects use it, but we ended
> up using in Neutron too, and since I lost this battle time
> and time again, all I am left with is this rant :)
>
>
>
> Hoping to get some positive feedback from API and DB
> lieutenants too.
>
>
> On Wed, Nov 4, 2015 at 1:06 PM, Salvatore Orlando
> <salv.orlando at gmail.com <mailto:salv.orlando at gmail.com>>
> wrote:
>
> Arbitrary blobs are a powerful tools to circumvent
> limitations of an API, as well as other constraints
> which might be imposed for versioning or portability
> purposes.
> The parameters that should end up in such blob are
> typically specific for the target IPAM driver (to an
> extent they might even identify a specific driver to
> use), and therefore an API consumer who knows what
> backend is performing IPAM can surely leverage it.
>
> Therefore this would make a lot of sense, assuming
> API portability and not leaking backend details are
> not a concern.
> The Neutron team API & DB lieutenants will be able
> to provide more input on this regard.
>
> In this case other approaches such as a vendor
> specific extension are not a solution - assuming
> your granularity level is the allocation pool;
> indeed allocation pools are not first-class neutron
> resources, and it is not therefore possible to have
> APIs which associate vendor specific properties to
> allocation pools.
>
> Salvatore
>
> On 4 November 2015 at 21:46, Shraddha Pandhe
> <spandhe.openstack at gmail.com
> <mailto:spandhe.openstack at gmail.com>> wrote:
>
> Hi folks,
>
> I have a small question/suggestion about IPAM.
>
> With IPAM, we are allowing users to have their
> own IPAM drivers so that they can manage IP
> allocation. The problem is, the new ipam tables
> in the database have the same columns as the old
> tables. So, as a user, if I want to have my own
> logic for ip allocation, I can't actually get
> any help from the database. Whereas, if we had
> an arbitrary json blob in the ipam tables, I
> could put any useful information/tags there,
> that can help me for allocation.
>
> Does this make sense?
>
> e.g. If I want to create multiple allocation
> pools in a subnet and use them for different
> purposes, I would need some sort of tag for each
> allocation pool for identification. Right now,
> there is no scope for doing something like that.
>
> Any thoughts? If there are any other way to
> solve the problem, please let me know
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for
> usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage
> questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list