Open Stack

Thanks for the comments Mandeep!  Responses in-line #PCM

On May 23, 2014, at 8:57 PM, Mandeep Dhami <dhami at noironetworks.com> wrote:

> My preferences:
> 
> For where, I'd go with Gary's recommendation (A) for two reasons (1) Consistency and (2) I don't think it will create any boilerplate requirements since the abstract class provides the default implementation.

#PCM Actually, (A) requires a lot of additional code that doesn’t currently exist today. ABC methods would be needed for each validation. In addition, to be consistent, there should be an abstract method for the apply function, and a “pass” implementation on each of the concrete classes. Overall, it is adding a lot of code, for arguably little benefit.


> 
> For naming, I'd prefer to go with ML2 terminology for two reasons (1) Again Consistency, and (2) it is then clear what actions are happening within a transaction or outside of it. With a "validation function" no "transaction fence" is implied by it's name - but for any validation that depends on what currently exists in the database, these transaction semantics are important.

#PCM I’m still struggling with the naming for question #2.  I must admit, I like the naming of validate/apply because it makes it clear as to what is happening with the calls. With pre-commit/post-commit, it doesn’t indicate what the functions are actually doing, and in one case (for the Cisco VPN service driver) the post-commit will actually be doing a database commit in addition to applying the changes to the device driver (so both the postcommt and apply naming don’t clearly indicate the actions - an alternative would be to call the driver for the commit phase too, but this is a rare case).

To muddy it up a bit more, the main functions have no indication in the name that they deal with persisting to the database. The hierarchy currently is (using create_vpnservice...

class VPNPluginBase(service_base.ServicePluginBase):   # ABC with Northbound API definition
    @abc.abstractmethod
    def create_vpnservice(self, context, vpnservice):
        pass

class VPNPluginDb(vpnaas.VPNPluginBase, base_db.CommonDbMixin):  # DBase concrete class
    def create_vpnservice(self, context, vpnservice):
        vpns = vpnservice['vpnservice']
        tenant_id = self._get_tenant_id_for_create(context, vpns)
        with context.session.begin(subtransactions=True):
            …

class VPNPlugin(vpn_db.VPNPluginDb):
    …
class VPNDriverPlugin(VPNPlugin, vpn_db.VPNPluginRpcDbMixin):  # Plugin child class
    def create_vpnservice(self, context, vpnservice):
        driver = self._get_driver_for_vpnservice(vpnservice)
        super(VPNDriverPlugin, self).create_vpnservice(context, vpnservice)
        driver.create_vpnservice(context, vpnservice)

class VpnDriver(object):  # ABC for service driver
    # Nothing for create_vpnservice()

class IPsecVPNDriver(service_drivers.VpnDriver):  # Service driver class
    # Nothing for create_vpnservice()


In the proposals, we talking about a sequence of this (2X) at the plugin:

    def create_vpnservice(self, context, vpnservice):
        driver = self._get_driver_for_vpnservice(vpnservice)
        driver.create_vpnservice_precommit(vpnservice)
        super(VPNDriverPlugin, self).create_vpnservice(context, vpnservice)
        driver.create_vpnservice_postcommit(context, vpnservice)

versus this (2Y)…

    def create_vpnservice(self, context, vpnservice):
        driver = self._get_driver_for_vpnservice(vpnservice)
        driver.validate_create_vpnservice(vpnservice)
        super(VPNDriverPlugin, self).create_vpnservice(context, vpnservice)
        driver.apply_create_vpnservice(context, vpnservice)


The former makes is clearer that there is a database operation in-between (implied by the naming). The latter makes it clearer what the service driver is doing before and after.  In both cases, the ABC for the service driver (VpnDriver) could have the default validation/precommit actions, and the service driver (IPsecVPNDriver) could optionally have any provider validation/precommit and would have the mandatory apply/post-commit actions.

Maybe I could do a WIP patch out for review to give something for concrete commenting…


Regards,

PCM



> 
> Regards,
> Mandeep
> 
> 
> 
> On Fri, May 23, 2014 at 4:25 PM, Paul Michali (pcm) <pcm at cisco.com> wrote:
> Thanks for the comment Carl. See @PCM inline
> 
> 
> PCM (Paul Michali)
> 
> MAIL …..…. pcm at cisco.com
> IRC ……..… pcm_ (irc.freenode.com)
> TW ………... @pmichali
> GPG Key … 4525ECC253E31A83
> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
> 
> 
> 
> On May 23, 2014, at 6:09 PM, Carl Baldwin <carl at ecbaldwin.net> wrote:
> 
>> Paul,
>> 
>> On Fri, May 23, 2014 at 8:24 AM, Paul Michali (pcm) <pcm at cisco.com> wrote:
>>> Hi,
>>> 
>>> I’m working on a task for a BP to separate validation from persistence logic
>>> in L3 services code (VPN currently), so that providers can override/extend
>>> the validation logic (before persistence).
>>> 
>>> So I’ve separated the code for one of the create APIs, placed the default
>>> validation into an ABC class (as a non-abstract method) that the service
>>> drivers inherit from, and modified the plugin to invoke the validation
>>> function in the service driver, before doing the persistence step.
>>> 
>>> The flow goes like this…
>>> 
>>>    def create_vpnservice(self, context, vpnservice):
>>>        driver = self._get_driver_for_vpnservice(vpnservice)
>>>        driver.validate_create_vpnservice(context, vpnservice)
>>>        super(VPNDriverPlugin, self).create_vpnservice(context, vpnservice)
>>>        driver.apply_create_vpnservice(context, vpnservice)
>>> 
>>> If the service driver has a validation routine, it’ll be invoked, otherwise,
>>> the default method in the ABC for the service driver will be called and will
>>> handle the “baseline” validation. I also renamed the service driver method
>>> that is used for applying the changes to the device driver as apply_*
>>> instead of using the same name as is used for persistence (e.g.
>>> create_vpnservice -> apply_create_vpnservice).
>>> 
>>> The questions I have is…
>>> 
>>> 1) Should I create new validation methods A) for every create (and update?)
>>> API (regardless of whether they currently have any validation logic, B) for
>>> resources that have some validation logic already, or C) only for resources
>>> where there are providers with different validation needs?  I was thinking
>>> (B), but would like to hear peoples’ thoughts.
>> 
>> I think B.  C may leave a little too much inconsistency.  A feels like
>> extra boiler-plate.  Would there be any benefit to creating a higher
>> level abstraction for the create/update API calls?  I'm not suggesting
>> you do so but if you did then you could add a validation method to
>> that interface with a default pass.  Otherwise, I'd stick with B until
>> there is a need for more.
> 
> @PCM Yeah, there is somewhat of an abstraction. In VPN, for the service drivers, they all inherit from an ABC. For the apply, there are already abstract methods in the ABC, so that the service driver is forced to provide a function (of course it is inconsistent currently - they are there for VPN services, but not for IPSec connections, which are provided in the service driver). For the validation, I was creating methods in the ABC, with implementations, but not abstract, so that the service driver could provide a function, if it wanted to extend or override (and call super, if desired), or leave it out and the base class function would be used.
> 
> So, I guess it’s at two points, the validate and apply methods, where we should probably be consistent with providing base class methods for all that present in the service driver.
> 
> 
>> 
>>> 2) I’ve added validation_* and modified the other service driver call to
>>> apply_*. Should I instead, use the ML2 terminology of pre commit_* and post
>>> commit_*? I personally favor the former, as it is more descriptive of what
>>> is happening in the methods, but I understand the desire for consistency
>>> with other code.
>> 
>> I'm on the fence.  ML2 is not where I'm most familiar and I don't know
>> the history behind that design.  Without considering ML2 and
>> consistency, I think I like your terminology better.
> 
> @PCM I’m in the same camp. I don’t know ML2 well, and I like the explicit terminology. It *seems* like ML2 is dealing with this all at the same level of abstraction (pre-commit, commit, post-commit all for the same plugin or driver), whereas with L3 services, it is at two levels (validate at service driver and then plugin, commit at plugin, apply at service driver only). I guess I can leave it and see what happens at review…I have some asbestos here somewhere.
> 
> 
>> 
>>> 3) Should I create validation methods for code, where defaults are being set
>>> for missing (optional) information? For example, VPN IKE Policy lifetime
>>> being set to units=seconds, value=3600, if not set. Currently, provider
>>> implementations have same defaults, but could potentially use different
>>> defaults. The alternative is to leave this in the persistence code and not
>>> allow it to be changed. This could be deferred, if 1C is chosen above.
>> 
>> I'm tempted to say punt on this until there is a need for it.
> 
> @PCM I think you’re right. It may turn out to be YAGNI.
> 
> Thanks!
> 
> PCM
> 
> 
>> 
>> Carl
>> 
>>> Looking forward to your thoughts...
>>> 
>>> 
>>> Thanks!
>>> 
>>> PCM (Paul Michali)
>>> 
>>> MAIL …..…. pcm at cisco.com
>>> IRC ……..… pcm_ (irc.freenode.com)
>>> TW ………... @pmichali
>>> GPG Key … 4525ECC253E31A83
>>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140527/ebdbc51a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140527/ebdbc51a/attachment.pgp>