Open Stack

 ---- On Fri, 08 Mar 2019 17:13:19 -0600 Chris Hoge <chris at openstack.org> wrote ---- 
 > Speaking as just one member of the group, and not as one of the 
 > co-chairs, I’m ok with strict checking going in for a few reasons. 
 >  
 > * We're doing it for Nova, so a prescedent has been set. 
 > * Vendors aren't supposed to change the API code as all endpoints 
 >   designated sections in the interoperability guidelines. 
 > * This adds a layer of robustness to the API by forcing correct 
 >   responses.
 >  
 > As for our normal policy, if vendors object to the change we have release 
 > valves in place for them to flag tests and possibly have temporary 
 > policies put into place to give them an opportunity to migrate modified 
 > API code to upstream versions. 
 >  
 > The only thing I might advise is a temporary flag to disable checks so 
 > that we can give vendors an appropriate period of time to adjust to the 
 > new test requirements. I recall that the sudden switch of test behaviors 
 > caught vendors off-guard and was the primary source of problems 
 > surrounding the change. 

Actually, it is hard to provide the flag to disable the validation. From tempest point of view
we can hold of mergint the patches till stein release and once vendor are confirmed then,
we can add it. validation with disable option might cause different result on different env
which is main reason of doing steict validation. 

-gmann

 >  
 > -Chris 
 >  
 > > On Jan 21, 2019, at 2:53 AM, Ghanshyam Mann <gmann at ghanshyammann.com> wrote: 
 > >  
 > > ---- On Mon, 03 Dec 2018 10:58:51 +0900 Ghanshyam Mann <gmann at ghanshyammann.com> wrote ----  
 > >> ---- On Sat, 01 Dec 2018 02:58:45 +0900 Mark Voelker <mvoelker at vmware.com> wrote ----   
 > >>>  
 > >>>> On Nov 29, 2018, at 9:28 PM, Matt Riedemann <mriedemos at gmail.com> wrote:   
 > >>>>  
 > >>>> On 11/29/2018 10:17 AM, Ghanshyam Mann wrote:   
 > >>>>> - To improve the volume API testing to avoid the backward compatible changes. Sometime we accidentally change the API in backward incompatible way and strict validation with JSON schema help to block those.   
 > >>>>  
 > >>>> +1 this is very useful to avoid unintentionally breaking the API.   
 > >>>>  
 > >>>>> We want to hear from cinder and interop team about any impact of this change to them.   
 > >>>>  
 > >>>> I'm mostly interested in what the interop WG would do about this given it's a potentially breaking change for interop without changes to the guidelines. Would there be some sort of grace period for clouds to conform to the changes in tempest?   
 > >>>>  
 > >>>  
 > >>> That’s more or less what eventually happened when we began enforcing strict validation on Nova a few years ago after considerable debate.  Clouds that were compliant with the interop guidelines before the strict validation patch landed and started failing once it went in could apply for a waiver while they worked on removing or upstreaming the nonstandard stuff. For those not familiar, here’s the patch that created a waiver program:   
 > >>>  
 > >>> https://review.openstack.org/#/c/333067/   
 > >>>  
 > >>> Note that this expired with the 2017.01 Guideline:   
 > >>>  
 > >>> https://review.openstack.org/#/c/512447/   
 > >>>  
 > >>> While not everyone was totally happy with the solution, it seemed to work out as a middle ground solution that helped get clouds on a better path in the end.  I think we’ll discuss whether we’d need to do something like this again here.  I’d love to hear:   
 > >>>  
 > >>> 1.)  If anyone knows of clouds/products that would be fail interop testing because of this.  Not looking to name and shame, just to get an idea of whether or not we have a concrete problem and how big it is.   
 > >>>  
 > >>> 2.)  Opinions on how the waiver program went last time and whether the rest of the community feels like it’s something we should consider again.   
 > >>>  
 > >>> Personally I’m supportive of the general notion of improving API interoperability here…as usual it’s figuring out the mechanics of the transition that take a little figuring. =)   
 > >>  
 > >> Thanks Mark for response. I think point 1 is important, it is good to  get the list of clouds or failure due to this this strict validation change.  And accordingly, we can wait on Tempest side to merge those changes for this cycle (but personally I do not want to delay that if everything is fine),  so that we can avoid the immediate failure of interop program.   
 > >  
 > > Any update/feedback from interop/cloud provider side on strict API validation ?  We are holding the Tempest patches to merge and waiting to hear form interop group.   
 > >  
 > >  
 > > -gmann 
 > >  
 > >>  
 > >> -gmann  
 > >>  
 > >>>  
 > >>> At Your Service,   
 > >>>  
 > >>> Mark T. Voelker   
 > >>>  
 > >>>  
 > >>>> --    
 > >>>>  
 > >>>> Thanks,   
 > >>>>  
 > >>>> Matt   
 > >>>>  
 > >>>> _______________________________________________   
 > >>>> Interop-wg mailing list   
 > >>>> Interop-wg at lists.openstack.org   
 > >>>> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Finterop-wg&data=02%7C01%7Cmvoelker%40vmware.com%7C82a07fe28afe488c2eea08d6566b9734%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636791417437738014&sdata=lEx%2BbbTVzC%2FRC7ebmARDrFhfMsToM7Rwx8EKYtE7iFM%3D&reserved=0   
 >  
 >  
 >