<div dir="ltr"><div><font face="arial, sans-serif">Renat,</font></div><div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">Thanks for the detailed explanation. It is quite instructive and helps frame the question accurately by providing the necessary context.</font></div>
<ul style="font-family:arial,sans-serif;font-size:13px"><li style="margin-left:15px">Is the approach itself reasonable?<br></li></ul><div><font face="arial, sans-serif">[Manas] I think so. A repeatable workflow requires these sort of separation i.e. a model in which to save the specification as supplied by the user and a model in which to save the execution state - current and past.</font></div>
<ul style="font-family:arial,sans-serif;font-size:13px"><li style="margin-left:15px">Do we have better ideas on how to work with DSL? A good mental exercise here would be to imagine that we have more than one DSL, not only YAML but say XML. How would it change the picture?</li>
</ul><div><font face="arial, sans-serif">[Manas] As long as we form an abstraction between the DSL format i.e. YAML/XML and it consumption we will be able to move between various formats. (wishful) My personal preference is to not even have DSL show up anywhere in Mistral code apart from take it as input and transforming into this first level specification model - I know this is not the current state.</font></div>
<ul style="font-family:arial,sans-serif;font-size:13px"><li style="margin-left:15px">How can we clearly distinguish between these two models so that it wouldn’t be confusing?</li><li style="margin-left:15px">Do we have a better naming in mind?</li>
</ul><div><font face="arial, sans-serif">[Manas] I think we all would agree that the best approach is to have precise naming.</font></div><div><br></div><div>I see your point of de-normalizing the dsl data into respective db model objects.</div>
<div><br></div><div>In a previous email I suggested using *Spec. I will try to build on this -</div><div>1. Everything specified via the YAML input is a specification or definition or template. Therefore I suggest we suffix all these types with Spec/Definition/Template. So TaskSpec/TaskDefinition/TaskTemplate etc.. As per the latest change these are TaskData ... ActionData. </div>
<div>2. As per current impl the YAML is stored as a key-value in the DB. This is fine since that is front-ended by objects that Nikolay has introduced. e.g. TaskData, ActionData etc.</div><div>3. As per my thinking a model object that ends up in the DB or a model object that is in memory all can reside in the same module. I view persistence as an orthogonal concern so no real reason to distinguish the module names of the two set of models. If we do choose to distinguish as per latest change i.e. mistral/workbook that works too.</div>
<div><br></div><div>@Nikolay - I am generally ok with the approach. I hope that this helps clarify my thinking and perception. Please ask more questions.</div><div><br></div><div>Overall I like the approach of formalizing the 2 models. I am ok with current state of the review and have laid out my preferences.</div>
<div><br></div><div>Thanks,</div><div>Manas</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 5, 2014 at 3:39 AM, Nikolay Makhotkin <span dir="ltr"><<a href="mailto:nmakhotkin@mirantis.com" target="_blank">nmakhotkin@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">I think today and I have a good name for package (instead of 'mistral/model')<br>
<br>How do you think about to name it 'mistral/workbook'? I.e., It means that it contains modules for work with workbook representation - tasks, services, actions and workflow.<br>
<br>This way we able to get rid of any confusing.<br><div><br></div><div><br></div><div class="gmail_extra">Best Regards,<br>Nikolay<div><div class="h5"><br><br><div class="gmail_quote">On Wed, Mar 5, 2014 at 8:50 AM, Renat Akhmerov <span dir="ltr"><<a href="mailto:rakhmerov@mirantis.com" target="_blank">rakhmerov@mirantis.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word">I think we forgot to point to the commit itself. Here it is: <a href="https://review.openstack.org/#/c/77126/" target="_blank">https://review.openstack.org/#/c/77126/</a><div>
<br></div><div>Manas, can you please provide more details on your suggestion?</div><div><br></div><div>For now let me just describe the background of Nikolay’s question.</div><div><br></div><div>Basically, we are talking about how we are working with data inside Mistral. So far, for example, if a user sent a request to Mistral “start workflow” then Mistral would do the following:</div>
<div><ul><li>Get workbook DSL (YAML) from the DB (given that it’s been already persisted earlier).</li><li>Load it into a dictionary-like structure using standard ‘yaml’ library.</li><li>Based on this dictionary-like structure create all necessary DB objects to track the state of workflow execution objects and individual tasks.</li>
<li>Perform all the necessary logic in engine and so on. The important thing here is that DB objects contain corresponding DSL snippets as they are described in DSL (e.g. tasks have property “task_dsl”) to reduce the complexity of relational model that we have in DB. Otherwise it would be really complicated and most of the queries would contain lots of joins. The example of non-trivial relation in DSL is “task”->”action name”->”service”->”service actions”->”action”, as you can see it would be hard to navigate to action in the DB from a task if our relational model matches to what we have in DSL. this approach leads to the problem of addressing any dsl properties using hardcoded strings which are spread across the code and that brings lots of pain when doing refactoring, when trying to understand the structure of the model we describe in DSL, it doesn’t allow to do validation easily and so on.</li>
</ul><div><br></div></div><div>So what we have in DB we’ve called “model” so far and we’ve called just “dsl” the dictionary structure coming from DSL. So if we got a part of the structure related to a task we would call it “dsl_task”.</div>
<div><br></div><div>So what Nikolay is doing now is he’s reworking the approach how we work with DSL. Now we assume that after we parsed a workbook DSL we get some “model”. So that we don’t use “dsl” in the code anywhere this “model” describes basically the structure of what we have in DSL and that would allow to address the problems I mentioned above (hardcoded strings are replaced with access methods, we clearly see the structure of what we’re working with, we can easily validate it and so on). So when we need to access some DSL properties we would need to get workbook DSL from DB, build this model out of it and continue to work with it.</div>
<div><br></div><div>Long story short, this model parsed from DSL is not the model we store in DB but they’re both called “model” which may be confusing. For me this non-DB model more looks like “domain model” or something like this. So the question I would ask ourselves here:</div>
<div><ul><li>Is the approach itself reasonable?</li><li>Do we have better ideas on how to work with DSL? A good mental exercise here would be to imagine that we have more than one DSL, not only YAML but say XML. How would it change the picture?</li>
<li>How can we clearly distinguish between these two models so that it wouldn’t be confusing?</li><li>Do we have a better naming in mind?</li></ul><div><br></div></div><div>Thanks.</div><div><span><font color="#888888"><br>
<div>
<div>Renat Akhmerov</div><div>@ Mirantis Inc.</div><div><br></div><br>
</div></font></span><div><div>
<br><div><div>On 05 Mar 2014, at 08:56, Manas Kelshikar <<a href="mailto:manas@stackstorm.com" target="_blank">manas@stackstorm.com</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr">Since the renaming is for types in mistral.model.*. I am thinking we suffix with Spec e.g. <div>
<br></div><div>TaskObject -> TaskSpec</div><div>ActionObject -> ActionSpec and so on.</div><div><br></div>
<div>The "Spec" suggest that it is a specification of the final object that ends up in the DB and not the actual object. Multiple actual objects can be derived from these Spec objects which fits well with the current paradigm. Thoughts?</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Mar 3, 2014 at 9:43 PM, Manas Kelshikar <span dir="ltr"><<a href="mailto:manas@stackstorm.com" target="_blank">manas@stackstorm.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">Hi Nikolay - <div><br></div><div>Is your concern that mistral.db.sqlalchemy.models.* and mistral.model.* will lead to confusion or something else? </div>
<div><br></div><div>IMHO as per your change model seems like the appropriate usage while what is stored in the DB is also a model. If we pick appropriate names to distinguish between nature of the objects we should be able to avoid any confusion and whether or not model appears in the module name should not matter much.</div>
<div><br></div><div>Thanks,</div><div>Manas</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Mon, Mar 3, 2014 at 8:43 AM, Nikolay Makhotkin <span dir="ltr"><<a href="mailto:nmakhotkin@mirantis.com" target="_blank">nmakhotkin@mirantis.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><div dir="ltr">Hi, team! <br><br>Please look at the commit .<br>
<div><br></div><div>Module 'mistral/model' now is responsible for object model representation which is used for accessing properties of actions, tasks etc.<br clear="all">
<div><br></div><div>We have a name problem - looks like we should rename module 'mistral/model' since we have DB models and they are absolutely different.<br></div><div><br></div><div><br></div><div>Thoughts?</div>
<div><br></div> <br><div dir="ltr"><div><font>Best Regards,</font></div><div><font>Nikolay</font></div></div>
</div></div>
<br></div></div><div>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></div></blockquote></div><br></div>
</blockquote></div><br></div>
_______________________________________________<br>OpenStack-dev mailing list<br><a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div><br></div></div></div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br>
</div></div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>