On Sun, 10 Feb 2019, Chris Dent wrote:
It's not clear at this point whether these sorts of things should be documented in projects, or somewhere more central. So perhaps we can just talk about it here in email and figure something out. I'll followup with some I have for placement, since that's the project I've given the most attention.
Conversation on vision reflection for placement [1] is what reminded me that this part 2 is something we should be doing. I should disclaim that I'm the author of a lot of the architecture of placement so I'm hugely biased. Please call me out where my preferences are clouding reality. Other contributors to placement probably have other ideas. They would be great to hear. However, it's been at least two years since we started, so I think we can extract some useful lessons. Things have have worked out well (you can probably see a theme): * Placement is a single purpose service with, until very recently, only the WSGI service as the sole moving part. There are now placement-manage and placement-status commands, but they are rarely used (thankfully). This makes the system easier to reason about than something with multiple agents. Obviously some things need lots of agents. Placement isn't one of them. * Using gabbi [2] as the framework for functional tests of the API and using them to enable test-driven-development, via those functional tests, has worked out really well. It keeps the focus on that sole moving part: The API. * No RPC, no messaging, no notifications. * Very little configuration, reasonable defaults to that config. It's possible to run a working placement service with two config settings, if you are not using keystone. Keystone adds a few more, but not that much. * String adherence to WSGI norms (that is, any WSGI server can run a placement WSGI app) and avoidance of eventlet, but see below. The combination of this with small number of moving parts and little configuration make it super easy to deploy placement [3] in lots of different setups, from tiny to huge, scaling and robustifying those setups as required. * Declarative URL routing. There's a dict which maps HTTP method:URL pairs to python functions. Clear dispatch is a _huge_ help when debugging. Look one place, as a computer or human, to find where to go. * microversion-parse [4] has made microversion handling easy. Things that haven't gone so well (none of these are dire) and would have been nice to do differently had we but known: * Because of a combination of "we might need it later", "it's a handy tool and constraint" and "that's the way we do things" the interface between the placement URL handlers and the database is mediated through oslo versioned objects. Since there's no RPC, nor inter-version interaction, this is overkill. It also turns out that OVO getters and setters are a moderate factor in performance. Initially we were versioning the versioned objects, which created a lot of cognitive overhead when evolving the system, but we no longer do that, now that we've declared RPC isn't going to happen. * Despite the strict adherence to being a good WSGI citizen mentioned above, placement is using a custom (very limited) framework for the WSGI application. An initial proof of concept used flask but it was decided that introducing flask into the nova development environment would be introducing another thing to know when decoding nova. I suspect the expected outcome was that placement would reuse nova's framework, but the truth is I simply couldn't do it. Declarative URL dispatch was a critical feature that has proven worth it. The resulting code is relatively straightforward but it is unicorn where a boring pony would have been the right thing. Boring ponies are very often the right thing. I'm sure there are more here, but I've run out of brain. [1] https://review.openstack.org/#/c/630216/ [2] https://gabbi.readthedocs.io/ [3] https://anticdent.org/placement-from-pypi.html [4] https://pypi.org/project/microversion_parse/ -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent