Open Stack

We held the inaugural Heat mid-cycle meetup in Raleigh, North Carolina 
this week. There were a dozen folks in attendance, and I think everyone 
agreed that it was a very successful event. Notes from the meetup are on 
the Etherpad here:

https://etherpad.openstack.org/p/heat-juno-midcycle-meetup

Here are a few of the conclusions:

* Everyone wishes the Design Summit worked like this.
The meetup seemed a lot more productive than the design summit ever is. 
It's really nice to be in a room small enough that you can talk normally 
and hear everyone, instead of in a room designed for 150 people. It's 
really nice to be able to discuss stuff that isn't related to a 
particular feature - we had a long discussion about how to get through 
the review backlog, for example. It's really nice to not have fixed time 
slots for discussions - because everyone was in the room the whole time, 
we could dip in and out of different topics at will. Often we came back 
to one that we'd previously discussed because we had discovered new 
information. Finally, it's critical to be in a room covered in 
full-sized whiteboards that everyone can see. A single tiny flip chart 
doesn't cut it.

* 3 days seems to be about the right length.
Not a lot got done on day 3, and people started to drift out at various 
times to catch flights, but the fact that everyone was there for _all_ 
of day 1 and 2 was essential (the critical Convergence plan was 
finalised around 7.30pm on Tuesday).

* There was a lot more discussion than hacking.
The main value of the meet-up was more in the discussions you'd hope to 
be able to have at the design summit than in working collaboratively on 
code.

* Marconi is now called Zaqar.
Who knew?

* Marc^W Zaqar is critical to pretty much every major non-Convergence 
feature on the roadmap.
We knew that we wanted to use it for notifications, but we also want to 
make those a replacement for events, and a conduit for warnings and 
debugging information to the user. This is becoming so important that 
we're going to push ahead with an implementation now without waiting to 
see when Zaqar will graduate. Zaqar would also be a good candidate for 
pushing metadata changes to servers, to resolve the performance issues 
currently caused by polling.

* We are on track to meet the immediate requirements of TripleO.
Obviously it would be nice to have Convergence now, but failing that the 
most critical bugs are under control. In the immediate future, we need 
to work on finding a consensus on running with multiple worker by 
default, split trees of nested stacks so that each nested stack runs in 
a separate engine, and find a way to push metadata out from Heat instead 
of having servers poll us for it.

* We have a plan for what Convergence will look like.
Here's some horrific photos of a whiteboard: 
https://www.dropbox.com/sh/tamoc8dhhckb81w/AAA6xp2be9xv20P7SWx-xnZba?dl=0
Clint is, I believe, working on turning that into something more 
consumable. This was definitely the biggest success of the meet-up. 
Before this I had no idea what convergence would look like; now I have a 
fair idea how it will work and where the tricky bits might be. I doubt 
this could have happened at a design summit.

* We probably don't need TaskFlow.
After coming up with the Convergence plan we realised that, while 
TaskFlow would be useful/essential if we were planning a more modest 
refactoring of Heat, the architecture of Convergence should actually 
eliminate the need for it. All we think we need is a bunch of work 
queues that can be provided by oslo.messaging. TaskFlow seems great for 
the problem it solves, but we have the opportunity to not create that 
problem for ourselves in the first place.

* Convergence probably won't buy us much in the short term.
I think we all hoped that our incremental work on Convergence would 
render incremental benefits for Heat. After figuring out the rough 
outline of how Convergence could work, we realised that the incremental 
steps along the way (like implementing the observer process) will 
actually not have a big impact. So while, of course, we'll continue to 
work incrementally, we don't expect to see major benefits until nearer 
the end of the process.


Thanks to everyone who made the trip, and of course also to everyone who 
contributed input via IRC and generally held down the fort while we were 
meeting. If I misstated or just plain missed anything above, please feel 
free to weigh in.

cheers,
Zane.