Open Stack

>> I don't think there's anyone beating down the door needing this right
>> now, but it's one of those things that will take a little time to filter
>> in (since we have to shard-ify the existing data) and we can hardly even
>> propose the conversation until it's at least planned. Most of the other
>> services are going through a more painful "okay we need to shard our
>> data" phase right now, so the longer we wait to do this, the harder it
>> will be (and the more latency involved when we do). Easy for me to say,
>> I know.
>
> In a sense we're already sharded now, it's just that everyone is in
> the same default null shard.

Yep.

> So, for the sake of making it explicit, what's wrong with:
>
> * add a nullable shard column
> * enable the openstack-shard header I suggested at the last ptg [1]
> * let deployments start using that, but only if they want to. If
>   they do all rp writes and queries use it.

This sounds like a trap, so I'm curious... What, um, is left in such a
feature beyond this? :)

When we discussed this some number of Denvers ago, I think I said that I
would want everything to declare a shard identifier all the time, and
you (and I think Jay) wanted a "null is the default shard" type
behavior. So, what you say above seems to map to the latter, no?

I'm not crazy opposed to everyone being in the undeclared null shard
until they need to be in something else. I don't prefer it because:

 - I think it will be better tested (and test-able) if it's not optional
 - It's an identifier, and we'd never say "we don't need a non-null row
   id column until later when we need it"
 - I think that other services that may start reporting to or using
   placement may just omit that part in early development

However, whether the null default thing is transitional or lives
forever, doing what you said above is better than doing nothing, IMHO.

--Dan