Open Stack

On 06/24/2016 05:12 AM, Thierry Carrez wrote:
> Angus Lees wrote:
>> [...]
>> None of these are great, but:
>>
>> Possibility 1:  Backdoor rootwrap
>>
>> However if we assume rootwrap already exists then we _could_ rollout a
>> new version of oslo.rootwrap that contains a backdoor that allows
>> privsep-helper to be run as root for any context, without the need to
>> install a new rootwrap filter.
>>
>> Disclaimers:
>>
>> - It wouldn't work for virtualenvs, because the "privsep-helper"
>> executable won't be in sudo's usual PATH.
>>
>> - Retro-fitting something like that to rootwrap feels like it's skirting
>> close to some sort of ethical contract we've made with admins regarding
>> rootwrap's featureset.  Not saying we shouldn't do it, just that we
>> should think about how an operator is going to feel about that.
>>
>>
>> Possibility 2: Wider rootwrap filter
>>
>> In the past, I've been proposing rootwrap filters that match only
>> specific privsep "privileged contexts" by name.  On further reflection,
>> if we're assuming the existing python modules installed into root's
>> python path are already trustworthy (and we _are_ assuming that), then
>> it might also be reasonable to trust *any* privsep entrypoint declared
>> within that module path.  This gives a larger attack surface to think
>> about (particularly if python libraries including privsep decorators
>> were installed for some reason other than providing privsep entry
>> points), but there's no reason why this is _necessarily_ an issue.
>>
>> This allows us to get to a single rootwrap filter per-project (or
>> rather, "per-rootwrap") since projects use separate rootwrap config
>> directories - so we would still have to do a thing once per project.
>>
>>
>> Possibility 3: Skip rootwrap, use just sudo
>>
>> sudoers isn't very expressive - but we could install a new rootwrap-like
>> wrapper into sudoers once system-wide, which includes some sort of logic
>> to start privsep-helpers.  This could be as simple as a small shell
>> script.  The advantage this has over rootwrap is that it would contain
>> some sort of system-wide config, rather than per-project.
>>
>> Downsides
>>
>> - Would still need to be installed once system-wide.
>>
>> - Would need to be configured per-virtualenv, since otherwise we have no
>> way to know which virtualenvs should be given root powers.
>>
>>
>> Possibility 4: Run as root initially
>>
>> Another option would be to follow the usual Unix daemon model: Start the
>> process with all required privileges, and avoid sudo/rootwrap entirely.
>>
>> In this version, we take a once-off hit to tell everyone to start
>> running their OpenStack agents as root (probably from init/systemd), and
>> right at the top of main() we fork() the privsep-helper and then drop to
>> a regular uid.  No sudo or rootwrap ever (although the unprivileged code
>> could continue to use it while we clean up all the legacy code).
>>
>> A glorious future, but still a big per-project deployment change that
>> has to be managed somehow.
> 
> I'm adding Possibility (0): change Grenade so that rootwrap filters from
> N+1 are put in place before you upgrade.

If you do that as general course what you are saying is that every
installer and install process includes overwriting all of rootwrap
before every upgrade. Keep in mind we do upstream upgrade as offline,
which means that we've fully shut down the cloud. This would remove the
testing requirement that rootwrap configs were even compatible between N
and N+1. And you think this is theoretical, you should see the patches
I've gotten over the years to grenade because people didn't see an issue
with that at all. :)

I do get that people don't like the constraints we've self imposed, but
we've done that for very good reasons. The #1 complaint from operators,
for ever, has been the pain and danger of upgrading. That's why we are
still trademarking new Juno clouds. When you upgrade Apache, you don't
have to change your config files.

As an asside, one of our major inhibitors to smooth upgrade are these
giant conf trees which exist for users to modify in /etc, but not in a
really clear override way: policy, paste, rootwrap. Policy is being
solved right now by moving defaults to code (in Nova, but all the
infrastructure is in oslo.policy, and should be a quick transition for
other projects once we've proved it out) and keeping etc as only
overrides. Morgan and I talked about the same kind of approach with
paste, he's going to sniff that one out. Once we get to privsep, I think
we have it solved for rootwrap. But that transition is hard. Because the
existing system was designed without thinking about the upgrade
implications.

> No perfect answer here... I'm hesitating between (0), (1) and (4). (4)
> is IMHO the right solution, but it's a larger change for downstream. (1)
> is a bit of a hack, where we basically hardcode in rootwrap that it's
> being transitioned to privsep. That's fine, but only if we get rid of
> rootwrap soon. So only if we have a plan for (4) anyway. Option (0) is a
> bit of a hard sell for upgrade procedures -- if we need to take a hit in
> that area, let's do (4) directly...
> 
> In summary, I think the choice is between (1)+(4) and doing (4)
> directly. How doable is (4) in the timeframe we have ? Do we all agree
> that (4) is the endgame ?

4 seems like the right endgame to me. Honestly, with the shipped nova
compute policy we could probably rewrite the filesystem during nova
compute start and do this anyway


-- 
Sean Dague
http://dague.net