[infra] Patch submitters can remove core reviewer votes?