Open Stack

Reviewed:  https://review.opendev.org/704508
Committed: https://git.openstack.org/cgit/openstack/oslo.cache/commit/?id=c31dd1aaac0a1dd8ca3f77b6da911ae85de6dc7a
Submitter: Zuul
Branch:    stable/queens

commit c31dd1aaac0a1dd8ca3f77b6da911ae85de6dc7a
Author: Morgan Fainberg <morgan.fainberg at gmail.com>
Date:   Fri Mar 22 12:35:16 2019 -0700

    Pass `flush_on_reconnect` to memcache pooled backend
    
    If a memcache server disappears and then reconnects when multiple memcache
    servers are used (specific to the python-memcached based backends) it is
    possible that the server will contain stale data. The default is now to
    supply the ``flush_on_reconnect`` optional argument to the backend. This
    means that when the service connects to a memcache server, it will flush
    all cached data in the server. The pooled backend is more likely to
    run into issues with this as it does not explicitly use a thread.local
    for the client. The non-pooled backend was not touched, it is not
    the recommended production use-case.
    
    See the help from python-memcached:
    
        @param flush_on_reconnect: optional flag which prevents a
    	scenario that can cause stale data to be read: If there's more
    	than one memcached server and the connection to one is
    	interrupted, keys that mapped to that server will get
    	reassigned to another. If the first server comes back, those
    	keys will map to it again. If it still has its data, get()s
    	can read stale data that was overwritten on another
    	server. This flag is off by default for backwards
    	compatibility.
    
    Change-Id: I3e335261f749ad065e8abe972f4ac476d334e6b3
    closes-bug: #1819957
    (cherry picked from commit 1192f185a5fd2fa6177655f157146488a3de81d1)


** Tags added: in-stable-queens

-- 
You received this bug notification because you are a member of OpenStack
Security SIG, which is subscribed to OpenStack.
https://bugs.launchpad.net/bugs/1819957

Title:
  Caching with stale data when a server disconnects due to network
  partition and reconnects

Status in OpenStack Identity (keystone):
  Invalid
Status in keystonemiddleware:
  Triaged
Status in oslo.cache:
  Fix Released
Status in OpenStack Security Advisory:
  Won't Fix

Bug description:
  The flush_on_reconnect optional flag is not used. This can cause stale
  data to be utilized from a cache server that disconnected due to a
  network partition. This has security concerns as follows:

  1*  Password changes/user changes may be reverted for the cache TTL
  1a* User may get locked out if PCI-DSS is on and the password change happens during the network
      partition.
  2*  Grant changes may be reverted for the cache TTL
  3*  Resources (all types) may become "undeleted" for the cache TTL
  4*  Tokens (KSM) may become valid again during the cache TTL

  
  As noted in the python-memcached library:

      @param flush_on_reconnect: optional flag which prevents a
              scenario that can cause stale data to be read: If there's more
              than one memcached server and the connection to one is
              interrupted, keys that mapped to that server will get
              reassigned to another. If the first server comes back, those
              keys will map to it again. If it still has its data, get()s
              can read stale data that was overwritten on another
              server. This flag is off by default for backwards
              compatibility.

  The solution is to explicitly pass flush_on_reconnect as an optional
  argument. A concern with this model is that the memcached servers may
  be utilized by other tooling and may lose cache state (in the case the
  oslo.cache connection is the only thing affected by the network
  partitioning).

  This similarly needs to be addressed in pymemcache when it is utilized
  in lieu of python-memcached.

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1819957/+subscriptions