Open Stack

Oleg Gelbukh [mailto:ogelbukh at mirantis.com] wrote:


Motonobu-san,

The concept of 'distance' between regions used to determine where to go for replica if 'local' one is missing seems interesting, but definitely needs some additional thinking on our side. Our design does not assume that reigon ID is meaningful number, however, that 'distance' semantics can be added easily enough.

However, before I think about the semantic of distance, it's occured to me that we have variants of how regions can be connected. The dedicated replication network feature adds variety that I'll try to describe.

> Basically we have 3 networks in Swift cluster: 'public' network, connecting proxy servers to the world (or to the load-balancer
> or firewall, which is more likely), 'storage' network, connecting proxy servers to storage servers, and 'management' network.
> 'Storage' network usually use private IP addresses. Obviously, we need to connect storage networks of 2 regions (clusters)
> if we want proxy-servers to be able to read objects from foreign regions. It seems to be a bad idea to use globally routed IP
> range for 'storage' network, so we'll likely need a VPN between two 'storage' networks in two clusters.

> With dedicated 'replication' network, we only need connection between storage servers on the 'replication' network to get
> the proxy-server behavior you propose (write locally and let replication move objects to foreign region destinations).
> However, if we want proxies to read from foreign regions, we still need to connect 'storage' networks.

> So, we will have 2 'levels' of regions proximity which affect proxy-server behavior:
> * connected 'replication' networks -- in this case proxy-server can only read from 'local' region's storage servers
> * connected 'storage' and 'replication' networks -- in this case proxy-server can read from both 'local' and 'foreign' region

Excellent points.

It is especially important to recognize that inter-region replication will almost certainly be done not only with
VPN tunnels, but with traffic shaped VPN tunnels. 

Everyone agrees that replicating content to other regions is important, but nobody wants it to take precedence
over accessing the email server.