[openstack-dev] [Quantum] continuing todays discussion about the l3 agents
Gary Kotton
gkotton at redhat.com
Mon Dec 3 09:32:56 UTC 2012
On 12/03/2012 04:16 AM, gong yong sheng wrote:
> On 12/03/2012 02:29 AM, Vinay Bannai wrote:
>> My understanding of the "scheduler" approach based on what I read on
>> the ML's is to have a mechanism where the DHCP agents can land on
>> different nodes. For example, just like we have compute hosts in
>> nova, we have a bunch of DHCP capable hosts (and L3 capable hosts
>> etc) that can be selected to host the network service for a tenant
>> when the network/subnet is created. The process of selecting the host
>> to run the service is based on a "scheduler". This allows a graceful
>> horizontal scaling. This approach is similar to what nova does. You
>> have a bunch of hosts capable of providing a network service and the
>> "scheduler" picks them based on filters and other tunable knobs. I
>> think you already know this:-). I was spelling it out so that you
>> can see where I am coming from.
> If we don't want all dhcp agents to host the data of all the networks,
> My Idea is:
> 1. let quantum server have the ability to know all about the dhcp
> agents. for example we can have quantum agents-list to show all the
> agents running in the quantum deloyment,
> and the network they are hosting.
ok, this may be useful for debugging. will this display and have the
status of the dhcp agent, say for example i deploy and agent and it has
an exception do to a bug?
> 2. let admin user have the ability to config the dhcp agents what
> networks they should host. For example, quantum dhcpagent-update
> dhcpagent1 --networks network1 network2 network3. or quantum
> net-create network1 --dhcpagents agent1 agent2. And if admin user does
> not specify which agent to host which network, we can let scheduler to
> decide automatically
this is exactly what i am suggesting, except we do not need to change
the quantum api to provide this. the agent can receive this as an input
parameter. in principle we agree on what needs to be done, but the
question is how.
> So for scale vertically:
> we can specify much agents host some same networks
> So for scale horizontally:
> we can add as many as dhcp agents. quantum scheduler will distribute
> new networks automatically or admin user can specify.
i have a number of problems with a quantum "scheduler". the first being
a single point of failure. the second being the fact that it needs to be
aware of the state and load of a dhcp agent. how will the scheduler
provide for HA?
> For us to run multiple dhcp agents, we need to make sure our dhcp anti
> spoofing work.
>>
>> Either way we look at it, I think it will be helpful if we decoupled
>> the horizontal (scaling to multiple nodes) and vertical scaling
>> (redundancy and failover). One should not imply the other. In your
>> last paragraph, you mention "orchestration tool" and dhcp agents
>> configured to handle specific networks. I have not been able to wrap
>> my head around this completely but it appears to b ea different
>> variant of the "scheduler" approach where it is configured manually.
>> Is my understanding correct? Or if you don't mind, can you elaborate
>> further on that idea.
>>
>> Thanks
>> Vinay
>>
>> On Sun, Dec 2, 2012 at 7:16 AM, Gary Kotton <gkotton at redhat.com
>> <mailto:gkotton at redhat.com>> wrote:
>>
>> On 12/01/2012 03:31 AM, gong yong sheng wrote:
>>> On 12/01/2012 07:49 AM, Vinay Bannai wrote:
>>>> Gary and Mark,
>>>>
>>>> You brought up the issue of scaling horizontally
>>>> and vertically in your earlier email. In the case of horizontal
>>>> scaling, I would agree that it would have to be based on the
>>>> "scheduler" approach proposed by Gong and Nachi.
>>
>> I am not sure that I understand the need for a scheduler when it
>> comes to the DHCP agent. In my opinion this is unnecessary
>> overhead and it is not necessarily required.
>>
>> Last week Mark addressed the problem with all of the DHCP agents
>> all listening on the same message queue. In theory we are able to
>> run more than one DHCP agents in parallel. This offers HA at the
>> expense of an IP per DHCP agent per subnet.
>>
>> I think that for the DHCP agents we need to look into enabling
>> the DHCP agents to treat specific networks. This can be done in a
>> very rudimentary way - have a configuration variable for the DHCP
>> agent indicating a list of networks to be treated by the agent. A
>> orchestration tool can just configure the network ID's and launch
>> the service - then we will have scalable and highly available
>> DHCP service. I would prefer not to have to add this into the
>> Quantum API as it just complicates things.
>>
>>
>>
>>
>>
>>>> On the issue of vertical scaling (I am using the DHCP
>>>> redundancy as an example), I think it would be good to base our
>>>> discussions on the various methods that have been discussed and
>>>> do pro/con analysis in terms of scale, performance and other
>>>> such metrics.
>>>>
>>>> - Split scope DHCP (two or more servers split the IP address
>>>> and there is no overlap)
>>>> pros: simple
>>>> cons: wastes IP addresses,
>>>>
>>>> - Active/Standby model (might have run VRRP or hearbeats to
>>>> dictate who is active)
>>>> pros: load evenly shared
>>>> cons: needs shared knowledge of address assignments,
>>>> need hearbeats or VRRP to keep track of failovers
>>> another one is the IP address waste. we need one VIP, and 2+
>>> more address for VRRP servers. ( we can use dhcp server's ip if
>>> we don't want to do load balancing behind the VRRP servers)
>>> another one is it will make system complicated.
>>>>
>>>> - LB method (use load balancer to fan out to multiple dhcp servers)
>>>> pros: scales very well
>>>> cons: the lb becomes the single point of failure,
>>>> the lease assignments needs to be shared between the
>>>> dhcp servers
>>>>
>>> LB method will also wast ip address. First we at lease need a
>>> VIP address. then we will need more dhcp servers running for one
>>> network.
>>> If we need to VRRP the VIP, we will need 2+ more addresses.
>>> another one is it will make system complicated.
>>>> I see that the DHCP agent and the quantum server communicate
>>>> using RPC. Is the plan to leave it alone or migrate it towards
>>>> something like AMQP based server in the future when the
>>>> "scheduler" stuff is implemented.
>>> I am not very clear your point. But current RPC is on AMQP.
>>>>
>>>> Vinay
>>>>
>>>>
>>>> On Wed, Nov 28, 2012 at 8:03 AM, Mark McClain
>>>> <mark.mcclain at dreamhost.com
>>>> <mailto:mark.mcclain at dreamhost.com>> wrote:
>>>>
>>>>
>>>> On Nov 28, 2012, at 8:03 AM, gong yong sheng
>>>> <gongysh at linux.vnet.ibm.com
>>>> <mailto:gongysh at linux.vnet.ibm.com>> wrote:
>>>>
>>>> > On 11/28/2012 08:11 AM, Mark McClain wrote:
>>>> >> On Nov 27, 2012, at 6:33 PM, gong yong sheng
>>>> <gongysh at linux.vnet.ibm.com
>>>> <mailto:gongysh at linux.vnet.ibm.com>> wrote:
>>>> >>
>>>> >> Just wanted to clarify two items:
>>>> >>
>>>> >>>> At the moment all of the dhcp agents receive all of
>>>> the updates. I do not see why we need the quantum service
>>>> to indicate which agent runs where. This will change the
>>>> manner in which the dhcp agents work.
>>>> >>> No. currently, we can run only one dhcp agent since we
>>>> are using a topic queue for notification.
>>>> >> You are correct. There is a bug in the underlying Oslo
>>>> RPC implementation that sets the topic and queue names to
>>>> be same value. I didn't get a clear explanation of this
>>>> problem until today and will have to figure out a fix to oslo.
>>>> >>
>>>> >>> And one problem with multiple agents serving the same
>>>> ip is:
>>>> >>> we will have more than one agents want to update the
>>>> ip's leasetime now and than.
>>>> >> This is not a problem. The DHCP protocol was designed
>>>> for multiple servers on a network. When a client accepts a
>>>> lease, the server that offered the accepted lease will be
>>>> the only process attempting to update the lease for that
>>>> port. The other DHCP instances will not do anything, so
>>>> there won't be any chance for a conflict. Also, when a
>>>> client renews it sends a unicast message to that previous
>>>> DHCP server and so there will only be one writer in this
>>>> scenario too. Additionally, we don't have to worry about
>>>> conflicting assignments because the dhcp agents use the
>>>> same static allocations from the Quantum database.
>>>> > I mean dhcp agent is trying to update leasetime to
>>>> quantum server. If we have more than one dhcp agents, this
>>>> will cause confusion.
>>>> > def update_lease(self, network_id, ip_address,
>>>> time_remaining):
>>>> > try:
>>>> >
>>>> self.plugin_rpc.update_lease_expiration(network_id,
>>>> ip_address,
>>>> >
>>>> time_remaining)
>>>> > except:
>>>> > self.needs_resync = True
>>>> > LOG.exception(_('Unable to update lease'))
>>>> > I think it is our dhcp agent's defect. Why does our dhcp
>>>> agent need the lease time? all the IPs are managed in our
>>>> quantum server, there is not need for dynamic ip management
>>>> in dhcp server managed by dhcp agent.
>>>>
>>>> There cannot be confusion. The dhcp client selects only
>>>> one server to accept a lease, so only one agent will update
>>>> this field at a time. (See RFC2131 section 4.3.2 for
>>>> protocol specifics). The dnsmasq allocation database is
>>>> static in Quantum's setup, so the lease renewal needs to
>>>> propagate to the Quantum Server. The Quantum server then
>>>> uses the lease time to avoid allocating IP addresses before
>>>> the lease has expired. In Quantum, we add an additional
>>>> restriction that expired allocations are not reclaimed
>>>> until the associated port has been deleted as well.
>>>>
>>>> mark
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> <mailto:OpenStack-dev at lists.openstack.org>
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Vinay Bannai
>>>> Email: vbannai at gmail.com <mailto:vbannai at gmail.com>
>>>> Google Voice: 415 938 7576 <tel:415%20938%207576>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org <mailto:OpenStack-dev at lists.openstack.org>
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org <mailto:OpenStack-dev at lists.openstack.org>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> <mailto:OpenStack-dev at lists.openstack.org>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> --
>> Vinay Bannai
>> Email: vbannai at gmail.com <mailto:vbannai at gmail.com>
>> Google Voice: 415 938 7576
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121203/f82a0791/attachment.html>
More information about the OpenStack-dev
mailing list