[Openstack] error creating instance
Andreas Scheuring
scheuran at linux.vnet.ibm.com
Tue Jun 30 14:50:55 UTC 2015
OK, this is what I got
- 2x External network (flat or fixed vlan?)
- Tenant network: That allows for connecting to other external servers
on the same l2 net + gre
--> I assume you have a fixed vlan that you need to use, right?
--> In addition you want to have gre tenant networks?
OK, now coming to the requirements: Each l2 network requires its own
interface. Background: for l2 networking (flat/vlan) the physical port
is plugged into the vswitch. Obviously a port can only be plugged in one
vswitch at once. So you need at least 2 interfaces for your external
networks and one for your tenant vlan network. By interface I mean
either an ethx interface or a vlan device.
But let's start with the tenant network! Don't care about the external
networks at first.
Upfront: I'm not an expert - I only have experience with a single type
of tenant networks. So this is what I can explain. There might be other
options that I'm not aware of
You have 3 choices, it depends on what you need:
1) you want multiple gre & 1 vlan network for tenants
- Use gre for tenant networks
- Create a single provider network that matches your l2 network
2) you want multiple vlan networks for tenants
- use vlan for tenant networks (vlan ids will be picked dynamically
along a range you provide)
3) You just need one tenant network, that matches your specific vlan id
- Create a single provider network that matches your l2 network
- Or use vlan tenant networks providing a range with a single vlan id
Which one would be the right way?
On Di, 2015-06-30 at 13:51 +0000, Yngvi Páll Þorfinnsson wrote:
> Hi Andreas,
>
> Our plan is (or was) to do the following.
>
> Connect the servers with two physical interfaces, configured into a bound interface in order to provide failover resilance. Use this bound interface to provide all connectivity for the node (management, ext nets and Tennants).
> Additionally we wish to configure multible (at least two) external networks. One to connect to the open internet and the other to connect to trusted internal network within the company.
> And at last we wish the have the Tennant network configured as VLAN (at least as an option in addition to GRE and VXLAN if possible). The VLAN config is needed, since we also need (in addition to the extra external networks) to be able to create a Tennant subnet that connects on L2 to physical servers.
>
> After working on this, with great help from you, it seems to be rather difficult to get this running:-) So perhaps we need to rethink our plans? It also seems to be an security issue to use the same physical interface to provide ext, tennant and management networks.
>
> As to implement VLANs I configured
>
> root at network2:/# cat /proc/net/vlan/config
> VLAN Dev name | VLAN ID
> Name-Type: VLAN_NAME_TYPE_RAW_PLUS_VID_NO_PAD
> bond0.48 | 48 | bond0
> bond0.47 | 47 | bond0
> bond0.45 | 45 | bond0
>
> bond0.48 -> mgtm network
> bond0.47 -> tunnel network
> bond0.45 -> exteral network no.1
>
> currently only one external network has been configured.
>
> Best regards
> Yngvi
>
> -----Original Message-----
> From: Andreas Scheuring [mailto:scheuran at linux.vnet.ibm.com]
> Sent: 30. júní 2015 12:36
> To: Yngvi Páll Þorfinnsson
> Cc: openstack at lists.openstack.org
> Subject: Re: [Openstack] error creating instance
>
> Ok, that was not clear to me! I thought your intention was, to have the data network as vlan (which you configured with vlan 47 or something like that on top of your bond) and then use gre on this vlan for tenant networks.
>
> So you want to have one network that uses vlan 47? Or do you want to have the cloudy thing, that you can create as many network as you like each with an increasing vlan id?
>
>
> So let's exactly define what you're looking for, before making other suggestions ;)
>
>
> I came to this gre assumption, as all your openvswitch-agent config files still have gre configured! (see ml2.conf - always tunneling
> enabled)
>
>
> To see where we are, could you please authenticate as admin and dump
>
> > neutron net-show demo-net1
>
>
>
>
>
> On Di, 2015-06-30 at 11:22 +0000, Yngvi Páll Þorfinnsson wrote:
> > HI Andreas
> >
> > Just to make it clear,
> > we need to have our tenant networks as VLANs, in order to have native
> > connection from VM to physical host on our server networks.
> >
> > Best regards
> > Yngvi
> >
> > -----Original Message-----
> > From: Yngvi Páll Þorfinnsson
> > Sent: 30. júní 2015 08:15
> > To: Andreas Scheuring
> > Cc: openstack at lists.openstack.org
> > Subject: Re: [Openstack] error creating instance
> >
> > Well, I don't have the dashboard yet. I've installed, but I'm waiting for access from the network guys...
> >
> > I'm now in a kind of dilemme, trying to delete the network;
> >
> > root at controller2:/# neutron port-list
> > +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
> > | id | name | mac_address | fixed_ips |
> > +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
> > | 0bcdb649-28cc-47d3-9a84-62b9aece589e | | fa:16:3e:9c:cb:6d | {"subnet_id": "c40fa8e3-cd8e-4566-ade6-5f3eabed121c", "ip_address": "157.157.8.51"} |
> > | 231895ce-4d1f-4fba-a5e7-8dc12843cb3e | | fa:16:3e:97:68:fe | {"subnet_id": "2c79bb00-0ace-4319-8151-81210ee3dfb2", "ip_address": "172.22.18.3"} |
> > | f371a118-8270-496a-9868-5e921949094d | | fa:16:3e:eb:1c:29 | {"subnet_id": "2c79bb00-0ace-4319-8151-81210ee3dfb2", "ip_address": "172.22.18.1"} |
> > +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
> > root at controller2:/#
> > root at controller2:/# neutron router-list
> > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
> > | id | name | external_gateway_info | distributed | ha |
> > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
> > | 80dd1cff-a530-43c2-97e8-7cb331e928cb | demo-router | {"network_id": "b43da44a-42d5-4b1f-91c2-d06a923deb29", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "c40fa8e3-cd8e-4566-ade6-5f3eabed121c", "ip_address": "157.157.8.51"}]} | False | False |
> > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
> > root at controller2:/#
> > root at controller2:/# neutron port-delete
> > 0bcdb649-28cc-47d3-9a84-62b9aece589e
> > Port 0bcdb649-28cc-47d3-9a84-62b9aece589e has owner
> > network:router_gateway and therefore cannot be deleted directly via
> > the port API. (HTTP 409) (Request-ID:
> > req-0aff43f0-5650-4a9c-82c0-211fb6fcebb6)
> > root at controller2:/# neutron router-delete
> > 80dd1cff-a530-43c2-97e8-7cb331e928cb
> > Router 80dd1cff-a530-43c2-97e8-7cb331e928cb still has ports (HTTP 409)
> > (Request-ID: req-bf34e06f-41e8-419a-8e5a-00061cd9d5f9)
> > root at controller2:/#
> >
> >
> > 2) The demo network
> >
> > Oh, I just followed the manual, so it's like this:
> >
> > source demo-openrc.sh
> > neutron net-create demo-net
> > neutron subnet-create demo-net --name demo-subnet \ --gateway x.y.z.1
> > x.y.z.0/24
> >
> > And the router was also created according to the manual;
> >
> > neutron router-create demo-router
> > neutron router-interface-add demo-router demo-subnet
> >
> > and then is was attached to the external net:
> >
> > Attach the router to the external network by setting it as the gateway:
> >
> > neutron router-gateway-set demo-router ext_net1101
> >
> > Best regards
> > Yngvi
> >
> > -----Original Message-----
> > From: Andreas Scheuring [mailto:scheuran at linux.vnet.ibm.com]
> > Sent: 30. júní 2015 08:01
> > To: Yngvi Páll Þorfinnsson
> > Cc: openstack at lists.openstack.org
> > Subject: Re: [Openstack] error creating instance
> >
> > Ok,
> >
> > The best thing is to login via the dashboard. Go to your router and remove the interface to the internal network. There's also an api call, but I don't know it from the top of my head.
> > Then try to remove the network again.
> >
> > Before recreating it:
> >
> > How did you create your demo-network? Did you specify to have a vlan
> > network? If NOT, you should update the following configuration on your
> > controller node tenant_network_types = vlan,gre -->
> > tenant_network_types = gre This ensures, that only gre network are
> > being created (I'm not 100% sure how this behehaves with multiple
> > entries here...)
> >
> > Restart your neutron-server
> >
> > Create a network without specifiying anything about gre or vlan or something else.
> >
> > Your gre network then will created and used within your static host vlan you configured (vlan 47).
> >
> >
> > Andreas
> >
> >
> >
> > On Di, 2015-06-30 at 07:53 +0000, Yngvi Páll Þorfinnsson wrote:
> > > OK Andreas, I have different id's now on the networks, Since I
> > > dropped neutron db yesterday, and created again, ( as well as the
> > > networks)
> > >
> > > But it's the same setup, and now the demo network is like this:
> > >
> > > root at controller2:/# neutron net-list
> > > +--------------------------------------+-------------+-----------------------------------------------------+
> > > | id | name | subnets |
> > > +--------------------------------------+-------------+-----------------------------------------------------+
> > > | b43da44a-42d5-4b1f-91c2-d06a923deb29 | ext_net1101 | c40fa8e3-cd8e-4566-ade6-5f3eabed121c 157.157.8.0/24 |
> > > | 3446e54b-346f-45e5-89a2-1ec4eef251ab | demo-net | 2c79bb00-0ace-4319-8151-81210ee3dfb2 172.22.18.0/24 |
> > > +--------------------------------------+-------------+-----------------------------------------------------+
> > > root at controller2:/#
> > > root at controller2:/# neutron net-show
> > > 3446e54b-346f-45e5-89a2-1ec4eef251ab
> > > +---------------------------+--------------------------------------+
> > > | Field | Value |
> > > +---------------------------+--------------------------------------+
> > > | admin_state_up | True |
> > > | id | 3446e54b-346f-45e5-89a2-1ec4eef251ab |
> > > | name | demo-net |
> > > | provider:network_type | vlan |
> > > | provider:physical_network | external |
> > > | provider:segmentation_id | 1102 |
> > > | router:external | False |
> > > | shared | False |
> > > | status | ACTIVE |
> > > | subnets | 2c79bb00-0ace-4319-8151-81210ee3dfb2 |
> > > | tenant_id | f976b7c713c64b028c0e89f3956795ed |
> > > +---------------------------+--------------------------------------+
> > > root at controller2:/#
> > >
> > > and this is the one I should delete then ( and re-create).
> > >
> > > I'm having difficulties deleting the network ;
> > >
> > > root at controller2:/# source demo-openrc.sh root at controller2:/#
> > > root at controller2:/# neutron net-delete
> > > 3446e54b-346f-45e5-89a2-1ec4eef251ab
> > > Unable to complete operation on network
> > > 3446e54b-346f-45e5-89a2-1ec4eef251ab. There are one or more ports
> > > still in use on the network. (HTTP 409) (Request-ID:
> > > req-432fee24-e554-42fb-bfc4-f9d597903eea)
> > > root at controller2:/#
> > > root at controller2:/# neutron router-list
> > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> > > | id | name | external_gateway_info |
> > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> > > | 80dd1cff-a530-43c2-97e8-7cb331e928cb | demo-router | {"network_id":
> > > | "b43da44a-42d5-4b1f-91c2-d06a923deb29", "enable_snat": true,
> > > | "external_fixed_ips": [{"subnet_id":
> > > | "c40fa8e3-cd8e-4566-ade6-5f3eabed121c", "ip_address":
> > > | "157.157.8.51"}]} |
> > > +--------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
> > > root at controller2:/#
> > > root at controller2:/# neutron router-delete
> > > 80dd1cff-a530-43c2-97e8-7cb331e928cb
> > > Router 80dd1cff-a530-43c2-97e8-7cb331e928cb still has ports (HTTP
> > > 409)
> > > (Request-ID: req-a90df408-2551-4f18-86d4-263daa325307)
> > >
> > > But , nova list is empty;
> > >
> > > root at controller2:/# nova list
> > > +----+------+--------+------------+-------------+----------+
> > > | ID | Name | Status | Task State | Power State | Networks |
> > > +----+------+--------+------------+-------------+----------+
> > > +----+------+--------+------------+-------------+----------+
> > > root at controller2:/#
> > >
> > >
> > > Best regards
> > > Yngvi
> > >
> > > -----Original Message-----
> > > From: Andreas Scheuring [mailto:scheuran at linux.vnet.ibm.com]
> > > Sent: 30. júní 2015 06:43
> > > To: Yngvi Páll Þorfinnsson
> > > Cc: openstack at lists.openstack.org
> > > Subject: Re: [Openstack] error creating instance
> > >
> > > Now all makes absolutely sense:
> > >
> > >
> > > +---------------------------+--------------------------------------+
> > > | Field | Value |
> > > +---------------------------+--------------------------------------+
> > > | admin_state_up | True |
> > > | id | 7a344656-815c-4116-b697-b52f9fdc6e4c |
> > > | name | demo-net |
> > > | provider:network_type | vlan |
> > > | provider:physical_network | external |
> > > | provider:segmentation_id | 1102 |
> > > | router:external | False |
> > > | shared | False |
> > > | status | ACTIVE |
> > > | subnets | c57880bf-2869-4b1d-b31c-0d2d083d05e4 |
> > > | tenant_id | f976b7c713c64b028c0e89f3956795ed |
> > > +---------------------------+--------------------------------------+
> > >
> > > This is you demo network. It's mapped to the pyhsical network "external" (defined in bridgemapping) and uses vlan.
> > >
> > > But that the wrong way. If I get you right, you still want to use gre but now over your static vlan, right? So what you need to do is, to delete your demo-net.
> > >
> > > > neutron net-delete 7a344656-815c-4116-b697-b52f9fdc6e4c
> > >
> > > And just create a new one without specifying any provider attributes
> > >
> > > > neutron net-create ...
> > > Not sure, but I guess it's sufficient to specify a name!
> > >
> > > > neutron net-show <your-new-net>
> > >
> > > It should look somehting like this:
> > >
> > > --> no pyhsical network
> > > --> network_type = gre
> > > +---------------------------+--------------------------------------+
> > > > | Field | Value |
> > > > +---------------------------+--------------------------------------+
> > > > | admin_state_up | True |
> > > > | id | ef6552a5-be39-4bcc-9dde-2a200eaca64d |
> > > > | mtu | 0 |
> > > > | name | private |
> > > > | provider:network_type | vxlan |
> > > > | provider:physical_network | |
> > > > | provider:segmentation_id | 1001 |
> > > > | router:external | False |
> > > > | shared | False |
> > > > | status | ACTIVE |
> > > > | subnets | 4b539feb-b104-4f69-83ba-76f746a2c592 |
> > > > | | ac255618-afe9-4aea-b86d-b662b68e9d9d |
> > > > | tenant_id | 3c4ddcff52a74f2b97b71392300aa74d |
> > > > +---------------------------+--------------------------------------+
> > >
> > >
> > >
> > > If you want to switch over totally to vlan (not using gre in your static vlan anymore), you have to reconfigure the ml2 config files. But I'm not sure, if dynamic Openstack vlans nested into a static host vlan work out. But let's discuss this if relevant.
> > >
> > >
> > > You can also ping me on irc (freenode, #openstack, to speed up debugging. My nic name is 'scheuran'.
> > >
> > >
> > >
> > > Andreas
> > >
> > >
> > > On Mo, 2015-06-29 at 15:54 +0000, Yngvi Páll Þorfinnsson wrote:
> > > > OK,
> > > >
> > > > This is the network list
> > > >
> > > > root at controller2:/# neutron net-list
> > > > +--------------------------------------+-------------+-----------------------------------------------------+
> > > > | id | name | subnets |
> > > > +--------------------------------------+-------------+-----------------------------------------------------+
> > > > | 1cb21927-6996-4022-8d3e-292390dad6d0 | ext_net1101 | 7b84ec73-eaa7-479f-872f-41c6cb2083c5 157.157.8.0/24 |
> > > > | 7a344656-815c-4116-b697-b52f9fdc6e4c | demo-net | c57880bf-2869-4b1d-b31c-0d2d083d05e4 172.22.18.0/24 |
> > > > +--------------------------------------+-------------+-----------------------------------------------------+
> > > > root at controller2:/#
> > > > root at controller2:/#
> > > > root at controller2:/# neutron net-show
> > > > 1cb21927-6996-4022-8d3e-292390dad6d0
> > > > +---------------------------+--------------------------------------+
> > > > | Field | Value |
> > > > +---------------------------+--------------------------------------+
> > > > | admin_state_up | True |
> > > > | id | 1cb21927-6996-4022-8d3e-292390dad6d0 |
> > > > | name | ext_net1101 |
> > > > | provider:network_type | vlan |
> > > > | provider:physical_network | external |
> > > > | provider:segmentation_id | 1101 |
> > > > | router:external | True |
> > > > | shared | False |
> > > > | status | ACTIVE |
> > > > | subnets | 7b84ec73-eaa7-479f-872f-41c6cb2083c5 |
> > > > | tenant_id | c96aec0da5e542ad8e3198aaccc498c7 |
> > > > +---------------------------+--------------------------------------+
> > > > root at controller2:/# neutron net-show
> > > > 7a344656-815c-4116-b697-b52f9fdc6e4c
> > > > +---------------------------+--------------------------------------+
> > > > | Field | Value |
> > > > +---------------------------+--------------------------------------+
> > > > | admin_state_up | True |
> > > > | id | 7a344656-815c-4116-b697-b52f9fdc6e4c |
> > > > | name | demo-net |
> > > > | provider:network_type | vlan |
> > > > | provider:physical_network | external |
> > > > | provider:segmentation_id | 1102 |
> > > > | router:external | False |
> > > > | shared | False |
> > > > | status | ACTIVE |
> > > > | subnets | c57880bf-2869-4b1d-b31c-0d2d083d05e4 |
> > > > | tenant_id | f976b7c713c64b028c0e89f3956795ed |
> > > > +---------------------------+--------------------------------------+
> > > > root at controller2:/#
> > > >
> > > > and this is how I created the external network, with CMD ;
> > > >
> > > > neutron net-create ext_net1101 --provider:network_type vlan
> > > > --provider:physical_network external --provider:segmentation_id
> > > > 1101 --router:external=True
> > > >
> > > >
> > > > But there seems not to be info on this one root at controller2:/#
> > > > neutron net-show cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca
> > > > Unable to find network with name 'cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca'
> > > >
> > > >
> > > > best regards
> > > > Yngvi
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Andreas Scheuring [mailto:scheuran at linux.vnet.ibm.com]
> > > > Sent: 29. júní 2015 15:25
> > > > To: Yngvi Páll Þorfinnsson
> > > > Cc: uwe.sauter.de at gmail.com; openstack at lists.openstack.org
> > > > Subject: Re: [Openstack] error creating instance
> > > >
> > > >
> > > >
> > > > Attempting to bind port 2bf4a49b-2ad6-4ead-a656-65814ad0724e on
> > > > network 7a344656-815c-4116-b697-b52f9fdc6e4c
> > > > bind_port
> > > > /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_
> > > > ag
> > > > en
> > > > t.py:57
> > > > 2015-06-29 14:28:55.924 5328 DEBUG
> > > > neutron.plugins.ml2.drivers.mech_agent
> > > > [req-9fe66e60-1a70-4ad6-b21e-ef91aca8a931 None] Checking agent:
> > > > {'binary': u'neutron-openvswitch-agent', 'description': None,
> > > > 'admin_state_up': True, 'heartbeat_timestamp': datetime.datetime(2015, 6, 29, 14, 28, 45), 'alive': True, 'id':
> > > > u'1c06fb08-105c-4659-ae0e-4a905931311e', 'topic': u'N/A', 'host':
> > > > u'compute5', 'agent_type': u'Open vSwitch agent', 'started_at':
> > > > datetime.datetime(2015, 6, 29, 14, 27, 45), 'created_at':
> > > > datetime.datetime(2015, 6, 26, 14, 51, 14), 'configurations':
> > > > {u'arp_responder_enabled': False, u'tunneling_ip':
> > > > u'172.22.15.17',
> > > > u'devices': 0, u'l2_population': False, u'tunnel_types': [u'gre'],
> > > > u'enable_distributed_routing': False, u'bridge_mappings': {}}}
> > > > bind_port
> > > > /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_
> > > > ag
> > > > en
> > > > t.py:65
> > > > 2015-06-29 14:28:55.925 5328 DEBUG
> > > > neutron.plugins.ml2.drivers.mech_openvswitch
> > > > [req-9fe66e60-1a70-4ad6-b21e-ef91aca8a931 None] Checking segment:
> > > > {'segmentation_id': 1102L, 'physical_network': u'external', 'id':
> > > > u'cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca', 'network_type': u'vlan'}
> > > > for
> > > > mappings: {} with tunnel_types: [u'gre'] check_segment_for_agent
> > > > /usr/lib/python2.7/dist-packages/neutron/plugins/ml2/drivers/mech_
> > > > op
> > > > en
> > > > vswitch.py:52
> > > >
> > > >
> > > >
> > > > ===
> > > > Checking segment: {'segmentation_id': 1102L, 'physical_network':u'external', 'id':
> > > > u'cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca', 'network_type': u'vlan'}
> > > > for
> > > > mappings: {} with tunnel_types: [u'gre']
> > > >
> > > > This looks strange: Seems like your tenant network has a physical_network of type vlan assigned. That shouldn't be the case.
> > > >
> > > > Could you please provide the following information:
> > > >
> > > > Information of all Openstack networks available:
> > > >
> > > > > neutron net-list
> > > >
> > > > > neutron net-show <uuid>
> > > >
> > > > Especially of this one:
> > > > > neturon net-show cf6489c4-7ed6-43dc-85aa-f4b8c6b501ca
> > > >
> > > >
> > > > Usually your network should look like this (in this case vxlan):
> > > >
> > > > +---------------------------+--------------------------------------+
> > > > | Field | Value |
> > > > +---------------------------+--------------------------------------+
> > > > | admin_state_up | True |
> > > > | id | ef6552a5-be39-4bcc-9dde-2a200eaca64d |
> > > > | mtu | 0 |
> > > > | name | private |
> > > > | provider:network_type | vxlan |
> > > > | provider:physical_network | |
> > > > | provider:segmentation_id | 1001 |
> > > > | router:external | False |
> > > > | shared | False |
> > > > | status | ACTIVE |
> > > > | subnets | 4b539feb-b104-4f69-83ba-76f746a2c592 |
> > > > | | ac255618-afe9-4aea-b86d-b662b68e9d9d |
> > > > | tenant_id | 3c4ddcff52a74f2b97b71392300aa74d |
> > > > +---------------------------+--------------------------------------+
> > > >
> > > > How did you create yours? via the UI? Or are you attaching your instance to the external network instead? In any case you need to attach it to your tenant network!! If it's not visible via the UI, maybe you have to switch to another tenant to get it.
> > > >
> > > > Hope we're close to finding the issue ;)
> > > >
> > > >
> > > > Andreas
> > > >
> > > >
> > >
> > >
> >
> > --
> > Andreas
> > (IRC: scheuran)
> >
> >
> > _______________________________________________
> > Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> > Post to : openstack at lists.openstack.org
> > Unsubscribe :
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
> --
> Andreas
> (IRC: scheuran)
>
>
--
Andreas
(IRC: scheuran)
More information about the Openstack
mailing list