<div dir="ltr">Zhou,<div><br></div><div>On Wed, Jun 24, 2015 at 8:38 PM Zhou Zheng Sheng / 周征晟 <<a href="mailto:zhengsheng@awcloud.com">zhengsheng@awcloud.com</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hi,<br>
<br>
I also notice that in latest ceph-deploy source code, it allows us
to define public_network and cluster_network argument when calling
'ceph-deploy new'. It also checks if monitor IP address is in either
of the two networks.<br>
<br>
<a href="https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/new.py#L158" target="_blank">https://github.com/ceph/ceph-deploy/blob/master/ceph_deploy/new.py#L158</a><br>
<br>
I'm not sure that Ceph monitor operating in the public network is
just an artificial requirement, or it's an inherent precondition
when they implement Ceph and breaking the precondition might lead to
some strange problems. If it's the latter case, when we change Ceph
public network, we have to change Ceph monitor IP address
configuration at the same time.</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
<div>on 2015/06/25 11:01, Zhou Zheng Sheng /
周征晟 wrote:<br>
</div>
</div><div bgcolor="#FFFFFF" text="#000000"><blockquote type="cite">
Hello Andrew,<br>
<br>
I did some similar modifications in Fuel 6.0 as [1][2]. However in
[3] it says Ceph monitor have to run on the public network.<br></blockquote></div></blockquote><div>It doesn't have to run on fuel public network, ceph's "public" is required for MONs and OSDs in this case "public" is because it's expected that the Ceph clients will connect to it. Where "cluster" is used for intra-OSD communication. We started by assigning Ceph's "public" to Fuel's "management"("internal" in this scope) and Ceph's "cluster" to Fuel's "stroage"</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><blockquote type="cite">
In the file "deployment/puppet/ceph/manifests/conf.pp", we can see
it calls 'ceph-deploy new ${::hostname}:${::internal_address}'.<br>
In "deployment/puppet/ceph/manifests/mon.pp", it calls
'ceph-deploy mon create ${::hostname}:${::internal_address}'.<br></blockquote></div></blockquote><div>Thanks, I thought we removed these codes already and where using the class entry points, We will fix these as part of the change over.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><blockquote type="cite"></blockquote></div><div bgcolor="#FFFFFF" text="#000000"><blockquote type="cite">
After we set Ceph public network to OpenStack storage network,
Ceph monitor still runs on management network. I can see in QEMU
command line, it connects to Ceph via management network IP
address. Though this setup does not follow the rule suggested in
[3], it probably works because Ceph monitor only provides PG
location information, and then QEMU should talk to OSDs directly
using the storage network addresses.</blockquote></div><div bgcolor="#FFFFFF" text="#000000"><blockquote type="cite"><br>
<br>
That's why I didn't take your patches as-is. I meant to setup a
Fuel 6.1 environment and test the modifications before I can give
an accurate comment.<br>
<br>
[3]
<a href="http://ceph.com/docs/master/rados/configuration/network-config-ref/#monitor-ip-tables" target="_blank">http://ceph.com/docs/master/rados/configuration/network-config-ref/#monitor-ip-tables</a><br>
<br>
<div>on 2015/06/25 03:20, Andrew Woodward
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Zhou,
<div><br>
</div>
<div>As mentioned, Please review [1][2]. This is the interface
we will support as we implement the advanced networking
parts. In your case, just fip the role map in your patch to
nailgun and the library interface will remain the same.</div>
<div><br>
</div>
<div>[1] <a href="https://review.openstack.org/#/c/194434/" target="_blank">https://review.openstack.org/#/c/194434/</a></div>
<div>[2] <a href="https://review.openstack.org/#/c/194438/" target="_blank">https://review.openstack.org/#/c/194438/</a><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, Jun 24, 2015 at 3:21 AM Stanislav
Makar <<a href="mailto:smakar@mirantis.com" target="_blank">smakar@mirantis.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>Hello<br>
</div>
My five cents :)<br>
</div>
<div>I also very proud of our 6.1 release<br>
</div>
Unfortunately we still do not have this separation for
ceph public and cluster networks in 6.1.<br>
<br>
</div>
<div>One little hint <br>
In 6.1 we have granular deployment (<a href="https://docs.mirantis.com/fuel/fuel-master/release-notes.html#granular-deployment-based-on-pluggable-tasks" target="_blank">https://docs.mirantis.com/fuel/fuel-master/release-notes.html#granular-deployment-based-on-pluggable-tasks</a>)
and all tasks which are connected with ceph deployment
are in <br>
</div>
<div><br>
<a href="https://github.com/stackforge/fuel-library/tree/stable/6.1/deployment/puppet/osnailyfacter/modular/ceph" target="_blank">https://github.com/stackforge/fuel-library/tree/stable/6.1/deployment/puppet/osnailyfacter/modular/ceph</a><br>
<br>
</div>
<div>To have your patch working in 6.1 you should hack
all these files now<br>
<br>
</div>
<div>If you have question feel free to ask.<br>
</div>
<div><br>
</div>
<div>Thanks.<br>
</div>
<div><br>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr"><font face="times new
roman, serif" size="2">--</font>
<div><font face="times new roman,
serif" size="2">All the best,</font>
<div><font face="times new roman,
serif" size="2">Stanislav
Makar<br>
skype: makar_stanislav </font></div>
<div><font face="times new roman,
serif" size="2">irc: stamak</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div dir="ltr">
<div class="gmail_extra"> <br>
<div class="gmail_quote">On Wed, Jun 24, 2015 at 12:04
PM, Sergii Golovatiuk <span dir="ltr"><<a href="mailto:sgolovatiuk@mirantis.com" target="_blank">sgolovatiuk@mirantis.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>Hi Zhou,<br>
<br>
</div>
Try Fuel 6.1 where we have a lot of very nice
features including 'advanced networking'. Feel
free to read release notes [1]. It's one of the
most significant releases with many many
features and improvements. I am very proud of
it.<br>
<br>
[1] <a href="https://docs.mirantis.com/fuel/fuel-master/release-notes.html" target="_blank">https://docs.mirantis.com/fuel/fuel-master/release-notes.html</a><br>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div>
<div dir="ltr">--<br>
Best regards,<br>
Sergii Golovatiuk,<br>
Skype #golserge<br>
IRC #holser<br>
</div>
</div>
</div>
<div>
<div> <br>
<div class="gmail_quote">On Wed, Jun 24,
2015 at 11:42 AM, Zhou Zheng Sheng / 周征晟 <span dir="ltr"><<a href="mailto:zhengsheng@awcloud.com" target="_blank">zhengsheng@awcloud.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thank you
guys!<br>
<br>
I'm reading the advanced networking spec
and think it's good. I'll try<br>
to comment it. It's too epic for me for
now. I wrote a patch to move<br>
Ceph public network to storage network.
It's based on Fuel 6.0, so I'm<br>
trying to rebase it to lastest master
then maybe ask you for a review.<br>
<div>
<div><br>
on 2015/06/23 19:21, Igor Kalnitsky
wrote:<br>
> Hello,<br>
><br>
> That makes sense to me. Still,
I want to point that we're going to<br>
> implement advanced networking
and with this feature you'll be able
to<br>
> assign every single network
role to any network.<br>
><br>
> That means, you'll be able to
assign ceph network role to storage,<br>
> management or
whatever-you-want network. Sounds
cool, ha? :)<br>
><br>
> Feel free to read a design spec
[1].<br>
><br>
> Thanks,<br>
> Igor<br>
><br>
> [1]: <a href="https://review.openstack.org/#/c/115340/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/115340/</a><br>
><br>
> On Tue, Jun 23, 2015 at 1:13
PM, Zhou Zheng Sheng / 周征晟<br>
> <<a href="mailto:zhengsheng@awcloud.com" target="_blank">zhengsheng@awcloud.com</a>>
wrote:<br>
>> Hi!<br>
>><br>
>> I notice that in OpenStack
deployed by Fuel, Ceph public
network is on<br>
>> management network. In some
environments, not all NICs of a
physical<br>
>> server are 10Gb. Sometimes
1 or 2 among the NICs on a machine
may be<br>
>> 1Gb. Usually on this type
of machine we assign management
network to 1Gb<br>
>> NIC, and storage network to
10Gb NIC. If Ceph public network is
with<br>
>> management network, the
QEMU accesses Ceph using management
network, and<br>
>> the performance is not
optimal.<br>
>><br>
>> In a small deployment,
cloud controller and Ceph OSD may be
assigned to<br>
>> the same machine, so it
would be more effective to keep Ceph
client<br>
>> traffic separated from
MySQL, RabbitMQ, Pacemaker traffic.
Maybe it's<br>
>> better to place Ceph public
network on the storage network.
Agree?<br>
>><br>
>> --<br>
>> Best wishes!<br>
>> Zhou Zheng Sheng, Software
Developer<br>
>> Beijing AWcloud Software
Co., Ltd.<br>
>><br>
>><br>
>><br>
>><br>
>>
__________________________________________________________________________<br>
>> OpenStack Development
Mailing List (not for usage
questions)<br>
>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>
__________________________________________________________________________<br>
> OpenStack Development Mailing
List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
--<br>
Best wishes!<br>
</div>
</div>
Zhou Zheng Sheng / 周征晟 Software
Developer<br>
<div>
<div>Beijing AWcloud Software Co.,
Ltd.<br>
<br>
<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List
(not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage
questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage
questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote>
</div>
</div>
<div dir="ltr">-- <br>
</div>
<p dir="ltr">--</p>
<p dir="ltr">Andrew Woodward</p>
<p dir="ltr">Mirantis</p>
<p dir="ltr">Fuel Community Ambassador</p>
<p dir="ltr">Ceph Community</p>
<br>
<fieldset></fieldset>
<br>
<pre>__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
<pre cols="72">--
Best wishes!
Zhou Zheng Sheng / 周征晟 Software Developer
Beijing AWcloud Software Co., Ltd.</pre>
</blockquote></div><div bgcolor="#FFFFFF" text="#000000">
<br>
<pre cols="72">--
Best wishes!
Zhou Zheng Sheng / 周征晟 Software Developer
Beijing AWcloud Software Co., Ltd.</pre>
</div>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div></div><div dir="ltr">-- <br></div><div dir="ltr">--<div>Andrew Woodward</div><div>Mirantis</div><div>Fuel Community Ambassador</div><div>Ceph Community </div></div>