[openstack-dev] [vitrage] Extending Topology
Afek, Ifat (Nokia - IL/Kfar Sava)
ifat.afek at nokia.com
Mon Mar 27 10:32:20 UTC 2017
Hi,
Let me try and explain the more general use case.
You can query OVS for the switches information, and understand how they are mapped to one another. This is not enough for knowing the exact route of the network traffic for a certain VM.
A certain switch can be connected to more than one other switch. You can, as you said, query the network type (encapsulation) information from Neutron. But then you will need in addition to query the rules of the specific switch from OVS, in order to know which route to take for each encapsulation type.
Another problematic use case is when the switches are not connected to each other. The traffic can be redirected by a network-stack software component, so you will have to query it in addition in order to tell the route.
And on top of all this, we need to think how to best represent this information in Vitrage (i.e. how to draw the graph, which vertices to connect to one another, etc.).
IMO, this is all feasible and will give a lot of value to Vitrage. Just not easy to implement.
Best Regards,
Ifat.
On 22/03/2017, 08:50, "Muhammad Usman" <usman476 at gmail.com> wrote:
Hello Ifat,
I tried to see more in depth about the issues you mentioned regarding
the extension of vSwitches. Due to a lot of complexity involved in
generating this topology and associated effects, I believe we need to
setup some baseline (e.g. adding a configuration file for specifying
bridges in existing deployment setup). Then using that baseline,
topology can be constructed as well as type of network can be
extracted from neutron and associated path followed (e.g. vlan or
vxlan). However, more general case you mentioned, I cannot get it. Do
you mean nova-network?
Regarding the sunburst representation - Yes I agree, if you want to
keep compute hierarchy separate then addition of networking components
is not a good idea.
Also, suggestions from other vitrage members are welcomed.
> On Thu, Mar 16, 2017 at 6:44 PM, Afek, Ifat (Nokia - IL) <
> ifat.afek at nokia.com <javascript:;>> wrote:
>
>> Hi,
>>
>>
>>
>> Adding switches to the Vitrage topology is generally a good idea, but the
>> implementation might be very complex. Your diagram shows a simple use
> case,
>> where all switches are linked to one another and it is easy to determine
>> the effect of a failure on the vms. However, in the more general case
> there
>> might be switches without a connecting port (e.g. they might be connected
>> via the network stack). In such cases, it is not clear how to model the
>> switches topology in Vitrage. Another case to consider is when the
>> network
>> type affects the packets path, like vlan vs. vxlan. If you have an idea
>> of
>> how to solve these issues, I will be happy to hear it.
>>
>>
>>
>> Regarding the sunburst representation – I’m not sure I understand your
>> diagram. Currently the sunburst is meant to show (only) the compute
>> hierarchy: zones, hosts and instances. It is arranged in a containment
>> relationship, i.e. every instance on the outer ring appears next to its
>> host in the inner ring. If you add the bridges in the middle, you lose
> this
>> containment relationship. Can you please explain to me the suggested
>> diagram?
>>
>>
>>
>> BTW, you can send such questions to OpenStack mailing list (
>> openstack-dev at lists.openstack.org <javascript:;>) with [vitrage] tag in
> the title, and
>> possibly get replies from other contributors as well.
>>
>>
>>
>> Best Regards,
>>
>> Ifat.
>>
>>
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Monday, 13 March 2017 at 09:16
>>
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *Re: OpenStack Vitrage
>>
>>
>>
>> Hi Ifat,
>>
>> I attached our idea of extending the Vitrage Topology to include Virtual
>> switches.
>>
>> The reason, I mentioned about adding switches part in Vitrage is because
>> we experienced looping issues that effect all infrastructure resources
>> (i.e. physical host as well as vm's) performance. Therefore, it's
> important
>> to monitor the virtual switches as well to assist overall monitoring/RCA
>> tasks.
>>
>> I think this idea will extend the Vitrage scope to touch some portion of
>> SDN (e.g. if we consider the SDN managed virtual switches) as well.
>>
>>
>>
>> On Thu, Mar 9, 2017 at 6:49 PM, Muhammad Usman <usman at smartx.kr
> <javascript:;>> wrote:
>>
>> Dear Ifat,
>>
>> Thanks for your guidance, I managed to install Vitrage properly using
>> Master branches for both OpenStack and Vitrage.
>>
>> Now, I will look into the visualization as well as other aspects.
>>
>>
>>
>>
>>
>> On Thu, Mar 9, 2017 at 2:43 PM, Afek, Ifat (Nokia - IL) <
>> ifat.afek at nokia.com <javascript:;>> wrote:
>>
>> Hi,
>>
>>
>>
>> I have also noticed this problem, that Vitrage Ocata is not compatible
>> with Horizon Newton.
>>
>> If you just want an OpenStack working, you should use a stable version.
>> Stable/Ocata is the newest one (just released a few weeks ago). On the
>> other hand, if you want to contribute code, you better take the master
>> branch. Alternatively, you can take stable/ocata for all projects, and
>> the
>> master for Vitrage. This should work (for now, since Pike has just
> started).
>>
>>
>>
>> Best Regards,
>>
>> Ifat.
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Wednesday, 8 March 2017 at 15:21
>>
>>
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *Re: OpenStack Vitrage
>>
>>
>>
>> Ifat,
>>
>> after adding the mentioned line in /etc/heat/policy.json first error "You
>> are not authorized to use global_index" seems to be solved.
>>
>> However, in Horizon I still see same error (file is attached).
>>
>> So, After looking inside the code I found that I installed OpenStack
>> using
>> stable/newton branch but Vitrage is installed from Master branch. Since,
>> there are few changes in code (python-vitrageclient/
> vitrageclient/client.py)
>> that's why I think this error is occurring. Therefore, I will try to fix
>> this problem by using same branch (i.e. stable/newton).
>>
>>
>>
>> On Wed, Mar 8, 2017 at 9:08 PM, Afek, Ifat (Nokia - IL) <
>> ifat.afek at nokia.com <javascript:;>> wrote:
>>
>> Hi,
>>
>>
>>
>> Try to edit /etc/heat/policy.json:
>>
>>
>>
>> "stacks:global_index": "rule:deny_stack_user",
>>
>>
>>
>> Then restart vitrage-graph.
>>
>>
>>
>> Let me know if it solves the problem,
>>
>> Ifat.
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Wednesday, 8 March 2017 at 13:36
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *Re: OpenStack Vitrage
>>
>>
>>
>> Dear Ifat,
>>
>> That's fine :).
>>
>> Since, Vitrage project is already in mature phase and I don't have prior
>> experience of working on the collaboration projects, so, that's why it's
>> a
>> little bit time taking to understand the stuff.
>>
>> I attached the images for the topology (where square boxes are physical
>> servers, oval shapes are virtual switches inside the box and on top we
> have
>> vm's) and errors I am encountering.
>>
>>
>>
>> On Wed, Mar 8, 2017 at 6:20 PM, Afek, Ifat (Nokia - IL) <
>> ifat.afek at nokia.com <javascript:;>> wrote:
>>
>> Hi,
>>
>>
>>
>> Sorry for not answering so far.
>>
>>
>>
>> Can you please send the pictures as attachments? It is very hard to read
>> them this way (both the topology diagram and the errors you had).
>>
>> Regarding the Pike design sessions – I’m glad that you joined J we have
>> discussed it in the last several IRC meetings, but only now I realize
>> that
>> maybe you were not aware of it. Sorry for that.
>>
>>
>>
>> Best Regards,
>>
>> Ifat.
>>
>>
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Friday, 3 March 2017 at 04:31
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *Re: OpenStack Vitrage
>>
>>
>>
>> Hi Ifat,
>>
>> First of all, Regarding the virtual switches, I am not aware about any
>> OpenStack project at the moment but we can add new Data Source by using
> the
>> ovs tools like (ovs-vsctl) and ovs logs. Below you can see some of our
> work
>> (using ovs-tools) where we show how (also states) virtual switches are
>> connected to physical box and OpenStack Instances. For simplicity, we
>> have
>> fixed this topology.
>>
>> [image: ne image 4]
>>
>>
>>
>> *Vitrage Installation *
>> I installed Vitrage using Devstack. However, I am experiencing some
>> problems when querying the Vitrage either through cli or Horizon.
>>
>> I am using OpenStack Newton Release (Stable branch) Single Machine Setup
>> on Ubuntu 16.04 (Fresh Install).
>>
>> This is the log from Vitrage-graph
>> [image: ne image 2]
>>
>>
>>
>> This is the log from the Horizon
>>
>> [image: ne image 1]
>>
>> View From Horizon
>>
>> [image: ne image 3]
>>
>> Here is the policy.json
>> {
>> "get topology": "",
>> "get topology:all_tenants": "role:admin",
>> "get resource": "",
>> "list resources": "",
>> "list alarms": "",
>> "list alarms:all_tenants": "role:admin",
>> "get rca": "",
>> "get rca:all_tenants": "role:admin",
>> "template validate": "",
>> "template list": "",
>> "template show": "",
>> "event post": ""
>> }
>>
>> Here is the vitrage.conf
>> [DEFAULT]
>> debug = True
>> transport_url = rabbit://stackrabbit:secret@x.x.x.x:5672/
>>
>> [oslo_policy]
>> policy_file = /etc/vitrage/policy.json
>>
>> [service_credentials]
>> auth_url = http://x.x.x.x/identity
>> region_name = RegionOne
>> project_name = admin
>> password = secret
>> project_domain_id = default
>> user_domain_id = default
>> username = vitrage
>> auth_type = password
>>
>> [datasources]
>> types = nova.host,nova.instance,nova.zone,static_physical,cinder.
>> volume,neutron.network,neutron.port,heat.stack,doctor
>>
>> [keystone_authtoken]
>> memcached_servers = x.x.x.x:11211
>> signing_dir = /var/cache/vitrage
>> cafile = /opt/stack/data/ca-bundle.pem
>> auth_uri = http://x.x.x.x/identity
>> project_domain_name = Default
>> project_name = admin
>> user_domain_name = Default
>> password = secret
>> username = vitrage
>> auth_url = http://x.x.x.x/identity_v2_admin
>> auth_type = password
>>
>> [api]
>> pecan_debug = False
>>
>> [entity_graph]
>> datasources_values_dir = /etc/vitrage/datasources_values/
>>
>> [static_physical]
>> directory = /etc/vitrage/static_datasources/
>> changes_interval = 30
>> entities = switch
>>
>>
>>
>>
>>
>> On Wed, Mar 1, 2017 at 2:21 AM, Afek, Ifat (Nokia - IL) <
>> ifat.afek at nokia.com <javascript:;>> wrote:
>>
>> Hi,
>>
>>
>>
>> Sorry for the late response, I was on vacation last week.
>>
>>
>>
>> Vitrage visualization is implemented as an Horizon plugin, so it is
>> written in “the Horizon way”. I think that our visualization issues are
>> mainly around UX – when the entity graph becomes crowded, it is hard for
>> the user to get meaningful information out of it. You can find some of
>> our
>> thoughts in the etherpad of the design session we had in the OpenStack
>> Barcelona summit (look for the fishbowl session):
>> https://etherpad.openstack.org/p/vitrage-barcelona-design-summit
>>
>>
>>
>> Did you already install Vitrage? If so, I think it would be best if you
>> take a look at the UI by yourself and think what you would like to
> improve.
>>
>>
>>
>> Regarding the virtual switches – do you know of an OpenStack project (or
>> another project) that provides this information?
>>
>>
>>
>> Thanks,
>>
>> Ifat.
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Sunday, 26 February 2017 at 15:32
>>
>>
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *Re: OpenStack Vitrage
>>
>>
>>
>> Dear Ifat,
>>
>> I am really sorry for delayed reply because I had to wind-up couple of
>> urgent projects before getting involved in Vitrage project. However,
>> meanwhile I tried to go through Vitrage documentations.
>>
>> Sure, I would like to contribute in visualization part of Vitrage
>> project.
>> Can you let me know what are your current problems in visualization and
>> what tools you are using. Since, I utilized JavaScript libraries for
>> visualization of our Physical-Virtual resources topology visualization.
>>
>> Furthermore, how about adding virtual switches (assisted by OpenvSwitch)
>> between physical hosts and OpenStack instances because I don't see any
>> option in the mentioned Use-cases?
>>
>>
>>
>>
>>
>> On Mon, Jan 16, 2017 at 9:32 PM, Afek, Ifat (Nokia - IL) <
>> ifat.afek at nokia.com <javascript:;>> wrote:
>>
>> Dear Muhammad,
>>
>>
>>
>> Sounds great. If you are interested in the visualization area, we have a
>> lot of challenges in Vitrage (displaying a very big topology graph in a
> way
>> that is usable is not trivial).
>>
>> I will be happy to hear from you, and if you have any questions don’t
>> hesitate to ask.
>>
>>
>>
>> Best Regards,
>>
>> Ifat.
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Monday, 16 January 2017 at 13:41
>>
>>
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *Re: OpenStack Vitrage
>>
>>
>>
>> Dear Ifat,
>>
>> We are covering some aspect of both visibility collection (monitoring
>> data
>> collection) using Open-source Monitoring Tools (e.g. Intel
>> Snap-telemetry)
>> and it's transformation to specific Visualizations (UI) according to our
>> infrastructure requirements (e.g. topological view).
>>
>> Regarding, the links you send to us, we already created the launchpad and
>> gerrit accounts, also, subscribed to OpenStack-dev mailing list.
>>
>> Currently, we are going through the Vitrage official documentation.
>> Meanwhile, we will also try to install Vitrage in our testbed setup.
>>
>> We will keep in touch with you to discuss further about our
>> ideas/suggestions to Vitrage project.
>>
>>
>>
>> On Fri, Jan 13, 2017 at 7:10 AM, Afek, Ifat (Nokia - IL) <
>> ifat.afek at nokia.com <javascript:;>> wrote:
>>
>> Dear Muhammad,
>>
>>
>>
>> By visibility, do you mean UI? Or do you focus on monitoring?
>>
>>
>>
>> Vitrage is a stable project that is now being used in production as part
>> of Nokia products. However, while the base engine works well, there is a
>> lot more to do in order to improve it: add more datasources (either for
>> resources or new monitors), improve the templates language, identify new
>> RCA use cases, and improve the UX. You can find here some ideas that we
> put
>> on our roadmap:
>>
>> https://wiki.openstack.org/wiki/Vitrage/RoadMap
>>
>>
>>
>> The list of tasks we are currently working on, as well as future ones can
>> be found in launchpad:
>>
>> https://blueprints.launchpad.net/vitrage
>>
>>
>>
>> For the non-trivial blueprints we write an additional information in
>> github. The following project holds the blueprints:
>>
>> https://github.com/openstack/vitrage-specs
>>
>>
>>
>> After being pushed to github, the blueprints are “compiled” and you can
>> read the friendlier version in:
>>
>> http://specs.openstack.org/openstack/vitrage-specs/
>>
>>
>>
>> Vitrage code exists in three projects:
>>
>> · https://github.com/openstack/vitrage - the main code
>> (datasources, evaluator, notifiers)
>>
>> · https://github.com/openstack/python-vitrageclient - api and cli
>>
>> · https://github.com/openstack/vitrage-dashboard - Horizon plugin
>> for vitrage
>>
>>
>>
>> Vitrage bugs are managed here:
>>
>> https://bugs.launchpad.net/vitrage
>>
>>
>>
>> So how do you start?
>>
>> First, you should get familiar with OpenStack development process. Try to
>> create a devstack, register to launchpad and gerrit, and also to
>> OpenStack-dev mailing list.
>>
>> Then, you can either suggest new blueprints (of features that you feel
>> are
>> missing or should be enhanced) or start by fixing bugs.
>>
>>
>>
>> And of course, try to install and configure Vitrage. Understand how it
>> works, and if you encounter a problem – suggest a solution and/or open a
>> new bug. That’s a great way of starting.
>>
>>
>>
>> Instructions on how to install Vitrage on top of devstack:
>>
>> http://docs.openstack.org/developer/vitrage/installation-and-
>> configuration.html
>>
>>
>>
>> And in general, Vitrage official documentation is here:
>> http://docs.openstack.org/developer/vitrage/
>>
>>
>>
>> I hope I’m in the right direction. If not, or if you encounter any
>> problem, don’t hesitate to ask me. You may also submit questions in
>> openstack-dev mailing list with [vitrage] tag.
>>
>>
>>
>> Best Regards,
>>
>> Ifat.
>>
>>
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Wednesday, 11 January 2017 at 11:57
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *Re: OpenStack Vitrage
>>
>>
>>
>> Dear Ifat,
>>
>> Thanks for your kind reply.
>>
>> We are familiar with OpenStack project in general and also operating
>> multiple versions of OpenStack cloud in our lab. However, we don't have
> any
>> development experience to OpenStack or other Open Source projects.
>>
>> Overall, we both are working on visibility and monitoring Framework. It
>> mainly involves integration and development of visibility and
> visualization
>> tools for SDN and Cloud-based environments.
>>
>> When I said your assistance that means how we can start contributing to
>> Vitrage project in design, development, or any other starter capacity?
>> Especially, Visualization, RCA and extensions in Vitrage project
>> regrading
>> OVS or visibility sources.
>>
>>
>>
>> On Tue, Jan 10, 2017 at 9:28 PM, Afek, Ifat (Nokia - IL) <
>> ifat.afek at nokia.com <javascript:;>> wrote:
>>
>> Dear Muhammad,
>>
>>
>>
>> Thanks for your email, and for your interest in Vitrage. You are more
>> than
>> welcome to join our effort :-)
>>
>>
>>
>> In order for me to better understand your needs, can you please tell me
>> what your plans are regarding Vitrage? I’ll be happy to get a quick
>> overview of what you are doing, where do you see Vitrage fit in, and what
>> kind of contribution you would like to do.
>>
>>
>>
>> When you talk about my assistance, what kind of assistance are you
>> looking
>> for? Are you familiar with OpenStack in general? If not, here are two
> links
>> that might help:
>>
>> https://wiki.openstack.org/wiki/Getting_Started
>>
>> https://wiki.openstack.org/wiki/How_To_Contribute#How_can_I_help
>>
>>
>>
>> And information about Vitrage:
>>
>> https://wiki.openstack.org/wiki/Vitrage
>>
>>
>>
>> If you have other questions, please don’t hesitate to ask.
>>
>>
>>
>> Best Regards,
>>
>> Ifat.
>>
>>
>>
>>
>>
>> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
>> *Date: *Tuesday, 10 January 2017 at 13:26
>> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com <javascript:;>>
>> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
>> *Subject: *OpenStack Vitrage
>>
>>
>>
>> Dear Ifat Afek,
>>
>> We are two PhD students from NetCS lab (https://nm.gist.ac.kr/). We are
>> interested to contribute to OpenStack Vitrage project.
>>
>> However, we don't have any prior experience in working on Open Source
>> projects and their workflows.
>>
>> So, in this regard, we require your assistance to how to start
>> contributing to OpenStack Vitrage project.
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>>
>>
>>
>> --
>>
>> Best Regards,
>> Usman, Muhammad
>> ****************************************************
>> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
>> School of Information and Communications
>> GIST (Gwangju Inst. of Sci. and Tech.)
>> E-mail: usman at smartx.kr <javascript:;>
>> Office: +82-62-715-2273
>>
>
>
>
> --
> Best Regards,
> Usman, Muhammad
> ****************************************************
> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> School of Information and Communications
> GIST (Gwangju Inst. of Sci. and Tech.)
> E-mail: usman at smartx.kr <javascript:;>
> Office: +82-62-715-2273
>
>
>
> --
> Best Regards,
> Usman, Muhammad
> ****************************************************
> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> School of Information and Communications
> GIST (Gwangju Inst. of Sci. and Tech.)
> E-mail: usman at smartx.kr
> Office: +82-62-715-2273
>
--
--
*
Regards*
*
Muhammad Usman*
Application Engineer
LMK Resources (pvt) Limited
www.lmkr.com
+92 (323) 5599 068*
*
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list