[openstack-dev] [vitrage] Extending Topology
Muhammad Usman
usman476 at gmail.com
Thu Sep 21 01:59:13 UTC 2017
Dear Ifat,
Usman is here. Previously, I contacted you for contributing to OpenStack
Vitrage project but could not follow up with you for sometime due to
various reasons.
However, to get actively involved in OpenStack project, I have decided to
join OpenStack Summit in Sydney.
Also, based on my previous experience withe Vitrage project that is already
in shape, so its not easy to propose new ideas.
Therefore, a better way to start contributing to development side with
already proposed and on-going development.
--
*Regards*
*Muhammad Usman*
Application Engineer
LMK Resources (pvt) Limited
www.lmkr.com
+92 (323) 5599 068
On Mon, Mar 27, 2017 at 7:32 PM, Afek, Ifat (Nokia - IL/Kfar Sava) <
ifat.afek at nokia.com> wrote:
> Hi,
>
> Let me try and explain the more general use case.
>
> You can query OVS for the switches information, and understand how they
> are mapped to one another. This is not enough for knowing the exact route
> of the network traffic for a certain VM.
>
> A certain switch can be connected to more than one other switch. You can,
> as you said, query the network type (encapsulation) information from
> Neutron. But then you will need in addition to query the rules of the
> specific switch from OVS, in order to know which route to take for each
> encapsulation type.
>
> Another problematic use case is when the switches are not connected to
> each other. The traffic can be redirected by a network-stack software
> component, so you will have to query it in addition in order to tell the
> route.
>
> And on top of all this, we need to think how to best represent this
> information in Vitrage (i.e. how to draw the graph, which vertices to
> connect to one another, etc.).
>
> IMO, this is all feasible and will give a lot of value to Vitrage. Just
> not easy to implement.
>
> Best Regards,
> Ifat.
>
>
> On 22/03/2017, 08:50, "Muhammad Usman" <usman476 at gmail.com> wrote:
>
> Hello Ifat,
>
> I tried to see more in depth about the issues you mentioned regarding
> the extension of vSwitches. Due to a lot of complexity involved in
> generating this topology and associated effects, I believe we need to
> setup some baseline (e.g. adding a configuration file for specifying
> bridges in existing deployment setup). Then using that baseline,
> topology can be constructed as well as type of network can be
> extracted from neutron and associated path followed (e.g. vlan or
> vxlan). However, more general case you mentioned, I cannot get it. Do
> you mean nova-network?
>
> Regarding the sunburst representation - Yes I agree, if you want to
> keep compute hierarchy separate then addition of networking components
> is not a good idea.
>
> Also, suggestions from other vitrage members are welcomed.
>
>
> > On Thu, Mar 16, 2017 at 6:44 PM, Afek, Ifat (Nokia - IL) <
> > ifat.afek at nokia.com <javascript:;>> wrote:
> >
> >> Hi,
> >>
> >>
> >>
> >> Adding switches to the Vitrage topology is generally a good idea,
> but the
> >> implementation might be very complex. Your diagram shows a simple
> use
> > case,
> >> where all switches are linked to one another and it is easy to
> determine
> >> the effect of a failure on the vms. However, in the more general
> case
> > there
> >> might be switches without a connecting port (e.g. they might be
> connected
> >> via the network stack). In such cases, it is not clear how to model
> the
> >> switches topology in Vitrage. Another case to consider is when the
> >> network
> >> type affects the packets path, like vlan vs. vxlan. If you have an
> idea
> >> of
> >> how to solve these issues, I will be happy to hear it.
> >>
> >>
> >>
> >> Regarding the sunburst representation – I’m not sure I understand
> your
> >> diagram. Currently the sunburst is meant to show (only) the compute
> >> hierarchy: zones, hosts and instances. It is arranged in a
> containment
> >> relationship, i.e. every instance on the outer ring appears next to
> its
> >> host in the inner ring. If you add the bridges in the middle, you
> lose
> > this
> >> containment relationship. Can you please explain to me the suggested
> >> diagram?
> >>
> >>
> >>
> >> BTW, you can send such questions to OpenStack mailing list (
> >> openstack-dev at lists.openstack.org <javascript:;>) with [vitrage]
> tag in
> > the title, and
> >> possibly get replies from other contributors as well.
> >>
> >>
> >>
> >> Best Regards,
> >>
> >> Ifat.
> >>
> >>
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Monday, 13 March 2017 at 09:16
> >>
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *Re: OpenStack Vitrage
> >>
> >>
> >>
> >> Hi Ifat,
> >>
> >> I attached our idea of extending the Vitrage Topology to include
> Virtual
> >> switches.
> >>
> >> The reason, I mentioned about adding switches part in Vitrage is
> because
> >> we experienced looping issues that effect all infrastructure
> resources
> >> (i.e. physical host as well as vm's) performance. Therefore, it's
> > important
> >> to monitor the virtual switches as well to assist overall
> monitoring/RCA
> >> tasks.
> >>
> >> I think this idea will extend the Vitrage scope to touch some
> portion of
> >> SDN (e.g. if we consider the SDN managed virtual switches) as well.
> >>
> >>
> >>
> >> On Thu, Mar 9, 2017 at 6:49 PM, Muhammad Usman <usman at smartx.kr
> > <javascript:;>> wrote:
> >>
> >> Dear Ifat,
> >>
> >> Thanks for your guidance, I managed to install Vitrage properly
> using
> >> Master branches for both OpenStack and Vitrage.
> >>
> >> Now, I will look into the visualization as well as other aspects.
> >>
> >>
> >>
> >>
> >>
> >> On Thu, Mar 9, 2017 at 2:43 PM, Afek, Ifat (Nokia - IL) <
> >> ifat.afek at nokia.com <javascript:;>> wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> I have also noticed this problem, that Vitrage Ocata is not
> compatible
> >> with Horizon Newton.
> >>
> >> If you just want an OpenStack working, you should use a stable
> version.
> >> Stable/Ocata is the newest one (just released a few weeks ago). On
> the
> >> other hand, if you want to contribute code, you better take the
> master
> >> branch. Alternatively, you can take stable/ocata for all projects,
> and
> >> the
> >> master for Vitrage. This should work (for now, since Pike has just
> > started).
> >>
> >>
> >>
> >> Best Regards,
> >>
> >> Ifat.
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Wednesday, 8 March 2017 at 15:21
> >>
> >>
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *Re: OpenStack Vitrage
> >>
> >>
> >>
> >> Ifat,
> >>
> >> after adding the mentioned line in /etc/heat/policy.json first
> error "You
> >> are not authorized to use global_index" seems to be solved.
> >>
> >> However, in Horizon I still see same error (file is attached).
> >>
> >> So, After looking inside the code I found that I installed OpenStack
> >> using
> >> stable/newton branch but Vitrage is installed from Master branch.
> Since,
> >> there are few changes in code (python-vitrageclient/
> > vitrageclient/client.py)
> >> that's why I think this error is occurring. Therefore, I will try
> to fix
> >> this problem by using same branch (i.e. stable/newton).
> >>
> >>
> >>
> >> On Wed, Mar 8, 2017 at 9:08 PM, Afek, Ifat (Nokia - IL) <
> >> ifat.afek at nokia.com <javascript:;>> wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> Try to edit /etc/heat/policy.json:
> >>
> >>
> >>
> >> "stacks:global_index": "rule:deny_stack_user",
> >>
> >>
> >>
> >> Then restart vitrage-graph.
> >>
> >>
> >>
> >> Let me know if it solves the problem,
> >>
> >> Ifat.
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Wednesday, 8 March 2017 at 13:36
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *Re: OpenStack Vitrage
> >>
> >>
> >>
> >> Dear Ifat,
> >>
> >> That's fine :).
> >>
> >> Since, Vitrage project is already in mature phase and I don't have
> prior
> >> experience of working on the collaboration projects, so, that's why
> it's
> >> a
> >> little bit time taking to understand the stuff.
> >>
> >> I attached the images for the topology (where square boxes are
> physical
> >> servers, oval shapes are virtual switches inside the box and on top
> we
> > have
> >> vm's) and errors I am encountering.
> >>
> >>
> >>
> >> On Wed, Mar 8, 2017 at 6:20 PM, Afek, Ifat (Nokia - IL) <
> >> ifat.afek at nokia.com <javascript:;>> wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> Sorry for not answering so far.
> >>
> >>
> >>
> >> Can you please send the pictures as attachments? It is very hard to
> read
> >> them this way (both the topology diagram and the errors you had).
> >>
> >> Regarding the Pike design sessions – I’m glad that you joined J we
> have
> >> discussed it in the last several IRC meetings, but only now I
> realize
> >> that
> >> maybe you were not aware of it. Sorry for that.
> >>
> >>
> >>
> >> Best Regards,
> >>
> >> Ifat.
> >>
> >>
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Friday, 3 March 2017 at 04:31
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *Re: OpenStack Vitrage
> >>
> >>
> >>
> >> Hi Ifat,
> >>
> >> First of all, Regarding the virtual switches, I am not aware about
> any
> >> OpenStack project at the moment but we can add new Data Source by
> using
> > the
> >> ovs tools like (ovs-vsctl) and ovs logs. Below you can see some of
> our
> > work
> >> (using ovs-tools) where we show how (also states) virtual switches
> are
> >> connected to physical box and OpenStack Instances. For simplicity,
> we
> >> have
> >> fixed this topology.
> >>
> >> [image: ne image 4]
> >>
> >>
> >>
> >> *Vitrage Installation *
> >> I installed Vitrage using Devstack. However, I am experiencing some
> >> problems when querying the Vitrage either through cli or Horizon.
> >>
> >> I am using OpenStack Newton Release (Stable branch) Single Machine
> Setup
> >> on Ubuntu 16.04 (Fresh Install).
> >>
> >> This is the log from Vitrage-graph
> >> [image: ne image 2]
> >>
> >>
> >>
> >> This is the log from the Horizon
> >>
> >> [image: ne image 1]
> >>
> >> View From Horizon
> >>
> >> [image: ne image 3]
> >>
> >> Here is the policy.json
> >> {
> >> "get topology": "",
> >> "get topology:all_tenants": "role:admin",
> >> "get resource": "",
> >> "list resources": "",
> >> "list alarms": "",
> >> "list alarms:all_tenants": "role:admin",
> >> "get rca": "",
> >> "get rca:all_tenants": "role:admin",
> >> "template validate": "",
> >> "template list": "",
> >> "template show": "",
> >> "event post": ""
> >> }
> >>
> >> Here is the vitrage.conf
> >> [DEFAULT]
> >> debug = True
> >> transport_url = rabbit://stackrabbit:secret@x.x.x.x:5672/
> >>
> >> [oslo_policy]
> >> policy_file = /etc/vitrage/policy.json
> >>
> >> [service_credentials]
> >> auth_url = http://x.x.x.x/identity
> >> region_name = RegionOne
> >> project_name = admin
> >> password = secret
> >> project_domain_id = default
> >> user_domain_id = default
> >> username = vitrage
> >> auth_type = password
> >>
> >> [datasources]
> >> types = nova.host,nova.instance,nova.zone,static_physical,cinder.
> >> volume,neutron.network,neutron.port,heat.stack,doctor
> >>
> >> [keystone_authtoken]
> >> memcached_servers = x.x.x.x:11211
> >> signing_dir = /var/cache/vitrage
> >> cafile = /opt/stack/data/ca-bundle.pem
> >> auth_uri = http://x.x.x.x/identity
> >> project_domain_name = Default
> >> project_name = admin
> >> user_domain_name = Default
> >> password = secret
> >> username = vitrage
> >> auth_url = http://x.x.x.x/identity_v2_admin
> >> auth_type = password
> >>
> >> [api]
> >> pecan_debug = False
> >>
> >> [entity_graph]
> >> datasources_values_dir = /etc/vitrage/datasources_values/
> >>
> >> [static_physical]
> >> directory = /etc/vitrage/static_datasources/
> >> changes_interval = 30
> >> entities = switch
> >>
> >>
> >>
> >>
> >>
> >> On Wed, Mar 1, 2017 at 2:21 AM, Afek, Ifat (Nokia - IL) <
> >> ifat.afek at nokia.com <javascript:;>> wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> Sorry for the late response, I was on vacation last week.
> >>
> >>
> >>
> >> Vitrage visualization is implemented as an Horizon plugin, so it is
> >> written in “the Horizon way”. I think that our visualization issues
> are
> >> mainly around UX – when the entity graph becomes crowded, it is
> hard for
> >> the user to get meaningful information out of it. You can find some
> of
> >> our
> >> thoughts in the etherpad of the design session we had in the
> OpenStack
> >> Barcelona summit (look for the fishbowl session):
> >> https://etherpad.openstack.org/p/vitrage-barcelona-design-summit
> >>
> >>
> >>
> >> Did you already install Vitrage? If so, I think it would be best if
> you
> >> take a look at the UI by yourself and think what you would like to
> > improve.
> >>
> >>
> >>
> >> Regarding the virtual switches – do you know of an OpenStack
> project (or
> >> another project) that provides this information?
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Ifat.
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Sunday, 26 February 2017 at 15:32
> >>
> >>
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *Re: OpenStack Vitrage
> >>
> >>
> >>
> >> Dear Ifat,
> >>
> >> I am really sorry for delayed reply because I had to wind-up
> couple of
> >> urgent projects before getting involved in Vitrage project. However,
> >> meanwhile I tried to go through Vitrage documentations.
> >>
> >> Sure, I would like to contribute in visualization part of Vitrage
> >> project.
> >> Can you let me know what are your current problems in visualization
> and
> >> what tools you are using. Since, I utilized JavaScript libraries for
> >> visualization of our Physical-Virtual resources topology
> visualization.
> >>
> >> Furthermore, how about adding virtual switches (assisted by
> OpenvSwitch)
> >> between physical hosts and OpenStack instances because I don't see
> any
> >> option in the mentioned Use-cases?
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Jan 16, 2017 at 9:32 PM, Afek, Ifat (Nokia - IL) <
> >> ifat.afek at nokia.com <javascript:;>> wrote:
> >>
> >> Dear Muhammad,
> >>
> >>
> >>
> >> Sounds great. If you are interested in the visualization area, we
> have a
> >> lot of challenges in Vitrage (displaying a very big topology graph
> in a
> > way
> >> that is usable is not trivial).
> >>
> >> I will be happy to hear from you, and if you have any questions
> don’t
> >> hesitate to ask.
> >>
> >>
> >>
> >> Best Regards,
> >>
> >> Ifat.
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Monday, 16 January 2017 at 13:41
> >>
> >>
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *Re: OpenStack Vitrage
> >>
> >>
> >>
> >> Dear Ifat,
> >>
> >> We are covering some aspect of both visibility collection
> (monitoring
> >> data
> >> collection) using Open-source Monitoring Tools (e.g. Intel
> >> Snap-telemetry)
> >> and it's transformation to specific Visualizations (UI) according
> to our
> >> infrastructure requirements (e.g. topological view).
> >>
> >> Regarding, the links you send to us, we already created the
> launchpad and
> >> gerrit accounts, also, subscribed to OpenStack-dev mailing list.
> >>
> >> Currently, we are going through the Vitrage official documentation.
> >> Meanwhile, we will also try to install Vitrage in our testbed setup.
> >>
> >> We will keep in touch with you to discuss further about our
> >> ideas/suggestions to Vitrage project.
> >>
> >>
> >>
> >> On Fri, Jan 13, 2017 at 7:10 AM, Afek, Ifat (Nokia - IL) <
> >> ifat.afek at nokia.com <javascript:;>> wrote:
> >>
> >> Dear Muhammad,
> >>
> >>
> >>
> >> By visibility, do you mean UI? Or do you focus on monitoring?
> >>
> >>
> >>
> >> Vitrage is a stable project that is now being used in production as
> part
> >> of Nokia products. However, while the base engine works well, there
> is a
> >> lot more to do in order to improve it: add more datasources (either
> for
> >> resources or new monitors), improve the templates language,
> identify new
> >> RCA use cases, and improve the UX. You can find here some ideas
> that we
> > put
> >> on our roadmap:
> >>
> >> https://wiki.openstack.org/wiki/Vitrage/RoadMap
> >>
> >>
> >>
> >> The list of tasks we are currently working on, as well as future
> ones can
> >> be found in launchpad:
> >>
> >> https://blueprints.launchpad.net/vitrage
> >>
> >>
> >>
> >> For the non-trivial blueprints we write an additional information in
> >> github. The following project holds the blueprints:
> >>
> >> https://github.com/openstack/vitrage-specs
> >>
> >>
> >>
> >> After being pushed to github, the blueprints are “compiled” and you
> can
> >> read the friendlier version in:
> >>
> >> http://specs.openstack.org/openstack/vitrage-specs/
> >>
> >>
> >>
> >> Vitrage code exists in three projects:
> >>
> >> · https://github.com/openstack/vitrage - the main code
> >> (datasources, evaluator, notifiers)
> >>
> >> · https://github.com/openstack/python-vitrageclient - api
> and cli
> >>
> >> · https://github.com/openstack/vitrage-dashboard - Horizon
> plugin
> >> for vitrage
> >>
> >>
> >>
> >> Vitrage bugs are managed here:
> >>
> >> https://bugs.launchpad.net/vitrage
> >>
> >>
> >>
> >> So how do you start?
> >>
> >> First, you should get familiar with OpenStack development process.
> Try to
> >> create a devstack, register to launchpad and gerrit, and also to
> >> OpenStack-dev mailing list.
> >>
> >> Then, you can either suggest new blueprints (of features that you
> feel
> >> are
> >> missing or should be enhanced) or start by fixing bugs.
> >>
> >>
> >>
> >> And of course, try to install and configure Vitrage. Understand how
> it
> >> works, and if you encounter a problem – suggest a solution and/or
> open a
> >> new bug. That’s a great way of starting.
> >>
> >>
> >>
> >> Instructions on how to install Vitrage on top of devstack:
> >>
> >> http://docs.openstack.org/developer/vitrage/installation-and-
> >> configuration.html
> >>
> >>
> >>
> >> And in general, Vitrage official documentation is here:
> >> http://docs.openstack.org/developer/vitrage/
> >>
> >>
> >>
> >> I hope I’m in the right direction. If not, or if you encounter any
> >> problem, don’t hesitate to ask me. You may also submit questions in
> >> openstack-dev mailing list with [vitrage] tag.
> >>
> >>
> >>
> >> Best Regards,
> >>
> >> Ifat.
> >>
> >>
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Wednesday, 11 January 2017 at 11:57
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *Re: OpenStack Vitrage
> >>
> >>
> >>
> >> Dear Ifat,
> >>
> >> Thanks for your kind reply.
> >>
> >> We are familiar with OpenStack project in general and also operating
> >> multiple versions of OpenStack cloud in our lab. However, we don't
> have
> > any
> >> development experience to OpenStack or other Open Source projects.
> >>
> >> Overall, we both are working on visibility and monitoring
> Framework. It
> >> mainly involves integration and development of visibility and
> > visualization
> >> tools for SDN and Cloud-based environments.
> >>
> >> When I said your assistance that means how we can start
> contributing to
> >> Vitrage project in design, development, or any other starter
> capacity?
> >> Especially, Visualization, RCA and extensions in Vitrage project
> >> regrading
> >> OVS or visibility sources.
> >>
> >>
> >>
> >> On Tue, Jan 10, 2017 at 9:28 PM, Afek, Ifat (Nokia - IL) <
> >> ifat.afek at nokia.com <javascript:;>> wrote:
> >>
> >> Dear Muhammad,
> >>
> >>
> >>
> >> Thanks for your email, and for your interest in Vitrage. You are
> more
> >> than
> >> welcome to join our effort :-)
> >>
> >>
> >>
> >> In order for me to better understand your needs, can you please
> tell me
> >> what your plans are regarding Vitrage? I’ll be happy to get a quick
> >> overview of what you are doing, where do you see Vitrage fit in,
> and what
> >> kind of contribution you would like to do.
> >>
> >>
> >>
> >> When you talk about my assistance, what kind of assistance are you
> >> looking
> >> for? Are you familiar with OpenStack in general? If not, here are
> two
> > links
> >> that might help:
> >>
> >> https://wiki.openstack.org/wiki/Getting_Started
> >>
> >> https://wiki.openstack.org/wiki/How_To_Contribute#How_can_I_help
> >>
> >>
> >>
> >> And information about Vitrage:
> >>
> >> https://wiki.openstack.org/wiki/Vitrage
> >>
> >>
> >>
> >> If you have other questions, please don’t hesitate to ask.
> >>
> >>
> >>
> >> Best Regards,
> >>
> >> Ifat.
> >>
> >>
> >>
> >>
> >>
> >> *From: *Muhammad Usman <usman at smartx.kr <javascript:;>>
> >> *Date: *Tuesday, 10 January 2017 at 13:26
> >> *To: *"Afek, Ifat (Nokia - IL)" <ifat.afek at nokia.com
> <javascript:;>>
> >> *Cc: *JungSu Han <jshan at smartx.kr <javascript:;>>
> >> *Subject: *OpenStack Vitrage
> >>
> >>
> >>
> >> Dear Ifat Afek,
> >>
> >> We are two PhD students from NetCS lab (https://nm.gist.ac.kr/).
> We are
> >> interested to contribute to OpenStack Vitrage project.
> >>
> >> However, we don't have any prior experience in working on Open
> Source
> >> projects and their workflows.
> >>
> >> So, in this regard, we require your assistance to how to start
> >> contributing to OpenStack Vitrage project.
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >>
> >>
> >>
> >> --
> >>
> >> Best Regards,
> >> Usman, Muhammad
> >> ****************************************************
> >> M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> >> School of Information and Communications
> >> GIST (Gwangju Inst. of Sci. and Tech.)
> >> E-mail: usman at smartx.kr <javascript:;>
> >> Office: +82-62-715-2273
> >>
> >
> >
> >
> > --
> > Best Regards,
> > Usman, Muhammad
> > ****************************************************
> > M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> > School of Information and Communications
> > GIST (Gwangju Inst. of Sci. and Tech.)
> > E-mail: usman at smartx.kr <javascript:;>
> > Office: +82-62-715-2273
> >
> >
> >
> > --
> > Best Regards,
> > Usman, Muhammad
> > ****************************************************
> > M.S./PhD Integrated candidate of Networked Computing Systems Lab.
> > School of Information and Communications
> > GIST (Gwangju Inst. of Sci. and Tech.)
> > E-mail: usman at smartx.kr
> > Office: +82-62-715-2273
> >
>
>
> --
> --
> *
> Regards*
> *
> Muhammad Usman*
> Application Engineer
> LMK Resources (pvt) Limited
> www.lmkr.com
> +92 (323) 5599 068*
> *
>
> ____________________________________________________________
> ______________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.op
> enstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170921/77647198/attachment-0001.html>
More information about the OpenStack-dev
mailing list