May 2016 - OpenStack-HPC - lists.openstack.org

Re: [openstack-hpc] What's the state of openstack-hpc now?
by Blair Bethwaite 06 Oct '16

06 Oct '16

Hi, Apologies for top-posting but I don't intend to answer all the historical project points you've raised. Regarding old things floating around on github, your mileage may vary, but I doubt at this point you want to be looking at any of that in great detail. You haven't really explained what you mean by or want from HPC in this context, so I'm guessing a little based on your other questions... OpenStack is many things to different people and organisations, but at the software core is a very flexible infrastructure provisioning framework. HPC requires infrastructure (compute, network, storage), and OpenStack can certainly deliver it - make your deployment choices to suit your use-cases. A major choice would be whether you will use full system virtualisation or bare-metal or containers or <insert next trend> - that choice largely depends on your typical workloads and what style of cluster you want. Beyond that, compared to "typical" cloud hardware - faster CPUs, faster memory, faster network (probably with much greater east-west capacity), integration of a suitable parallel file-system. However, OpenStack is not a HPC management / scheduling / queuing / middleware system - there are lots of those already and you should pick one that fits your requirements and then (if it helps) run it atop an OpenStack cloud (it might help, e.g., if you want to run multiple logical clusters on the same physical infrastructure, if you want to mix other more traditional cloud workloads in, if you're just doing everything with OpenStack like the other cool kids). There are lots of nuances here, e.g., where one scheduler might lend itself better to more dynamic infrastructure (adding/removing instances), another might be lighter-weight for use with a Cluster-as-a-Service deployment model, whilst another suits a multi-user managed service style cluster. I'm sure there is good experience and opinion hidden on this list if you want to interrogate those sorts of choices more specifically. Most of the relevant choices you need to make with respect to running HPC workloads on infrastructure that is provisioned through OpenStack will come down to your hypervisor choices. My preference for now is to stick with the OpenStack community's most popular free OS and hypervisor (Ubuntu and KVM+Libvirt) - when I facilitated the hypervisor-tuning ops session at the Vancouver summit (with a bunch of folks interested in HPC on OpenStack) there was no-one in the room running a different hypervisor, though several were using RHEL. With the right tuning KVM can get you to within a hair's breadth of bare-metal performance for a wide range of CPU, memory and inter-process comms benchmarks, plus you can easily make use of PCI passthrough for latency sensitive or "difficult" devices like NICs/HCAs and GPGPUs. And the "right tuning" is not really some arcane knowledge, it's mainly about exposing host CPU capabilities, pinning vCPUs to pCPUs, and tuning or pinning and exposing NUMA topology - most of this is supported directly through OpenStack-native features now. To answer the GPU question more explicitly - yes you can do this. Mainly you need to ensure you're getting compatible hardware (GPU and relevant motherboard components) - most of the typical GPGPU choices (e.g. K80, K40, M60) will work, and you should probably be wary of PCIe switches unless you know exactly what you're doing (recommend trying before buying). At the OpenStack level you just define the PCI devices you want OpenStack Nova to provision and you can then define custom instance-types/flavors that will get a GPU passed through. Similar things go for networking. Lastly, just because you can do this doesn't make it a good idea... OpenStack is complex, HPC systems are complex, layering one complicated thing on another is a good way to create tricky problems that hide in the interface between the two layers. So make sure you're gaining something from having OpenStack in the mix here. HTH, Blair On 15 March 2016 at 23:00, <openstack-hpc-request(a)lists.openstack.org> wrote: > Message: 1 > Date: Tue, 15 Mar 2016 19:05:38 +0800 > From: "me,apporc" <appleorchard2000(a)gmail.com> > To: openstack-hpc(a)lists.openstack.org > Subject: [openstack-hpc] What's the state of openstack-hpc now? > Message-ID: > <CAOBTi0sftGTG-fscM-C5wLu6bTgZMaLaM2eXBJpa0a=vkPDusg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi, all > > I found this etherpad[1] which was created long time ago, inside which > there are some blueprints: support-heterogeneous-archs[2], > heterogeneous-instance-types[3] > and schedule-instances-on-heterogeneous-architectures[4] . > But those blueprints had been obselete since year 2014, and some of its > patches were abandoned. > There however is a forked branch github[5] or launchpad[6], which is > diverged far away from nova/trunk, and not updated since 2014 too. > > Is that we just abandoned those blueprints in openstack or else? > > Besides, there is a CaaS[7] project called Senlin[8], which refered to the > word "HPC" in its wiki. But it seems like not really related. "Cluster" can > mean many things, but hpc is some kind different. > > I can not get the status of GPU support in nova. As the case of network, > SR-IOV[9] seems ok. For storage, i don't know what the word "mi2" means in > etherpad[1]. > > According to what i got above, it seems we can not use hpc in openstack > now. But there are some videos here[10], here[11] and here[12].Since we can > not get GPU in nova instance, are they just building traditional hpcs > without GPU? > > > I need more information, thanks in advance. > > 1. https://etherpad.openstack.org/p/HVHsTqOQGc > 2. https://blueprints.launchpad.net/nova/+spec/support-heterogeneous-archs > 3. https://blueprints.launchpad.net/nova/+spec/heterogeneous-instance-types > 4. > https://blueprints.launchpad.net/nova/+spec/schedule-instances-on-heterogen… > 5. https://github.com/usc-isi/nova > 6. https://code.launchpad.net/~usc-isi/nova/hpc-trunk > 7. https://wiki.openstack.org/wiki/CaaS > 8. https://wiki.openstack.org/wiki/Senlin > 9. https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking > 10. > https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/… > 11. > https://www.openstack.org/summit/tokyo-2015/videos/presentation/hpc-on-open… > 12. > https://www.openstack.org/summit/tokyo-2015/videos/presentation/canonical-h… > -- > Regards, > apporc >

4 7

FedCSIS 2016 - CALL FOR POSITION PAPERS
by Federated Conference on Computer Science and Information Systems 24 May '16

24 May '16

Call for Position Papers 2016 Federated Conference on Computer Science and Information Systems (FedCSIS) Gdansk, Poland, 11-14 September, 2016 www.fedcsis.org (FedCSIS on www.ieee.org: http://tinyurl.com/FedCSIS-on-IEEE-2016) The FedCSIS 2016 Federated Conference invites submissions of POSITION PAPERS to its respective events. Position papers must not exceed 8 pages and they should relate to an ongoing research or experience. Position papers will be presented by the authors alongside regular papers. Position papers may be also submitted as DEMO PAPERS and presented as demonstrations of software tools and products. They should describe non-for-profit software tools in a prototype-, alpha-, or beta-version. We invite TWO TYPES OF POSITION PAPERS: EMERGING RESEARCH PAPERS present preliminary research results from work-in-progress based on sound scientific approach but presenting work not completely validated as yet. They must describe precisely the research problem and its rationale. They must also define the intended future work including the expected benefits from solution to the tackled problem. CHALLENGE PAPERS propose and describe research challenges in theory or practice of computer science and information systems. The papers in this category must be based on deep understanding of existing research or industrial problems and should be defining new promising research directions. Paper publication: Position papers will be published as a SEPARATE ELECTRONIC VOLUME (of the Annals of Computer Science and Information Systems series; annals-csis.org) WITH an ISBN, ISSN and DOI numbers. These papers will NOT be placed in the IEEE Digital Library and will NOT be submitted to Thomson Reuters Web of Science. However, they will be indexed in the CrossRef, BazEkon, J-Gate, and submitted for indexing to: SCOPUS, Inspec, DBLP Computer Science Bibliography and Google Scholar. Paper submission: Authors should submit a position paper in English, carefully checked for correct grammar and spelling, using the on-line submission procedure. The guidelines for paper formatting provided at the conference website ought to be used for all submitted papers. The required submission format is the same as the camera-ready format. Please check, and carefully follow, the instructions and templates provided. Papers that are out of scope of the selected event, or that contain any form of (self)plagiarism will be rejected without reviews. All position papers will be refereed before inclusion in the conference program. About FedCSIS 2016: FedCSIS 2016 is organized by the Polish Information Processing Society (PTI) in technical cooperation with the IEEE Region 8, IEEE Poland Section, IEEE Poland (Gdansk) Section Computer Society Chapter, Poland Section Computational Intelligence Society Chapter, ACM Special Interest Group on Applied Computing, Lodz ACM Chapter, Committee of Computer Science of Polish Academy of Sciences, European Alliance for Innovation, Polish Operational and Systems Research Society, Eastern Cluster ICT and Mazovia Cluster ICT. Chairs of FedCSIS Conference Series Maria Ganzha, Leszek A. Maciaszek, Marcin Paprzycki Contact FedCSIS at secretariat(a)fedcsis.org

1 0

VECPAR 2016: Summer School in Advanced Scientific Computing
by Carlos Ferreira 21 May '16

21 May '16

The VECPAR 2016 organizing committee has the pleasure to announce that an associated summer school will be held in Braga during the week previous to the conference: Summer School in Advanced Scientific Computing, to be held from June 20 to June 23, in Braga (UMinho, Campus de Gualtar, Auditorium at School Engineering), in conjunction with VECPAR 2016. Summer School website: http://www.di.uminho.pt/SS-AdvSciComp16/ Hope to see you there! Best regards, The VECPAR 2016 organizing committee Carlos Ferreira ISEP | Instituto Superior de Engenharia do Porto Rua Dr. António Bernardino de Almeida, 431 4249-015 Porto - PORTUGAL tel. +351 228 340 500 | fax +351 228 321 159 mail(a)isep.ipp.pt | www.isep.ipp.pt

1 0

VECPAR 2016: Registration is open
by Carlos Ferreira 03 May '16

03 May '16

The registration link is available! Please check at: http://vecpar.fe.up.pt/2016 Regarding accommodation, you can reserve it by yourself choosing any of the suggested hotels or you can request accommodation in the registration form. Besides hosting VECPAR, Portugal has some special attractions in June: June 23rd: St. John's day celebration, Porto (https://en.wikipedia.org/wiki/Festa_de_S%C3%A3o_Jo%C3%A3o_do_Porto) June 23rd to June 26th: Medieval Fair, Guimarães (http://www.guimaraesturismo.com/pages/283?geo_article_id=3226) Hope to see you soon! The VECPAR 2016 Organizing Committee. Carlos Ferreira ISEP | Instituto Superior de Engenharia do Porto Rua Dr. António Bernardino de Almeida, 431 4249-015 Porto - PORTUGAL tel. +351 228 340 500 | fax +351 228 321 159 mail(a)isep.ipp.pt | www.isep.ipp.pt

1 0