Open Stack

> On 2018. Oct 31., at 19:11, Mike Bayer <mike_mp at zzzcomputing.com> wrote:
> 
> On Wed, Oct 31, 2018 at 10:57 AM Bogdan Dobrelya <bdobreli at redhat.com> wrote:
>> 
>> (cross-posting openstack-dev)
>> 
>> Hello.
>> [tl;dr] I'm looking for co-author(s) to come up with "Edge clouds data
>> consistency requirements and challenges" a position paper [0] (papers
>> submitting deadline is Nov 8).
>> 
>> The problem scope is synchronizing control plane and/or
>> deployments-specific data (not necessary limited to OpenStack) across
>> remote Edges and central Edge and management site(s). Including the same
>> aspects for overclouds and undercloud(s), in terms of TripleO; and other
>> deployment tools of your choice.
>> 
>> Another problem is to not go into different solutions for Edge
>> deployments management and control planes of edges. And for tenants as
>> well, if we think of tenants also doing Edge deployments based on Edge
>> Data Replication as a Service, say for Kubernetes/OpenShift on top of
>> OpenStack.
>> 
>> So the paper should name the outstanding problems, define data
>> consistency requirements and pose possible solutions for synchronization
>> and conflicts resolving. Having maximum autonomy cases supported for
>> isolated sites, with a capability to eventually catch up its distributed
>> state. Like global database [1], or something different perhaps (see
>> causal-real-time consistency model [2],[3]), or even using git. And
>> probably more than that?.. (looking for ideas)
> 
> 
> I can offer detail on whatever aspects of the "shared  / global
> database" idea.  The way we're doing it with Galera for now is all
> about something simple and modestly effective for the moment, but it
> doesn't have any of the hallmarks of a long-term, canonical solution,
> because Galera is not well suited towards being present on many
> (dozens) of endpoints.     The concept that the StarlingX folks were
> talking about, that of independent databases that are synchronized
> using some kind of middleware is potentially more scalable, however I
> think the best approach would be API-level replication, that is, you
> have a bunch of Keystone services and there is a process that is
> regularly accessing the APIs of these keystone services and
> cross-publishing state amongst all of them.   Clearly the big
> challenge with that is how to resolve conflicts, I think the answer
> would lie in the fact that the data being replicated would be of
> limited scope and potentially consist of mostly or fully
> non-overlapping records.
> 
> That is, I think "global database" is a cheap way to get what would be
> more effective as asynchronous state synchronization between identity
> services.

Recently we’ve been also exploring federation with an IdP (Identity Provider) master: https://wiki.openstack.org/wiki/Keystone_edge_architectures#Identity_Provider_.28IdP.29_Master_with_shadow_users

One of the pros is that it removes the need for synchronization and potentially increases scalability.

Thanks,
Ildikó


> 
>> 
>> See also the "check" list in-line, which I think also meets the data
>> consistency topics well - it would be always nice to have some
>> theoretical foundations at hand, when repairing some
>> 1000-edges-spread-off and fully broken global database, by hand :)
>> 
>> PS. I must admit I have yet any experience with those IEEE et al
>> academic things and looking for someone who has it, to team and
>> co-author that positioning paper by. That's as a start, then we can
>> think of presenting it and expanding into work items for OpenStack Edge
>> WG and future development plans.
>> 
>> [0] http://conferences.computer.org/ICFC/2019/Paper_Submission.html
>> [1] https://review.openstack.org/600555
>> [2] https://jepsen.io/consistency
>> [3] http://www.cs.cornell.edu/lorenzo/papers/cac-tr.pdf
>> 
>> On 10/22/18 3:44 PM, Flavia Delicato wrote:
>>> =================================================================================
>>> IEEE International Conference on Fog Computing (ICFC 2019)
>>> June 24-26, 2019
>>> Prague, Czech Republic
>>> http://conferences.computer.org/ICFC/2019/
>>> Co-located with the IEEE International Conference on Cloud Engineering
>>> (IC2E 2019)
>>> ==================================================================================
>>> 
>>> Important Dates
>>> ---------------
>>> Paper registration and abstract: Nov 1st, 2018
>>> Full paper submission due: Nov 8th, 2018
>>> Notification of paper acceptance: Jan. 20th, 2019
>>> Workshop and tutorial proposals due: Nov 11, 2018
>>> Notification of proposal acceptance: Nov 18, 2018
>>> 
>>> Call for Contributions
>>> ----------------------
>>> Fog computing is the extension of cloud computing into its edge and
>>> the physical world to meet the data volume and decision velocity
>>> requirements in many emerging applications, such as augmented and
>>> virtual realities (AR/VR), cyber-physical systems (CPS), intelligent
>>> and autonomous systems, and mission-critical systems. The boundary
>>> between centralized, powerful computing cloud and massively
>>> distributed, Internet connected sensors, actuators, and things is
>>> blurred in this new computing paradigm.
>>> 
>>> The ICFC 2019 technical program will feature tutorials, workshops, and
>>> research paper sessions. We solicit high-quality contributions in the
>>> above categories. Details of submission is available on the conference
>>> Web site. Topics of interest include but are not limited to:
>>> 
>>> * System architecture for fog computing
>> 
>> (check)
>> 
>>> * Coordination between cloud, fog, and sensing/actuation endpoints
>>> * Connectivity, storage, and computation in the edge
>>> * Data processing and management for fog computing
>> 
>> (check)
>> 
>>> * Efficient and embedded AI in the fog
>>> * System and network manageability
>>> * Middleware and coordination platforms
>>> * Power, energy, and resource management
>>> * Device and hardware support for fog computing
>>> * Programming models, abstractions, and software engineering for fog computing
>> 
>> (check)
>> 
>>> * Security, privacy, and ethics issues related to fog computing
>>> * Theoretical foundations and formal methods for fog computing systems
>> 
>> (check)
>> 
>>> * Applications and experiences
>>> 
>>> Organizing Committee
>>> --------------------
>>> General Chairs:
>>> Hui Lei, IBM
>>> Albert Zomaya, The University of Sydney
>>> 
>>> PC Co-chairs:
>>> Erol Gelenbe, Imperial College London
>>> Jie Liu, Microsoft Research
>>> 
>>> Tutorials and Workshops Chair:
>>> David Bermbach, TU Berlin
>>> 
>>> Publicity Co-chairs:
>>> Flavia Delicato,Federal University of Rio de Janeiro
>>> Mathias Fischer, University Hamburg
>>> 
>>> Publication Chair
>>> Javid Taheri, Karlstad University
>>> 
>>> Webmaster
>>> Wei Li, The University of Sydney
>>> 
>>> Steering Committee
>>> ------------------
>>> Mung Chiang, Purdue University
>>> Erol Gelenbe, Imperial College London
>>> Christos Kozarakis, Stanford University
>>> Hui Lei, IBM
>>> Chenyang Lu, Washington University in St Louis
>>> Beng Chin Ooi, National University of Singapore
>>> Neeraj Suri, TU Darmstadt
>>> Albert Zomaya, The University of Sydney
>>> 
>>> Program Committee
>>> ------------------
>>> 
>>> Tarek Abdelzaher, UIUC
>>> Anne Benoit, ENS Lyon
>>> David Bermbach, TU Berlin
>>> Bharat Bhargava, Purdue University
>>> Olivier Brun, LAAS/CNRS Laboratory
>>> Jiannong Cao, Hong Kong Polytech
>>> Flavia C. Delicato, UFRJ, Brazil
>>> Xiaotie Deng, Peking University, China
>>> Schahram Dustdar, TU Wien, Germany
>>> Maria Gorlatova, Duke University
>>> Dharanipragada Janakiram, IIT Madras
>>> Wenjing Luo, Virginia Tech
>>> Pedro José Marrón, Universität Duisburg-Essen
>>> Geyong Min, University of Exeter
>>> Suman Nath, Microsoft Research
>>> Vincenzo Piuri, Universita Degli Studi Di Milano
>>> Yong Meng Teo, National University of Singapore
>>> Guoliang Xing, Chinese University of Hong Kong
>>> Yuanyuan Yang, SUNY Stony Brook
>>> Xiaoyun Zhu, Cloudera
>>> 
>> 
>> 
>> --
>> Best regards,
>> Bogdan Dobrelya,
>> Irc #bogdando
> 
> _______________________________________________
> Edge-computing mailing list
> Edge-computing at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/edge-computing