Open Stack

One more thing and a typo in my mail  -  we would be happy to point folks
to an very early version (in Django) we prototyped and opened up recently.

thx
debo



On Fri, May 22, 2015 at 8:48 AM, Debojyoti Dutta <ddutta at gmail.com> wrote:

> Hi Sergey
>
> Thanks a lot for your interest. The bare bones API proposal is up on the
> wiki page http://wiki.openstack.org/Cognitive
>
> Sahara is about deploying and managing big data workloads like hadoop,
> spark etc. Cognitive is about a simple API to do predictive analytics,
> learning, data science workflow etc etc. Thus the goals are different. AWS
> has Elastic MapReduce (in the same space as Sahara) and also AWS Machine
> Learning (http://aws.amazon.com/machine-learning/) for which there is no
> parallel. Should we point them to our internal 1st version of Cognitive on
> our github which we opened.
>
> If it requires to use big data toolchains to do our job, we will
> definitely leverage Sahara for that, and not replicate the good work done
> in Sahara. Our primary goal is to build (within the community) a simple
> machine learning API (and a service) that abstracts the pain of data
> science for the app developer.
>
>
> thx
>
> debo
>
>
> PS: FWIW  I am at the summit till tonight so we could catch up here.
>
>
>
> On Tue, May 19, 2015 at 2:16 PM, Sergey Lukjanov <slukjanov at mirantis.com>
> wrote:
>
>> Hi,
>>
>> as there is no any details on the project yet done, if this project will
>> deploy ML frameworks it'll be direct duplication of Sahara's functionality
>> (we already support HDP and CDH deployments and they are provided tons of
>> tools for ML). So, I think that it could be built on top of Sahara or even
>> as part of Sahara probably. I'd like to propose you to take a deeper look
>> on Sahara and avoid duplicating it.
>>
>> Thanks.
>>
>> On Thu, May 14, 2015 at 8:47 PM, Debojyoti Dutta <ddutta at gmail.com>
>> wrote:
>>
>>> Hi Salvatore
>>>
>>> Thanks a lot for your comments.
>>>
>>> Timing: Yes it is time to do this! The nature of applications running on
>>> clouds is indeed changing.
>>>
>>> Initial group: We asked around for folks interested and we got a lot
>>> more people than we expected. The idea is to get something out there in a
>>> stack forge project and build something good. This group already has people
>>> who have built things like this already in the past. Hence confident about
>>> the success.
>>>
>>> Participation: We want this to be inclusive from scratch independent of
>>> who is a PTL or a contributor or merely a curious individual to give us
>>> ideas :) The community will get it right. Maybe I should have clarified
>>> that these are the members interested in seeing this happen.
>>>
>>> Wiki page: The wiki page will be ready in 1-2 days. Also we would like
>>> to have a discussion during the summit to see what we should build in the
>>> community. Would be delighted to get your thoughts.
>>>
>>> Services: Some of the services this could provide:
>>> * create experiments: define data sources, train models, then perform
>>> classification, clustering, data cleaning etc.
>>> * have experiment templates that can be reused
>>> * have an editor (maybe a horizon plugin) to drag and drop the workflow
>>> and generate an API that when called from an app would provide results
>>> * ML primitives that could be targeted initially: 1) classification  2)
>>> clustering 3) Anomaly detection
>>>
>>> thx
>>> debo
>>>
>>> On Thu, May 14, 2015 at 5:02 PM, Salvatore Orlando <sorlando at nicira.com>
>>> wrote:
>>>
>>>>
>>>> On 15 May 2015 at 00:19, Debojyoti Dutta <ddutta at gmail.com> wrote:
>>>>
>>>>> Hi!
>>>>>
>>>>> It is a great pleasure to announce the development of a new project
>>>>> called Cognitive.  Cognitive provides Machine Learning [1] as a Service
>>>>> that enables operators to offer next generation data science based services
>>>>> on top of their OpenStack Clouds.
>>>>>
>>>>
>>>> I was indeed wondering when "Machine Learning as a Service" would come
>>>> up...
>>>>
>>>>
>>>>> This project will begin as a StackForge project baed upon an empty
>>>>> cookiecutter [2] repo.  The repos to work in are:
>>>>> Server: https://github.com/stackforge/cognitive
>>>>> Client: https://github.com/stackforge/python-cognitiveclient
>>>>>
>>>>> Please join us via iRC on #openstack-cognitive on freenode.
>>>>>
>>>>> We will be holding a doodle poll to select times for our first meeting
>>>>> the week after summit.  This doodle poll will close May 24th and meeting
>>>>> times will be announced on the mailing list at that time.  At our first IRC
>>>>> meeting, we will draft additional core team members. We would like to
>>>>> invite interested individuals to join this exciting new development effort!
>>>>>
>>>>
>>>> From my little experience, "drafting" core members before even actually
>>>> having a code base has drawbacks. Also, it seems the initial starting team
>>>> is already large enough for ensuring support for 1 or 2 release cycle.
>>>>
>>>>
>>>>>
>>>>>
>>>>
>>>>> Please commit your schedule in the doodle poll here:
>>>>> http://doodle.com/drrka5tgbwpbfbxy
>>>>>
>>>>> Initial core team: Steven Dake, Aparupa Das Gupa, Debo~ Dutta, Johnu
>>>>> George,  Kyle Mestery, Sarvesh Ranjan, Ralf Rantzau, Komei Shimamura, Marc
>>>>> Solanas, Manoj Sharma, Yathi Udupi, Kai Zhang.
>>>>>
>>>>
>>>> Hey! What's the Neutron PTL doing there? Sorry we need his reviews we
>>>> can't loan it to you!
>>>>
>>>>
>>>>>
>>>>> A little bit about Cognitive:
>>>>> Data driven applications on cloud infrastructure increasingly rely on
>>>>> Machine Learning. Most data driven applications today use Machine Learning
>>>>> (ML). This often requires application developers and data scientists to
>>>>> write their own machine learning stack or deploy other packages to do any
>>>>> kind of data science based applications. Data scientists also need to have
>>>>> an easy way to rapidly experiment with data without having to write basic
>>>>> infrastructure for data manipulations. Cognitive is a Machine Learning
>>>>> service on top of OpenStack and provides machine learning based services to
>>>>> tenants (API, workbench, compute service).
>>>>>
>>>>
>>>> I wonder what kind of services you would offer; also you could have
>>>> shared something about the architecture of this service. Is it providing a
>>>> full machine learning stack, or just facilitating the use of existing one?
>>>>
>>>> But I see that there's a link to a wiki page below. This might have all
>>>> the answers.
>>>>
>>>>
>>>>>
>>>>>
>>>>> For information about blueprints check out:
>>>>> https://blueprints.launchpad.net/cognitive
>>>>> https://blueprints.launchpad.net/python-cognitiveclient
>>>>>
>>>>> For more details, check out our Wiki:
>>>>> https://wiki.openstack.org/wiki/Cognitive
>>>>>
>>>>
>>>> ... and unfortunately the wiki is empty ;)
>>>>
>>>>
>>>>>
>>>>> Please join the awesome Cognitive team in designing a world class
>>>>> Machine Learning as a Service solution.
>>>>>
>>>>> We look forward to seeing you on IRC on #openstack-cognitive.
>>>>>
>>>>> Regards,
>>>>> Debo~ Dutta (on behalf of the initial team)
>>>>>
>>>>> [1] http://en.wikipedia.org/wiki/Machine_learning
>>>>> [2] https://github.com/openstack-dev/cookiecutter
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> -Debo~
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Sincerely yours,
>> Sergey Lukjanov
>> Sahara Technical Lead
>> (OpenStack Data Processing)
>> Principal Software Engineer
>> Mirantis Inc.
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> -Debo~
>



-- 
-Debo~
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150522/65c80ab3/attachment.html>