Open Stack

So, in case if Cognitive will use Sahara to create big data clusters - then
it sounds ok.

On Fri, May 22, 2015 at 8:54 AM, Debojyoti Dutta <ddutta at gmail.com> wrote:

> One more thing and a typo in my mail  -  we would be happy to point folks
> to an very early version (in Django) we prototyped and opened up recently.
>
> thx
> debo
>
>
>
> On Fri, May 22, 2015 at 8:48 AM, Debojyoti Dutta <ddutta at gmail.com> wrote:
>
>> Hi Sergey
>>
>> Thanks a lot for your interest. The bare bones API proposal is up on the
>> wiki page http://wiki.openstack.org/Cognitive
>>
>> Sahara is about deploying and managing big data workloads like hadoop,
>> spark etc. Cognitive is about a simple API to do predictive analytics,
>> learning, data science workflow etc etc. Thus the goals are different. AWS
>> has Elastic MapReduce (in the same space as Sahara) and also AWS Machine
>> Learning (http://aws.amazon.com/machine-learning/) for which there is no
>> parallel. Should we point them to our internal 1st version of Cognitive on
>> our github which we opened.
>>
>> If it requires to use big data toolchains to do our job, we will
>> definitely leverage Sahara for that, and not replicate the good work done
>> in Sahara. Our primary goal is to build (within the community) a simple
>> machine learning API (and a service) that abstracts the pain of data
>> science for the app developer.
>>
>>
>> thx
>>
>> debo
>>
>>
>> PS: FWIW  I am at the summit till tonight so we could catch up here.
>>
>>
>>
>> On Tue, May 19, 2015 at 2:16 PM, Sergey Lukjanov <slukjanov at mirantis.com>
>> wrote:
>>
>>> Hi,
>>>
>>> as there is no any details on the project yet done, if this project will
>>> deploy ML frameworks it'll be direct duplication of Sahara's functionality
>>> (we already support HDP and CDH deployments and they are provided tons of
>>> tools for ML). So, I think that it could be built on top of Sahara or even
>>> as part of Sahara probably. I'd like to propose you to take a deeper look
>>> on Sahara and avoid duplicating it.
>>>
>>> Thanks.
>>>
>>> On Thu, May 14, 2015 at 8:47 PM, Debojyoti Dutta <ddutta at gmail.com>
>>> wrote:
>>>
>>>> Hi Salvatore
>>>>
>>>> Thanks a lot for your comments.
>>>>
>>>> Timing: Yes it is time to do this! The nature of applications running
>>>> on clouds is indeed changing.
>>>>
>>>> Initial group: We asked around for folks interested and we got a lot
>>>> more people than we expected. The idea is to get something out there in a
>>>> stack forge project and build something good. This group already has people
>>>> who have built things like this already in the past. Hence confident about
>>>> the success.
>>>>
>>>> Participation: We want this to be inclusive from scratch independent of
>>>> who is a PTL or a contributor or merely a curious individual to give us
>>>> ideas :) The community will get it right. Maybe I should have clarified
>>>> that these are the members interested in seeing this happen.
>>>>
>>>> Wiki page: The wiki page will be ready in 1-2 days. Also we would like
>>>> to have a discussion during the summit to see what we should build in the
>>>> community. Would be delighted to get your thoughts.
>>>>
>>>> Services: Some of the services this could provide:
>>>> * create experiments: define data sources, train models, then perform
>>>> classification, clustering, data cleaning etc.
>>>> * have experiment templates that can be reused
>>>> * have an editor (maybe a horizon plugin) to drag and drop the workflow
>>>> and generate an API that when called from an app would provide results
>>>> * ML primitives that could be targeted initially: 1) classification  2)
>>>> clustering 3) Anomaly detection
>>>>
>>>> thx
>>>> debo
>>>>
>>>> On Thu, May 14, 2015 at 5:02 PM, Salvatore Orlando <sorlando at nicira.com
>>>> > wrote:
>>>>
>>>>>
>>>>> On 15 May 2015 at 00:19, Debojyoti Dutta <ddutta at gmail.com> wrote:
>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> It is a great pleasure to announce the development of a new project
>>>>>> called Cognitive.  Cognitive provides Machine Learning [1] as a Service
>>>>>> that enables operators to offer next generation data science based services
>>>>>> on top of their OpenStack Clouds.
>>>>>>
>>>>>
>>>>> I was indeed wondering when "Machine Learning as a Service" would come
>>>>> up...
>>>>>
>>>>>
>>>>>> This project will begin as a StackForge project baed upon an empty
>>>>>> cookiecutter [2] repo.  The repos to work in are:
>>>>>> Server: https://github.com/stackforge/cognitive
>>>>>> Client: https://github.com/stackforge/python-cognitiveclient
>>>>>>
>>>>>> Please join us via iRC on #openstack-cognitive on freenode.
>>>>>>
>>>>>> We will be holding a doodle poll to select times for our first
>>>>>> meeting the week after summit.  This doodle poll will close May 24th and
>>>>>> meeting times will be announced on the mailing list at that time.  At our
>>>>>> first IRC meeting, we will draft additional core team members. We would
>>>>>> like to invite interested individuals to join this exciting new development
>>>>>> effort!
>>>>>>
>>>>>
>>>>> From my little experience, "drafting" core members before even
>>>>> actually having a code base has drawbacks. Also, it seems the initial
>>>>> starting team is already large enough for ensuring support for 1 or 2
>>>>> release cycle.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>> Please commit your schedule in the doodle poll here:
>>>>>> http://doodle.com/drrka5tgbwpbfbxy
>>>>>>
>>>>>> Initial core team: Steven Dake, Aparupa Das Gupa, Debo~ Dutta, Johnu
>>>>>> George,  Kyle Mestery, Sarvesh Ranjan, Ralf Rantzau, Komei Shimamura, Marc
>>>>>> Solanas, Manoj Sharma, Yathi Udupi, Kai Zhang.
>>>>>>
>>>>>
>>>>> Hey! What's the Neutron PTL doing there? Sorry we need his reviews we
>>>>> can't loan it to you!
>>>>>
>>>>>
>>>>>>
>>>>>> A little bit about Cognitive:
>>>>>> Data driven applications on cloud infrastructure increasingly rely on
>>>>>> Machine Learning. Most data driven applications today use Machine Learning
>>>>>> (ML). This often requires application developers and data scientists to
>>>>>> write their own machine learning stack or deploy other packages to do any
>>>>>> kind of data science based applications. Data scientists also need to have
>>>>>> an easy way to rapidly experiment with data without having to write basic
>>>>>> infrastructure for data manipulations. Cognitive is a Machine Learning
>>>>>> service on top of OpenStack and provides machine learning based services to
>>>>>> tenants (API, workbench, compute service).
>>>>>>
>>>>>
>>>>> I wonder what kind of services you would offer; also you could have
>>>>> shared something about the architecture of this service. Is it providing a
>>>>> full machine learning stack, or just facilitating the use of existing one?
>>>>>
>>>>> But I see that there's a link to a wiki page below. This might have
>>>>> all the answers.
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> For information about blueprints check out:
>>>>>> https://blueprints.launchpad.net/cognitive
>>>>>> https://blueprints.launchpad.net/python-cognitiveclient
>>>>>>
>>>>>> For more details, check out our Wiki:
>>>>>> https://wiki.openstack.org/wiki/Cognitive
>>>>>>
>>>>>
>>>>> ... and unfortunately the wiki is empty ;)
>>>>>
>>>>>
>>>>>>
>>>>>> Please join the awesome Cognitive team in designing a world class
>>>>>> Machine Learning as a Service solution.
>>>>>>
>>>>>> We look forward to seeing you on IRC on #openstack-cognitive.
>>>>>>
>>>>>> Regards,
>>>>>> Debo~ Dutta (on behalf of the initial team)
>>>>>>
>>>>>> [1] http://en.wikipedia.org/wiki/Machine_learning
>>>>>> [2] https://github.com/openstack-dev/cookiecutter
>>>>>>
>>>>>>
>>>>>> __________________________________________________________________________
>>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>>> Unsubscribe:
>>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> -Debo~
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Sincerely yours,
>>> Sergey Lukjanov
>>> Sahara Technical Lead
>>> (OpenStack Data Processing)
>>> Principal Software Engineer
>>> Mirantis Inc.
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> -Debo~
>>
>
>
>
> --
> -Debo~
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Sincerely yours,
Sergey Lukjanov
Sahara Technical Lead
(OpenStack Data Processing)
Principal Software Engineer
Mirantis Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150522/ac3d7ab7/attachment.html>