[openstack-dev] [new][cognitive] Announcing Cognitive - a project to deliver Machine Learning as a Service for OpenStack

Debojyoti Dutta ddutta at gmail.com
Fri May 22 15:48:26 UTC 2015


Hi Sergey

Thanks a lot for your interest. The bare bones API proposal is up on the
wiki page http://wiki.openstack.org/Cognitive

Sahara is about deploying and managing big data workloads like hadoop,
spark etc. Cognitive is about a simple API to do predictive analytics,
learning, data science workflow etc etc. Thus the goals are different. AWS
has Elastic MapReduce (in the same space as Sahara) and also AWS Machine
Learning (http://aws.amazon.com/machine-learning/) for which there is no
parallel. Should we point them to our internal 1st version of Cognitive on
our github which we opened.

If it requires to use big data toolchains to do our job, we will definitely
leverage Sahara for that, and not replicate the good work done in Sahara.
Our primary goal is to build (within the community) a simple machine
learning API (and a service) that abstracts the pain of data science for
the app developer.


thx

debo


PS: FWIW  I am at the summit till tonight so we could catch up here.



On Tue, May 19, 2015 at 2:16 PM, Sergey Lukjanov <slukjanov at mirantis.com>
wrote:

> Hi,
>
> as there is no any details on the project yet done, if this project will
> deploy ML frameworks it'll be direct duplication of Sahara's functionality
> (we already support HDP and CDH deployments and they are provided tons of
> tools for ML). So, I think that it could be built on top of Sahara or even
> as part of Sahara probably. I'd like to propose you to take a deeper look
> on Sahara and avoid duplicating it.
>
> Thanks.
>
> On Thu, May 14, 2015 at 8:47 PM, Debojyoti Dutta <ddutta at gmail.com> wrote:
>
>> Hi Salvatore
>>
>> Thanks a lot for your comments.
>>
>> Timing: Yes it is time to do this! The nature of applications running on
>> clouds is indeed changing.
>>
>> Initial group: We asked around for folks interested and we got a lot more
>> people than we expected. The idea is to get something out there in a stack
>> forge project and build something good. This group already has people who
>> have built things like this already in the past. Hence confident about the
>> success.
>>
>> Participation: We want this to be inclusive from scratch independent of
>> who is a PTL or a contributor or merely a curious individual to give us
>> ideas :) The community will get it right. Maybe I should have clarified
>> that these are the members interested in seeing this happen.
>>
>> Wiki page: The wiki page will be ready in 1-2 days. Also we would like to
>> have a discussion during the summit to see what we should build in the
>> community. Would be delighted to get your thoughts.
>>
>> Services: Some of the services this could provide:
>> * create experiments: define data sources, train models, then perform
>> classification, clustering, data cleaning etc.
>> * have experiment templates that can be reused
>> * have an editor (maybe a horizon plugin) to drag and drop the workflow
>> and generate an API that when called from an app would provide results
>> * ML primitives that could be targeted initially: 1) classification  2)
>> clustering 3) Anomaly detection
>>
>> thx
>> debo
>>
>> On Thu, May 14, 2015 at 5:02 PM, Salvatore Orlando <sorlando at nicira.com>
>> wrote:
>>
>>>
>>> On 15 May 2015 at 00:19, Debojyoti Dutta <ddutta at gmail.com> wrote:
>>>
>>>> Hi!
>>>>
>>>> It is a great pleasure to announce the development of a new project
>>>> called Cognitive.  Cognitive provides Machine Learning [1] as a Service
>>>> that enables operators to offer next generation data science based services
>>>> on top of their OpenStack Clouds.
>>>>
>>>
>>> I was indeed wondering when "Machine Learning as a Service" would come
>>> up...
>>>
>>>
>>>> This project will begin as a StackForge project baed upon an empty
>>>> cookiecutter [2] repo.  The repos to work in are:
>>>> Server: https://github.com/stackforge/cognitive
>>>> Client: https://github.com/stackforge/python-cognitiveclient
>>>>
>>>> Please join us via iRC on #openstack-cognitive on freenode.
>>>>
>>>> We will be holding a doodle poll to select times for our first meeting
>>>> the week after summit.  This doodle poll will close May 24th and meeting
>>>> times will be announced on the mailing list at that time.  At our first IRC
>>>> meeting, we will draft additional core team members. We would like to
>>>> invite interested individuals to join this exciting new development effort!
>>>>
>>>
>>> From my little experience, "drafting" core members before even actually
>>> having a code base has drawbacks. Also, it seems the initial starting team
>>> is already large enough for ensuring support for 1 or 2 release cycle.
>>>
>>>
>>>>
>>>>
>>>
>>>> Please commit your schedule in the doodle poll here:
>>>> http://doodle.com/drrka5tgbwpbfbxy
>>>>
>>>> Initial core team: Steven Dake, Aparupa Das Gupa, Debo~ Dutta, Johnu
>>>> George,  Kyle Mestery, Sarvesh Ranjan, Ralf Rantzau, Komei Shimamura, Marc
>>>> Solanas, Manoj Sharma, Yathi Udupi, Kai Zhang.
>>>>
>>>
>>> Hey! What's the Neutron PTL doing there? Sorry we need his reviews we
>>> can't loan it to you!
>>>
>>>
>>>>
>>>> A little bit about Cognitive:
>>>> Data driven applications on cloud infrastructure increasingly rely on
>>>> Machine Learning. Most data driven applications today use Machine Learning
>>>> (ML). This often requires application developers and data scientists to
>>>> write their own machine learning stack or deploy other packages to do any
>>>> kind of data science based applications. Data scientists also need to have
>>>> an easy way to rapidly experiment with data without having to write basic
>>>> infrastructure for data manipulations. Cognitive is a Machine Learning
>>>> service on top of OpenStack and provides machine learning based services to
>>>> tenants (API, workbench, compute service).
>>>>
>>>
>>> I wonder what kind of services you would offer; also you could have
>>> shared something about the architecture of this service. Is it providing a
>>> full machine learning stack, or just facilitating the use of existing one?
>>>
>>> But I see that there's a link to a wiki page below. This might have all
>>> the answers.
>>>
>>>
>>>>
>>>>
>>>> For information about blueprints check out:
>>>> https://blueprints.launchpad.net/cognitive
>>>> https://blueprints.launchpad.net/python-cognitiveclient
>>>>
>>>> For more details, check out our Wiki:
>>>> https://wiki.openstack.org/wiki/Cognitive
>>>>
>>>
>>> ... and unfortunately the wiki is empty ;)
>>>
>>>
>>>>
>>>> Please join the awesome Cognitive team in designing a world class
>>>> Machine Learning as a Service solution.
>>>>
>>>> We look forward to seeing you on IRC on #openstack-cognitive.
>>>>
>>>> Regards,
>>>> Debo~ Dutta (on behalf of the initial team)
>>>>
>>>> [1] http://en.wikipedia.org/wiki/Machine_learning
>>>> [2] https://github.com/openstack-dev/cookiecutter
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> -Debo~
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Sincerely yours,
> Sergey Lukjanov
> Sahara Technical Lead
> (OpenStack Data Processing)
> Principal Software Engineer
> Mirantis Inc.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
-Debo~
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150522/cb8f74e1/attachment.html>


More information about the OpenStack-dev mailing list