Open Stack

Debo,

The Sahara cats are having a contributors meetup right now in room 218. @ ODS.  I have prior commitments so won’t be able to attend, but perhaps some of the cognitive folks could make the meetup there.

Sergey,

If the Sahara contributors plan on walking the city tonight, feel free to send me info off-list and I’ll try to make it.

Regards
-steve


From: Sergey Lukjanov <slukjanov at mirantis.com<mailto:slukjanov at mirantis.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Friday, May 22, 2015 at 9:21 AM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [new][cognitive] Announcing Cognitive - a project to deliver Machine Learning as a Service for OpenStack

So, in case if Cognitive will use Sahara to create big data clusters - then it sounds ok.

On Fri, May 22, 2015 at 8:54 AM, Debojyoti Dutta <ddutta at gmail.com<mailto:ddutta at gmail.com>> wrote:
One more thing and a typo in my mail  -  we would be happy to point folks to an very early version (in Django) we prototyped and opened up recently.

thx
debo



On Fri, May 22, 2015 at 8:48 AM, Debojyoti Dutta <ddutta at gmail.com<mailto:ddutta at gmail.com>> wrote:
Hi Sergey

Thanks a lot for your interest. The bare bones API proposal is up on the wiki page http://wiki.openstack.org/Cognitive

Sahara is about deploying and managing big data workloads like hadoop, spark etc. Cognitive is about a simple API to do predictive analytics, learning, data science workflow etc etc. Thus the goals are different. AWS has Elastic MapReduce (in the same space as Sahara) and also AWS Machine Learning (http://aws.amazon.com/machine-learning/) for which there is no parallel. Should we point them to our internal 1st version of Cognitive on our github which we opened.


If it requires to use big data toolchains to do our job, we will definitely leverage Sahara for that, and not replicate the good work done in Sahara. Our primary goal is to build (within the community) a simple machine learning API (and a service) that abstracts the pain of data science for the app developer.


thx

debo


PS: FWIW  I am at the summit till tonight so we could catch up here.



On Tue, May 19, 2015 at 2:16 PM, Sergey Lukjanov <slukjanov at mirantis.com<mailto:slukjanov at mirantis.com>> wrote:
Hi,

as there is no any details on the project yet done, if this project will deploy ML frameworks it'll be direct duplication of Sahara's functionality (we already support HDP and CDH deployments and they are provided tons of tools for ML). So, I think that it could be built on top of Sahara or even as part of Sahara probably. I'd like to propose you to take a deeper look on Sahara and avoid duplicating it.

Thanks.

On Thu, May 14, 2015 at 8:47 PM, Debojyoti Dutta <ddutta at gmail.com<mailto:ddutta at gmail.com>> wrote:
Hi Salvatore

Thanks a lot for your comments.

Timing: Yes it is time to do this! The nature of applications running on clouds is indeed changing.

Initial group: We asked around for folks interested and we got a lot more people than we expected. The idea is to get something out there in a stack forge project and build something good. This group already has people who have built things like this already in the past. Hence confident about the success.

Participation: We want this to be inclusive from scratch independent of who is a PTL or a contributor or merely a curious individual to give us ideas :) The community will get it right. Maybe I should have clarified that these are the members interested in seeing this happen.

Wiki page: The wiki page will be ready in 1-2 days. Also we would like to have a discussion during the summit to see what we should build in the community. Would be delighted to get your thoughts.

Services: Some of the services this could provide:
* create experiments: define data sources, train models, then perform classification, clustering, data cleaning etc.
* have experiment templates that can be reused
* have an editor (maybe a horizon plugin) to drag and drop the workflow and generate an API that when called from an app would provide results
* ML primitives that could be targeted initially: 1) classification  2) clustering 3) Anomaly detection

thx
debo

On Thu, May 14, 2015 at 5:02 PM, Salvatore Orlando <sorlando at nicira.com<mailto:sorlando at nicira.com>> wrote:

On 15 May 2015 at 00:19, Debojyoti Dutta <ddutta at gmail.com<mailto:ddutta at gmail.com>> wrote:
Hi!

It is a great pleasure to announce the development of a new project called Cognitive.  Cognitive provides Machine Learning [1] as a Service that enables operators to offer next generation data science based services on top of their OpenStack Clouds.

I was indeed wondering when "Machine Learning as a Service" would come up...

This project will begin as a StackForge project baed upon an empty cookiecutter [2] repo.  The repos to work in are:
Server: https://github.com/stackforge/cognitive
Client: https://github.com/stackforge/python-cognitiveclient

Please join us via iRC on #openstack-cognitive on freenode.

We will be holding a doodle poll to select times for our first meeting the week after summit.  This doodle poll will close May 24th and meeting times will be announced on the mailing list at that time.  At our first IRC meeting, we will draft additional core team members. We would like to invite interested individuals to join this exciting new development effort!

>From my little experience, "drafting" core members before even actually having a code base has drawbacks. Also, it seems the initial starting team is already large enough for ensuring support for 1 or 2 release cycle.



Please commit your schedule in the doodle poll here: http://doodle.com/drrka5tgbwpbfbxy

Initial core team: Steven Dake, Aparupa Das Gupa, Debo~ Dutta, Johnu George,  Kyle Mestery, Sarvesh Ranjan, Ralf Rantzau, Komei Shimamura, Marc Solanas, Manoj Sharma, Yathi Udupi, Kai Zhang.

Hey! What's the Neutron PTL doing there? Sorry we need his reviews we can't loan it to you!


A little bit about Cognitive:
Data driven applications on cloud infrastructure increasingly rely on Machine Learning. Most data driven applications today use Machine Learning (ML). This often requires application developers and data scientists to write their own machine learning stack or deploy other packages to do any kind of data science based applications. Data scientists also need to have an easy way to rapidly experiment with data without having to write basic infrastructure for data manipulations. Cognitive is a Machine Learning service on top of OpenStack and provides machine learning based services to tenants (API, workbench, compute service).

I wonder what kind of services you would offer; also you could have shared something about the architecture of this service. Is it providing a full machine learning stack, or just facilitating the use of existing one?

But I see that there's a link to a wiki page below. This might have all the answers.



For information about blueprints check out:
https://blueprints.launchpad.net/cognitive
https://blueprints.launchpad.net/python-cognitiveclient

For more details, check out our Wiki:
https://wiki.openstack.org/wiki/Cognitive

... and unfortunately the wiki is empty ;)


Please join the awesome Cognitive team in designing a world class Machine Learning as a Service solution.

We look forward to seeing you on IRC on #openstack-cognitive.

Regards,
Debo~ Dutta (on behalf of the initial team)

[1] http://en.wikipedia.org/wiki/Machine_learning
[2] https://github.com/openstack-dev/cookiecutter

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
-Debo~

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Sincerely yours,
Sergey Lukjanov
Sahara Technical Lead
(OpenStack Data Processing)
Principal Software Engineer
Mirantis Inc.

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
-Debo~



--
-Debo~

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Sincerely yours,
Sergey Lukjanov
Sahara Technical Lead
(OpenStack Data Processing)
Principal Software Engineer
Mirantis Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150522/1f67a35e/attachment.html>