Open Stack

Hi Quio,

> I know Storlets can provide user-defined computation functionalities,
> but I guess some capabilities can only be achieved using middleware.
> For example, a user may want such a feature: upon each PUT request, it
> creates a compressed copy of the object and stores both the original
> copy and compressed copy. It's feasible using middlware but I don't
> think Storlets provide such capability.

Interesting, exactly currently it's not supported to write to multi objects for a PUT request but as well as other middlewares we could adopt the feasibility into Storlets if you prefer.
Right now, the multi read (i.e. GET from multi sources) is only available and I think we would be able to expand the logic to PUT requests too. IIRC, in those days, we had discussion on sort of the
multi-out use cases and I'm sure the data structure inside Storlets are designed to be capable to that expantion. At that time, we called them "Tee" application on Storlets, I could not find the
historical discussion logs about how to implement tho, sorry. I believe that would be an use case for storlets if you prefer the user-defined application flexibilities rather than operator defined
Swift middleware.

The example of multi-read (GET from multi sources) are here:
https://github.com/openstack/storlets/blob/master/tests/functional/python/test_multiinput_storlet.py

And if you like to try to write multi write, please join us, I'm happy to help you anytime.


> Another example is that a user may want to install a Swift3-like
> middleware to provide APIs to a 3rd party, but she doesn't want other
> users to see this middleware.
>

If the definition can be made by operators, perhaps one possible solution that preparing different proxy-server endpoint for different users is available. i.e. an user uses no-s3api available proxy,
then the others use a different proxy-server endpoint that has the s3api in the pipeline.

Or, it sounds like kinda defaulter middleware[1], I don't think it has the scope turning on/off the middlewares for now.

1: https://review.openstack.org/#/c/342857/

Best,
Kota

(2018/09/18 11:34), Qiao Kang wrote:
> Kota,
> 
> Thanks for your reply, very helpful!
> 
> I know Storlets can provide user-defined computation functionalities,
> but I guess some capabilities can only be achieved using middleware.
> For example, a user may want such a feature: upon each PUT request, it
> creates a compressed copy of the object and stores both the original
> copy and compressed copy. It's feasible using middlware but I don't
> think Storlets provide such capability.
> 
> Another example is that a user may want to install a Swift3-like
> middleware to provide APIs to a 3rd party, but she doesn't want other
> users to see this middleware.
> 
> Regards,
> Qiao
> 
> On Mon, Sep 17, 2018 at 9:19 PM Kota TSUYUZAKI
> <tsuyuzaki.kota at lab.ntt.co.jp> wrote:
>>
>> With Storlets, users will be able to create their own applications that are able to run like as a Swift middeleware. The application (currently Python and Java are supported as the language but the
>> apps can calls any binaries in the workspace) can be uploaded as a Swift object, then, users can invoke them with just an extra header that specifies your apps.
>>
>> To fit your own use case, we may have to consider to invole or to integrate the system for you but I believe Storlets could be a choice for you.
>>
>> In detail, Storlets documantation is around there,
>>
>> Top Level Index: https://docs.openstack.org/storlets/latest/index.html
>> System Overview: https://docs.openstack.org/storlets/latest/storlet_engine_overview.html
>> APIs: https://docs.openstack.org/storlets/latest/api/overview_api.html
>>
>> Thanks,
>>
>> Kota
>>
>> (2018/09/17 8:59), John Dickinson wrote:
>>> You may be interested in Storlets. It's another OpenStack project, maintained by a Swift core reviewer, that provides this sort of user-defined middleware functionality.
>>>
>>> You can also ask about it in #openstack-swift
>>>
>>> --John
>>>
>>>
>>>
>>> On 16 Sep 2018, at 9:25, Qiao Kang wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm wondering whether Swift allows any user (not the administrator) to
>>>> specify which middleware that she/he wants his data object to go throught.
>>>> For instance, Alice wants to install a middleware but doesn't want Bob to
>>>> use it, where Alice and Bob are two accounts in a single Swift cluster.
>>>>
>>>> Or maybe all middlewares are pre-installed globally and cannot be
>>>> customized on a per-account basis?
>>>>
>>>> Thanks,
>>>> Qiao
>>>> _______________________________________________
>>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to     : openstack at lists.openstack.org
>>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>
>>> _______________________________________________
>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>> Post to     : openstack at lists.openstack.org
>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>>
>> --
>> ----------------------------------------------------------
>> Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.kota at lab.ntt.co.jp>
>> NTT Software Innovation Center
>> Distributed Computing Technology Project
>> Phone  0422-59-2837
>> Fax    0422-59-2965
>> -----------------------------------------------------------
>>
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> 
> 


-- 
----------------------------------------------------------
Kota Tsuyuzaki(露﨑　浩太)  <tsuyuzaki.kota at lab.ntt.co.jp>
NTT Software Innovation Center
Distributed Computing Technology Project
Phone  0422-59-2837
Fax    0422-59-2965
-----------------------------------------------------------