incubator-directmemory-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benoit Perroud <ben...@noisette.ch>
Subject Re: [DISCUSS] RFC Server implementation
Date Mon, 20 Feb 2012 10:53:05 GMT
Hi All,

First of all thanks for the warm welcome of the presentation, we'll
try to spread the word as much as possible.

My idea here would be to have several "cache region", like

${webPath}/${region}/${key}

and allowing some metadata regarding those regions :

- The serializer (thus no need to pass it at every request)
- Properties of the region : max size, eviction algorithm, slab size
(if my new implementation will be selected), ...

Objects of the same nature could be stored in the same "region".

>From the actual code base, there is some synchronization done to
allocate object in a Cache, thus having multiple region/caches in the
back would allow more concurrency. And would be able to fine tune the
cache for the objects stored (big almost static files vs small often
changing objects).

Cheers,

Benoit.




2012/2/20 Simone Tripodi <simonetripodi@apache.org>:
> Salut!
>
>> BTW nothing prevent to have both implementations (based on
>> Content-Type    "application/json;charset=UTF-8" for json model exchange
>> :-) )
>
> sure, that is why I like content type negotiation: same URI, different
> way to exchange content. Simple & cool at the same time :P
>
>> Doh sure we can "win" some packet on network layer :-)
>>
>
> lol, aren't we problem solvers? :D
>
>> Ok I will start as it. My first impl (push) will be with json exchange
>> as I have it locally :-).
>>
>> Then I will improve with the "raw" model based on Content-Type and http headers.
>
> looking forward to read the code, hope to be helpful!
> -Simo
>
> http://people.apache.org/~simonetripodi/
> http://simonetripodi.livejournal.com/
> http://twitter.com/simonetripodi
> http://www.99soft.org/
>
>
>
> On Mon, Feb 20, 2012 at 11:01 AM, Olivier Lamy <olamy@apache.org> wrote:
>> 2012/2/20 Simone Tripodi <simonetripodi@apache.org>:
>>> Salut!
>>>
>>> I see we have 2 different use cases in mind, that is why we are not in sync.
>>>
>>> I personally see that adopting your APIs layer, for each request, body
>>> content has to be serialized in a textual form - no matter how fast it
>>> is, that means that both client/server need to (de)serialize the
>>> content.
>>
>>>
>>> The client builds the request -> serialize it in JSON -> creates the
>>> JSON request -> send it to the server -> the server extract the JSON
>>> content -> stream it in the memory.
>> agree it can be a performance "problem".
>>
>
>>
>>>
>>> Let's explore more possibilities: you can still map your JSON request
>>> model to pure HTTP protocol:
>>>
>>> {"DirectMemoryRQ":{"key":"101","expiresIn":123,"cacheContent":"*here a
>>> json/xml string*
>>> ","serializer":"org.apache.directmemory.serialization.StandardSerializer"}}
>>>
>>> would become
>>>
>>> +----+
>>> PUT  ${webPath}/${key}
>>> Content-Type: text/xml
>> I would prefer use an other Content-Type IMHO
>> "application/x-java-serialized-object" was a good idea.
>>> Expires: Thu, 01 Dec 2012 16:00:00 GMT
>>>
>>> serialized-body
>>> +----+
>>>
>>> extra non-standard fields can be added via the X- prefix, for example,
>>> the preference of used serializer to store the content:
>> Good idea.
>>>
>>> +----+
>>> PUT  ${webPath}/${key}
>>> Content-Type: text/xml
>>> Expires: Thu, 01 Dec 2012 16:00:00 GMT
>>> X-DirectMemory-Serializer:
>>> org.apache.directmemory.serialization.StandardSerializer
>>>
>>> serialized-body
>>> +----+
>>>
>>> You should still be able to write the JS httpclient to interact with
>>> the server - it is just HTTP! - without paying the overhead of more
>>> parsing operations.
>>>
>>> Moreover, if you'll proceed to JSON, I think that the proposed format
>>> can be thiner:
>>>
>>> {"DirectMemoryRQ":{"key":"101","expiresIn":123,"cacheContent":"*here a
>>> json/xml string*
>>> ","serializer":"org.apache.directmemory.serialization.StandardSerializer"}}
>>>
>>> could be
>>>
>>> {"key":"101","expiresIn":123,"cacheContent":"*here a
>>> json/xml string*
>>> ","serializer":"org.apache.directmemory.serialization.StandardSerializer"}
>>>
>>> dropping the DirectMemoryRQ container (it is yet another data structure :P)
>>
>> Doh sure we can "win" some packet on network layer :-)
>>
>> Ok I will start as it. My first impl (push) will be with json exchange
>> as I have it locally :-).
>>
>> Then I will improve with the "raw" model based on Content-Type and http headers.
>>
>>>
>>> Thanks for brainstorming!
>>
>> You're welcome :P
>>
>>> -Simo
>>>
>>> http://people.apache.org/~simonetripodi/
>>> http://simonetripodi.livejournal.com/
>>> http://twitter.com/simonetripodi
>>> http://www.99soft.org/
>>>
>>>
>>>
>>> On Mon, Feb 20, 2012 at 9:55 AM, Olivier Lamy <olamy@apache.org> wrote:
>>>> Hello,
>>>>
>>>> 2012/2/20 Simone Tripodi <simonetripodi@apache.org>:
>>>>>> Hmmmm. Simone should answer this.
>>>>>>
>>>>>> If I'm really thinking about it, the content-type makes no sense.
;)
>>>>>
>>>>> it does instead, because the server has to be able to reply the
>>>>> serialization content type when clients ask for stored - or you expect
>>>>> the client automagically understands it? Or you expect to have just
>>>>> one client type per application, so the serialization type is always
>>>>> the same? Couldn't servers be populated by heterogeneous clients that
>>>>> read each other data?
>>>> The client will be responsible for de/serialization.
>>>> My preference for more "complex" content is for new feature and
>>>> enhance cache request/response with new field.
>>>>
>>>> retrieve object :
>>>> GET on ${webPath}/cache/${key}
>>>>
>>>> return json content if found
>>>>
>>>> {"DirectMemoryRS":{"key":"101","cacheContent":"Zm9vIGJhcg=="}}
>>>> Using this format we will be able to add new fields in the ison
>>>> "object" (cacheAge etc...)
>>>>
>>>> PUT on ${webPath}/cache
>>>>
>>>> json content posted
>>>>
>>>> {"DirectMemoryRQ":{"key":"101","expiresIn":123,"cacheContent":"rO0ABXNyACtvcmcuYXBhY2hlLmRpcmVjdG1lbW9yeS5zZXJ2ZXIuY29tbW9ucy5XaW5l5dYKhxyAjeECAAFMAARuYW1ldAASTGphdmEvbGFuZy9TdHJpbmc7eHB0AAhCb3JkZWF1eA==","serializer":"org.apache.directmemory.serialization.StandardSerializer"}}
>>>>
>>>> Again using a json "object" we will be able to easily add new "fields".
>>>>
>>>> Why using json I have in mind using cache access directly tru a
>>>> javascript application with something like :
>>>>
>>>> {"DirectMemoryRQ":{"key":"101","expiresIn":123,"cacheContent":"*here a
>>>> json/xml string*
>>>> ","serializer":"org.apache.directmemory.serialization.StandardSerializer"}}
>>>>
>>>> As it the server we will be able to serialize then store the json/xml content.
>>>> And you can use directly from your javascript application.
>>>>
>>>> Compare to an Api.
>>>>
>>>> wineService.findWine(year, name)
>>>>
>>>> wineService.find(wineSearchRequest)
>>>>
>>>> With
>>>> WineSearchRequest {
>>>>  year;
>>>>  name;
>>>> }
>>>>
>>>> I tend to prefer using the second pattern as it's more easy to improve
>>>> the model without breaking (normally :-) ) the backward comp.
>>>>
>>>> Makes sense ?
>>>>
>>>>>
>>>>> "makes no sense" is a hard assertion, I always think more than twice
>>>>> before saying that ;)
>>>>>
>>>>> http://people.apache.org/~simonetripodi/
>>>>> http://simonetripodi.livejournal.com/
>>>>> http://twitter.com/simonetripodi
>>>>> http://www.99soft.org/
>>>>
>>>>
>>>>
>>>> --
>>>> Olivier Lamy
>>>> Talend: http://coders.talend.com
>>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
>>
>>
>> --
>> Olivier Lamy
>> Talend: http://coders.talend.com
>> http://twitter.com/olamy | http://linkedin.com/in/olamy



-- 
sent from my Nokia 3210

Mime
View raw message