trafficserver-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean Baptiste Favre <webmas...@jbfavre.org>
Subject Re: Buffered transformation plugin question
Date Mon, 09 Dec 2013 19:13:44 GMT
On 09/12/2013 19:01, James Peach wrote:
> On Dec 6, 2013, at 3:09 PM, Jean Baptiste Favre <webmaster@jbfavre.org> wrote:
> 
>> Hello,
>>
>> I need to cache some json API response.
>> The API sent only indented json.
>>
>> To reduce network trafic and optimize cache, I need to be able to remove
>> space and new lines.
> 
> The LinkedIn folks wrote a PageSpeed plugin. I think it can do this, <https://github.com/linkedin/ats_pagespeed>.
I already tested it. But, either I missed some configuration or
pagespeed does not support 'application/json' content-type :-/
And pagespeed homepage does not mention JSON support:
https://developers.google.com/speed/pagespeed/module?hl=en

>> I'm considering to use gzip. Problem is, this would invalidate current
>> cache which is not an option. But, when ATS will revalidate URL, it'll
>> use gzip and both gziped & normal content will be cache.
> 
> Would is really get invalidated? If the client would accept compressed content and there
is already an uncompressed variant in the cache, would the gzip plugin be able to send a compressed
version from cache?
If a gzip request has been cached (and config option "cache" is set to
true), then later request with or without gzip will be served from cache.

But, for now, I only have non-gzip content cached. After enabling gzip,
first gzip request is seen as a miss, so request is forwarded to origin
server.

>> In the meantime, I'm trying to implement a plugin to handle json
>> transformation.
>> From what I understand, I need to write a buffered transformation plugin
>> to make sure I got the whole json. So I studied bnull_tranform plugin
>> and read the documentation.
>>
>> From what I read, I have to add my code into handle_output function.
>> I need to use data->output_reader to get content into a char, transform
>> it with json-c library function.
>> Char result will be added into a new TSIOBuffer.
>> Once done, I'll be able to write it into output_conn.
>>
>> Am I right ?
>>
>> My problem is that I don't know how to get data->output_reader content
>> as char.
>> As far as I've seen, output_reader is a TSIOBufferReader structure and I
>> can not find any definition of it.
>>
>> As you may have noticed, I'm quite a newbie in C, but any advice will be
>> apreciated.
> 
> I haven't written a transformation, but it seems they can be tricky. As Daniel Morilha
pointed out on a different thread, the LinkedIn C++ API is a good additional reference for
transformation code, <https://github.com/linkedin/atscppapi/blob/master/src/TransformationPlugin.cc>

I now have an "almost working plugin".
It's mostly based on bnull-tranformation example. I'm able to compact
response from origin server.
Now have to compact as well cached request so that I can spare some LAN
bandwitch :)

I agree this type of plugin can be very tricky to write.

I'll have a deeper look on pagespeed API module. I already saw it but
missed the TransformationPlugin part.

Finally, I had some trouble to understand bnull-tranformation example.
In fact, it does not work at all as it is. There is a very small
modification to do.
I'll make a pull request as soon as I can. At the same time, I'll
release my module so that you guys can have a look on it a criticize it :)

Best regards,
Jean-Baptiste

Mime
View raw message