avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lionel Delphin-Poulat (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-767) Support for message framing
Date Thu, 05 Jun 2014 09:38:01 GMT

    [ https://issues.apache.org/jira/browse/AVRO-767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018623#comment-14018623
] 

Lionel Delphin-Poulat commented on AVRO-767:
--------------------------------------------

I wonder if work has been done on the subject since 2011. I tried to insert it in version
1.7.6 of the library as a first step to add rpc support introduced by Gilles Gaillard in AVRO-777.
I have only a shallow understanding of avro and the improvements by Gilles.
Most of the code contained in frame_memory_patch could be reported in version 1.7.6. However,
there are some parts that are more delicate to insert. When the read type of the value is
in AVRO_BYTES and message framing is used (in value-readc.c), the read_bytes method does not
allocate memory but rather returns a pointer to the data. The pointed data should not be free.
So in this case, the free function specified in avro_wrapped_alloc_new should free the avro_wrapped_buffer
structure but not the data inside.

1. Is my understanding correct?
2. Does this mechanism apply for other value types? I think does not apply to simple types
such as AVRO_FLOAT, AVRO_DOUBLE, but I wonder if it applies to more complex types such as
AVRO_STRING, AVRO_ARRAY, AVRO_MAP, where memory copy might be avoided.

Thanks!

> Support for message framing
> ---------------------------
>
>                 Key: AVRO-767
>                 URL: https://issues.apache.org/jira/browse/AVRO-767
>             Project: Avro
>          Issue Type: Improvement
>          Components: c
>    Affects Versions: 1.5.0
>            Reporter: sebastien david
>              Labels: patch
>         Attachments: frame_memory_patch.diff
>
>   Original Estimate: 0h
>  Remaining Estimate: 0h
>
> Avro-c does not currently provide message framing, the attached patch implements this
improvement plus path throught for byte array encoding/decoding (don't copy anymore the bytes
but use direct pointer).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message