flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-987) Extend TypeSerializers and -Comparators to work directly on Memory Segments
Date Sat, 05 Jul 2014 12:23:33 GMT

    [ https://issues.apache.org/jira/browse/FLINK-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052847#comment-14052847

ASF GitHub Bot commented on FLINK-987:

Github user zentol commented on the pull request:

    should we postpone this PR until #53 and [FLINK-987] are done?

> Extend TypeSerializers and -Comparators to work directly on Memory Segments
> ---------------------------------------------------------------------------
>                 Key: FLINK-987
>                 URL: https://issues.apache.org/jira/browse/FLINK-987
>             Project: Flink
>          Issue Type: Improvement
>          Components: Local Runtime
>    Affects Versions: 0.6-incubating
>            Reporter: Stephan Ewen
>            Assignee: Aljoscha Krettek
>             Fix For: 0.6-incubating
> As per discussion with [~till.rohrmann], [~uce], [~aljoscha], we suggest to change the
way that the TypeSerialzers/Comparators and DataInputViews/DataOutputViews work.
> The goal is to allow more flexibility in the construction on the binary representation
of data types, and to allow partial deserialization of individual fields. Both is currently
prohibited by the fact that the abstraction of the memory (into which the data goes) is a
stream abstraction ({{DataInputView}}, {{DataOutputView}}).
> An idea is to offer a random-access buffer like view for construction and random-access
deserialization, as well as various methods to copy elements in a binary fashion between such
buffers and streams.
> A possible set of methods for the {{TypeSerializer}} could be:
> {code}
> long serialize(T record, TargetBuffer buffer);
> T deserialize(T reuse, SourceBuffer source);
> void ensureBufferSufficientlyFilled(SourceBuffer source);
> <X> X deserializeField(X reuse, int logicalPos, SourceBuffer buffer);
> int getOffsetForField(int logicalPos, int offset, SourceBuffer buffer);
> void copy(DataInputView in, TargetBuffer buffer);
> void copy(SourceBuffer buffer,, DataOutputView out);
> void copy(DataInputView source, DataOutputView target);
> {code}

This message was sent by Atlassian JIRA

View raw message