beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1065) FileBasedSource: replace SeekableByteChannel with open(spec, startingPosition)
Date Thu, 01 Dec 2016 16:51:58 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712452#comment-15712452
] 

Daniel Halperin commented on BEAM-1065:
---------------------------------------

The ability to open at a non-zero location has previously been thought to be linked to seekability.
Do we think there are channels that can be opened at non-0 offsets but cannot be seeked?

> FileBasedSource: replace SeekableByteChannel with open(spec, startingPosition)
> ------------------------------------------------------------------------------
>
>                 Key: BEAM-1065
>                 URL: https://issues.apache.org/jira/browse/BEAM-1065
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-java-core
>            Reporter: Pei He
>            Assignee: Pei He
>
> FileBasedReader should be able to open the file with the Source.getStartOffset(), and
then read forward to find the first input element.
> The benefits are:
> 1. It is easier to implement a ReadableByteChannel.
> 2. Dynamically splitting won't require file systems to support seeking.
> 3. Doesn't need to seek to position twice, which is what current API does.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message