flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kl0u <...@git.apache.org>
Subject [GitHub] flink pull request: FLINK-3717 - Be able to re-start reading from ...
Date Tue, 26 Apr 2016 17:21:46 GMT
Github user kl0u commented on the pull request:

    https://github.com/apache/flink/pull/1895#issuecomment-214817636
  
    The problem with this change would be that we do not have yet a way to deserialize a record,
so 
    when we are trying to advance readRecords in the block, we have a NULL pointer exception.
    
    Actually this is what we do in the AvroInputFormat case, but there it works.
    
    Let me know if you have any other ideas on how to do it, or if you think that it may be
a problem also 
    for the Avro format.
    
    
    > On Apr 25, 2016, at 4:24 PM, Aljoscha Krettek <notifications@github.com> wrote:
    > 
    > I had some inline comments but overall the changes look good!
    > 
    > I think can simplify the BinaryInputFormat by getting rid of the filePos and justReadAllRecords
fields and just snapshotting the blockPos. The fieldPos and justReadAllRecords information
functionally depend on the blockPos, so storing the filePos and justReadAllRecords fields
just adds more complexity since we're keeping track of all of them.
    > 
    > The snapshot would then just be (blockPos, readRecords), upon restore the correct
file read position can be derived from the block/split start position.
    > 
    > —
    > You are receiving this because you authored the thread.
    > Reply to this email directly or view it on GitHub <https://github.com/apache/flink/pull/1895#issuecomment-214358848>
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message