avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manuel Godbert (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1529) ConcatTool issue with empty files
Date Fri, 28 Aug 2015 08:40:45 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14718223#comment-14718223
] 

Manuel Godbert commented on AVRO-1529:
--------------------------------------

An I forgot to tell, as we just compacted small files for performance issues we never tried
to get one bigger than a block size!

> ConcatTool issue with empty files
> ---------------------------------
>
>                 Key: AVRO-1529
>                 URL: https://issues.apache.org/jira/browse/AVRO-1529
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.7.6
>            Reporter: Manuel Godbert
>
> I am using the ConcatTool to merge 2 avro files. The first one is empty, the second one
has records. The merged file is seen as empty when I am opening it.
> My investigations showed the cause is that DataFileWriter.appendAllFrom(...) copies blocks
from source to target even if they are empty, and DataFileReader.hasNext() returns false if
the first block of an avro file is empty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message