hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "satish (Jira)" <j...@apache.org>
Subject [jira] [Created] (HUDI-633) archival fails with large clean files
Date Tue, 25 Feb 2020 02:04:00 GMT
satish created HUDI-633:
---------------------------

             Summary: archival fails with large clean files
                 Key: HUDI-633
                 URL: https://issues.apache.org/jira/browse/HUDI-633
             Project: Apache Hudi (incubating)
          Issue Type: Bug
            Reporter: satish
            Assignee: satish


Caused by: java.lang.OutOfMemoryError: Java heap space

at java.util.Arrays.copyOf(Arrays.java:3236)
at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
at org.apache.avro.io.BufferedBinaryEncoder$OutputStreamSink.innerWrite(BufferedBinaryEncoder.java:216)
at org.apache.avro.io.BufferedBinaryEncoder.flushBuffer(BufferedBinaryEncoder.java:93)
at org.apache.avro.io.BufferedBinaryEncoder.ensureBounds(BufferedBinaryEncoder.java:108)
at org.apache.avro.io.BufferedBinaryEncoder.writeFixed(BufferedBinaryEncoder.java:153)
at org.apache.avro.io.BinaryEncoder.writeString(BinaryEncoder.java:55)
at org.apache.avro.io.Encoder.writeString(Encoder.java:121)
at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:213)
at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:208)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:76)
at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:138)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:68)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
at org.apache.avro.generic.GenericDatumWriter.writeMap(GenericDatumWriter.java:180)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:69)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
10:01
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)

at com.uber.hoodie.common.table.log.block.HoodieAvroDataBlock.getContentBytes(HoodieAvroDataBlock.java:124)
at com.uber.hoodie.common.table.log.HoodieLogFormatWriter.appendBlock(HoodieLogFormatWriter.java:126)
at com.uber.hoodie.io.HoodieCommitArchiveLog.writeToFile(HoodieCommitArchiveLog.java:267)
at com.uber.hoodie.io.HoodieCommitArchiveLog.archive(HoodieCommitArchiveLog.java:249)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message