hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharat Viswanadham (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDDS-1121) Key read failure when data is written parallel in to Ozone
Date Sat, 16 Feb 2019 03:32:00 GMT
Bharat Viswanadham created HDDS-1121:
----------------------------------------

             Summary: Key read failure when data is written parallel in to Ozone
                 Key: HDDS-1121
                 URL: https://issues.apache.org/jira/browse/HDDS-1121
             Project: Hadoop Distributed Data Store
          Issue Type: Bug
            Reporter: Bharat Viswanadham
            Assignee: Bharat Viswanadham


When hive is run with multiple threads for data ingestion to ozone. After ingestion is done,
during read we see this below error.

This issue is found during hive testing.
{code:java}
caused by: org.apache.hadoop.ozone.common.OzoneChecksumException: Checksum mismatch at index
0
 at org.apache.hadoop.ozone.common.ChecksumData.verifyChecksumDataMatches(ChecksumData.java:143)
 at org.apache.hadoop.ozone.common.Checksum.verifyChecksum(Checksum.java:239)
 at org.apache.hadoop.ozone.common.Checksum.verifyChecksum(Checksum.java:217)
 at org.apache.hadoop.hdds.scm.storage.BlockInputStream.readChunkFromContainer(BlockInputStream.java:227)
 at org.apache.hadoop.hdds.scm.storage.BlockInputStream.seek(BlockInputStream.java:259)
 at org.apache.hadoop.ozone.client.io.KeyInputStream$ChunkInputStreamEntry.seek(KeyInputStream.java:249)
 at org.apache.hadoop.ozone.client.io.KeyInputStream.seek(KeyInputStream.java:180)
 at org.apache.hadoop.fs.ozone.OzoneFSInputStream.seek(OzoneFSInputStream.java:62)
 at org.apache.hadoop.fs.FSInputStream.read(FSInputStream.java:82)
 at org.apache.hadoop.fs.FSInputStream.readFully(FSInputStream.java:121)
 at org.apache.hadoop.fs.FSDataInputStream.readFully(FSDataInputStream.java:111)
 at org.apache.orc.impl.ReaderImpl.extractFileTail(ReaderImpl.java:555)
 at org.apache.orc.impl.ReaderImpl.<init>(ReaderImpl.java:370)
 at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:61)
 at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:105)
 at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.populateAndCacheStripeDetails(OrcInputFormat.java:1647)
 at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:1533)
 at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$2700(OrcInputFormat.java:1329)
 at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1513)
 at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:1510)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1688)
 at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1510)
 at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:1329)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message