hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Supratim Deka (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDDS-1082) OutOfMemoryError while reading key of size 100GB
Date Fri, 15 Feb 2019 09:09:00 GMT

     [ https://issues.apache.org/jira/browse/HDDS-1082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Supratim Deka updated HDDS-1082:
--------------------------------
    Attachment: HDDS-1082.001.patch

> OutOfMemoryError while reading key of size 100GB
> ------------------------------------------------
>
>                 Key: HDDS-1082
>                 URL: https://issues.apache.org/jira/browse/HDDS-1082
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>            Reporter: Nilotpal Nandi
>            Assignee: Supratim Deka
>            Priority: Blocker
>             Fix For: 0.4.0
>
>         Attachments: HDDS-1082.000.patch, HDDS-1082.000.patch, HDDS-1082.001.patch
>
>
> steps taken :
> --------------------
>  # put key with size 100GB
>  # Tried to read back the key.
> error thrown:
> ------------------------------
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> Dumping heap to /tmp/heapdump.bin ...
> Heap dump file created [3883178021 bytes in 10.667 secs]
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>  at org.apache.ratis.thirdparty.com.google.protobuf.ByteString.toByteArray(ByteString.java:643)
>  at org.apache.hadoop.ozone.common.Checksum.verifyChecksum(Checksum.java:217)
>  at org.apache.hadoop.hdds.scm.storage.BlockInputStream.readChunkFromContainer(BlockInputStream.java:227)
>  at org.apache.hadoop.hdds.scm.storage.BlockInputStream.prepareRead(BlockInputStream.java:188)
>  at org.apache.hadoop.hdds.scm.storage.BlockInputStream.read(BlockInputStream.java:130)
>  at org.apache.hadoop.ozone.client.io.KeyInputStream$ChunkInputStreamEntry.read(KeyInputStream.java:232)
>  at org.apache.hadoop.ozone.client.io.KeyInputStream.read(KeyInputStream.java:126)
>  at org.apache.hadoop.ozone.client.io.OzoneInputStream.read(OzoneInputStream.java:49)
>  at java.io.InputStream.read(InputStream.java:101)
>  at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
>  at org.apache.hadoop.ozone.web.ozShell.keys.GetKeyHandler.call(GetKeyHandler.java:98)
>  at org.apache.hadoop.ozone.web.ozShell.keys.GetKeyHandler.call(GetKeyHandler.java:48)
>  at picocli.CommandLine.execute(CommandLine.java:919)
>  at picocli.CommandLine.access$700(CommandLine.java:104)
>  at picocli.CommandLine$RunLast.handle(CommandLine.java:1083)
>  at picocli.CommandLine$RunLast.handle(CommandLine.java:1051)
>  at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:959)
>  at picocli.CommandLine.parseWithHandlers(CommandLine.java:1242)
>  at picocli.CommandLine.parseWithHandler(CommandLine.java:1181)
>  at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:61)
>  at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:52)
>  at org.apache.hadoop.ozone.web.ozShell.Shell.main(Shell.java:83){noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message