hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13601) Connection leak during log splitting
Date Thu, 30 Apr 2015 20:03:08 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14522152#comment-14522152
] 

Hadoop QA commented on HBASE-13601:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12729538/HBASE-13601-0.98.patch
  against 0.98 branch at commit d88a4180f780e25ec442917de0c4e6f8a5c92686.
  ATTACHMENT ID: 12729538

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions
(2.4.1 2.5.2 2.6.0)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 protoc{color}.  The applied patch does not increase the total number of
protoc compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 25 warning messages.

    {color:green}+1 checkstyle{color}.  The applied patch does not increase the total number
of checkstyle errors

    {color:green}+1 findbugs{color}.  The patch does not introduce any  new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13901//testReport/
Release Findbugs (version 2.0.3) 	warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13901//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13901//artifact/patchprocess/checkstyle-aggregate.html

  Javadoc warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13901//artifact/patchprocess/patchJavadocWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13901//console

This message is automatically generated.

> Connection leak during log splitting
> ------------------------------------
>
>                 Key: HBASE-13601
>                 URL: https://issues.apache.org/jira/browse/HBASE-13601
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.0.0, 0.98.10
>            Reporter: Abhishek Singh Chouhan
>            Assignee: Abhishek Singh Chouhan
>             Fix For: 0.98.13, 1.0.2
>
>         Attachments: HBASE-13601-0.98.patch, HBASE-13601-1.0.0.patch
>
>
> Ran into an issue where Region server died with the following exception
> {noformat}
> 2015-04-29 17:10:11,856 WARN  [nector@0.0.0.0:60030] mortbay.log - EXCEPTION
> java.io.IOException: Too many open files
>         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
>         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
>         at org.mortbay.jetty.nio.SelectChannelConnector$1.acceptChannel(SelectChannelConnector.java:75)
>         at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:686)
>         at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
>         at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
>         at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
>         at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> {noformat}
> Realized that all the tcp sockets on the system were used out due to the regionserver
trying to split the log and failing multiple times and leaving a connection open -
> {noformat}
> java.io.IOException: Got error for OP_READ_BLOCK, self=/10..99.3:50695, remote=/10.232.99.36:50010,
for file /hbase/WALs/host1,60020,1425930917890-splitting/host1%2C60020%2C1425930917890.1429358890944,
for pool BP-181199659-10.232.99.2-1411124363096 block 1074497051_756497
>         at org.apache.hadoop.hdfs.RemoteBlockReader2.checkSuccess(RemoteBlockReader2.java:432)
>         at org.apache.hadoop.hdfs.RemoteBlockReader2.newBlockReader(RemoteBlockReader2.java:397)
>         at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReader(BlockReaderFactory.java:786)
>         at org.apache.hadoop.hdfs.BlockReaderFactory.getRemoteBlockReaderFromTcp(BlockReaderFactory.java:665)
>         at org.apache.hadoop.hdfs.BlockReaderFactory.build(BlockReaderFactory.java:325)
>         at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:567)
>         at org.apache.hadoop.hdfs.DFSInputStream.seekToNewSource(DFSInputStream.java:1446)
>         at org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:769)
>         at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:799)
>         at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:840)
>         at java.io.DataInputStream.read(DataInputStream.java:100)
>         at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:124)
>         at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:91)
>         at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:660)
>         at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:569)
>         at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:282)
>         at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:225)
>         at org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:143)
>         at org.apache.hadoop.hbase.regionserver.handler.HLogSplitterHandler.process(HLogSplitterHandler.java:82)
>         at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message