hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: strange SIGPIPE from "bin/ls -ld" in Shell.runCommand seen when running HBase tests
Date Wed, 07 Dec 2011 18:29:28 GMT
Mikhail:
Your patch was stripped by email server.

I assume you have verified permission for all components of path:
/data/users/mbautin/workdirs/hb-os/target/test-data/37d6e996-cba6-4a12-85bc-dbcf2e91d297

Cheers

On Tue, Dec 6, 2011 at 5:07 PM, Mikhail Bautin <
bautin.mailing.lists@gmail.com> wrote:

> Hello,
>
> I've been running into the following issue when running HBase tests. A lot
> of them would fail with an exception similar to  the one shown below (I
> added more information to the exception messages). Exit code 141 seems to
> correspond to SIGPIPE, but I did not find anything obvious in Shell.java in
> Hadoop. In Shell.runCommand we read the error stream on a separate thread
> and the output stream on the main thread, and consume both streams
> completely before waiting for the external process to terminate. I ended up
> doing a hacky work-around (patch attached) to be able to run HBase tests,
> but any insight about what could be causing this issue is appreciated.
> HBase trunk uses hadoop-0.20.205.0 by default.
>
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.544 sec
> <<< FAILURE!
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
>  Time elapsed: 0 sec  <<< ERROR!
> java.lang.RuntimeException: Error while running command to get file
> permissions : org.apache.hadoop.util.Shell$ExitCodeException: Command:
> [/bin/ls, -ld,
> /data/users/mbautin/workdirs/hb-os/target/test-data/37d6e996-cba6-4a12-85bc-dbcf2e91d297/dfscluster_76df8fc0-6827-4d9d-8728-eb5ee43b0bae/dfs/data/data3],
> message:
>  , exitCode: 141
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:283)
>         at org.apache.hadoop.util.Shell.run(Shell.java:183)
>         at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:405)
>         at org.apache.hadoop.util.Shell.execCommand(Shell.java:491)
>         at org.apache.hadoop.util.Shell.execCommand(Shell.java:474)
>         at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:703)
>         at
> org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:418)
>         at
> org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus.getPermission(RawLocalFileSystem.java:393)
>         at
> org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:146)
>         at
> org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:162)
>         at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1537)
>         at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1484)
>         at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1459)
>         at
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:417)
>         at
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
>         at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:369)
>         at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:537)
>         at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:493)
>         at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:480)
>         at
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort.setupBeforeClass(TestRegionServerCoprocessorExceptionWithAbort.java:94)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>         at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>         at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>         at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
>         at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>         at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>         at
> org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
>         at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
>         at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
>         at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
>         at
> org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
>         at
> org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:81)
>         at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
>
> Thanks,
> --Mikhail
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message