tajo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hyunsik Choi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (TAJO-231) In StorageManger v2, TestCSVCompression incurs unexpected end of stream occasionally
Date Mon, 14 Oct 2013 15:25:43 GMT

     [ https://issues.apache.org/jira/browse/TAJO-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hyunsik Choi updated TAJO-231:
------------------------------

    Resolution: Incomplete
        Status: Resolved  (was: Patch Available)

I've just committed it, fixing as incomplete.

> In StorageManger v2, TestCSVCompression incurs unexpected end of stream occasionally
> ------------------------------------------------------------------------------------
>
>                 Key: TAJO-231
>                 URL: https://issues.apache.org/jira/browse/TAJO-231
>             Project: Tajo
>          Issue Type: Bug
>          Components: storage
>            Reporter: Hyunsik Choi
>            Assignee: Hyunsik Choi
>            Priority: Minor
>             Fix For: 0.2-incubating
>
>         Attachments: TAJO-231.patch
>
>
> In StorageManger v2, TestCSVCompression incurs 'unexpected end of stream occasionally'.
> https://builds.apache.org/job/Tajo-trunk-postcommit/496/testReport/junit/org.apache.tajo.storage.v2/TestCSVCompression/testSplitCompressionData_0_/
> {noformat}
> 2013-10-05 13:21:32,289 INFO  [DiskFileScanner:0,sda] v2.DiskFileScanScheduler (DiskFileScanScheduler.java:run(104))
- Exit from Disk Queue:SplitCompression.bz2:0:65301_1380946892289
> 2013-10-05 13:21:32,289 INFO  [main] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117))
- EOF:SplitCompression.bz2:0:65301
> 2013-10-05 13:21:32,296 ERROR [main] v2.CSVFileScanner (CSVFileScanner.java:nextTuple(299))
- unexpected end of stream
> java.io.IOException: unexpected end of stream
> 	at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.getAndMoveToFrontDecode(CBZip2InputStream.java:969)
> 	at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.initBlock(CBZip2InputStream.java:504)
> 	at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.changeStateToProcessABlock(CBZip2InputStream.java:333)
> 	at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.read(CBZip2InputStream.java:399)
> 	at org.apache.hadoop.io.compress.BZip2Codec$BZip2CompressionInputStream.read(BZip2Codec.java:483)
> 	at java.io.InputStream.read(InputStream.java:101)
> 	at org.apache.tajo.storage.v2.CSVFileScanner.page(CSVFileScanner.java:214)
> 	at org.apache.tajo.storage.v2.CSVFileScanner.nextTuple(CSVFileScanner.java:282)
> 	at org.apache.tajo.storage.v2.FileScannerV2.next(FileScannerV2.java:200)
> 	at org.apache.tajo.storage.v2.TestCSVCompression.testSplitCompressionData(TestCSVCompression.java:147)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> 	at org.junit.runners.Suite.runChild(Suite.java:128)
> 	at org.junit.runners.Suite.runChild(Suite.java:24)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> 	at org.junit.runners.Suite.runChild(Suite.java:128)
> 	at org.junit.runners.Suite.runChild(Suite.java:24)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> 2013-10-05 13:21:32,298 INFO  [main] v2.ScheduledInputStream (ScheduledInputStream.java:close(348))
- Close:SplitCompression.bz2:0:65301
> 2013-10-05 13:21:32,298 INFO  [main] v2.FileScannerV2 (FileScannerV2.java:close(184))
- file:/home/hyunsik/Code/tajo/tajo-4/tajo-core/tajo-core-backend/target/test-data/v2/TestCSVCompression/SplitCompression.bz2:0
closed, totalScanTime=2
> 2013-10-05 13:21:32,299 INFO  [FileScanRunner:SplitCompression.bz2:65301:60218_1380946892299]
v2.ScheduledInputStream (ScheduledInputStream.java:<init>(81)) - Open:SplitCompression.bz2:65301:60218
> 2013-10-05 13:21:32,301 INFO  [FileScanRunner:SplitCompression.bz2:65301:60218_1380946892301]
v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117)) - EOF:SplitCompression.bz2:65301:60218
> 2013-10-05 13:21:32,302 INFO  [FileScanRunner:SplitCompression.bz2:65301:60218_1380946892301]
v2.ScheduledInputStream (ScheduledInputStream.java:readNext(121)) - EndOfStream:SplitCompression.bz2:65301:60218
> 2013-10-05 13:21:32,303 INFO  [main] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117))
- EOF:SplitCompression.bz2:65301:60218
> 2013-10-05 13:21:32,305 INFO  [main] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117))
- EOF:SplitCompression.bz2:65301:60218
> 2013-10-05 13:21:32,307 INFO  [main] v2.ScheduledInputStream (ScheduledInputStream.java:close(348))
- Close:SplitCompression.bz2:65301:60218
> 2013-10-05 13:21:32,307 INFO  [main] v2.FileScannerV2 (FileScannerV2.java:close(184))
- file:/home/hyunsik/Code/tajo/tajo-4/tajo-core/tajo-core-backend/target/test-data/v2/TestCSVCompression/SplitCompression.bz2:65301
closed, totalScanTime=2
> java.lang.AssertionError: 
> Expected :100000
> Actual   :0
>  <Click to see difference>
> 	at org.junit.Assert.fail(Assert.java:93)
> 	at org.junit.Assert.failNotEquals(Assert.java:647)
> 	at org.junit.Assert.assertEquals(Assert.java:128)
> 	at org.junit.Assert.assertEquals(Assert.java:472)
> 	at org.junit.Assert.assertEquals(Assert.java:456)
> 	at org.apache.tajo.storage.v2.TestCSVCompression.testSplitCompressionData(TestCSVCompression.java:159)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> 	at org.junit.runners.Suite.runChild(Suite.java:128)
> 	at org.junit.runners.Suite.runChild(Suite.java:24)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> 	at org.junit.runners.Suite.runChild(Suite.java:128)
> 	at org.junit.runners.Suite.runChild(Suite.java:24)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message