Return-Path: X-Original-To: apmail-tajo-dev-archive@minotaur.apache.org Delivered-To: apmail-tajo-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 641BC10079 for ; Mon, 14 Oct 2013 15:26:15 +0000 (UTC) Received: (qmail 40689 invoked by uid 500); 14 Oct 2013 15:26:15 -0000 Delivered-To: apmail-tajo-dev-archive@tajo.apache.org Received: (qmail 40651 invoked by uid 500); 14 Oct 2013 15:26:12 -0000 Mailing-List: contact dev-help@tajo.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tajo.incubator.apache.org Delivered-To: mailing list dev@tajo.incubator.apache.org Received: (qmail 40641 invoked by uid 99); 14 Oct 2013 15:26:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Oct 2013 15:26:09 +0000 X-ASF-Spam-Status: No, hits=-2000.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Mon, 14 Oct 2013 15:26:07 +0000 Received: (qmail 39011 invoked by uid 99); 14 Oct 2013 15:25:43 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Oct 2013 15:25:43 +0000 Date: Mon, 14 Oct 2013 15:25:43 +0000 (UTC) From: "Hyunsik Choi (JIRA)" To: dev@tajo.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (TAJO-231) In StorageManger v2, TestCSVCompression incurs unexpected end of stream occasionally MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/TAJO-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyunsik Choi updated TAJO-231: ------------------------------ Resolution: Incomplete Status: Resolved (was: Patch Available) I've just committed it, fixing as incomplete. > In StorageManger v2, TestCSVCompression incurs unexpected end of stream occasionally > ------------------------------------------------------------------------------------ > > Key: TAJO-231 > URL: https://issues.apache.org/jira/browse/TAJO-231 > Project: Tajo > Issue Type: Bug > Components: storage > Reporter: Hyunsik Choi > Assignee: Hyunsik Choi > Priority: Minor > Fix For: 0.2-incubating > > Attachments: TAJO-231.patch > > > In StorageManger v2, TestCSVCompression incurs 'unexpected end of stream occasionally'. > https://builds.apache.org/job/Tajo-trunk-postcommit/496/testReport/junit/org.apache.tajo.storage.v2/TestCSVCompression/testSplitCompressionData_0_/ > {noformat} > 2013-10-05 13:21:32,289 INFO [DiskFileScanner:0,sda] v2.DiskFileScanScheduler (DiskFileScanScheduler.java:run(104)) - Exit from Disk Queue:SplitCompression.bz2:0:65301_1380946892289 > 2013-10-05 13:21:32,289 INFO [main] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117)) - EOF:SplitCompression.bz2:0:65301 > 2013-10-05 13:21:32,296 ERROR [main] v2.CSVFileScanner (CSVFileScanner.java:nextTuple(299)) - unexpected end of stream > java.io.IOException: unexpected end of stream > at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.getAndMoveToFrontDecode(CBZip2InputStream.java:969) > at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.initBlock(CBZip2InputStream.java:504) > at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.changeStateToProcessABlock(CBZip2InputStream.java:333) > at org.apache.hadoop.io.compress.bzip2.CBZip2InputStream.read(CBZip2InputStream.java:399) > at org.apache.hadoop.io.compress.BZip2Codec$BZip2CompressionInputStream.read(BZip2Codec.java:483) > at java.io.InputStream.read(InputStream.java:101) > at org.apache.tajo.storage.v2.CSVFileScanner.page(CSVFileScanner.java:214) > at org.apache.tajo.storage.v2.CSVFileScanner.nextTuple(CSVFileScanner.java:282) > at org.apache.tajo.storage.v2.FileScannerV2.next(FileScannerV2.java:200) > at org.apache.tajo.storage.v2.TestCSVCompression.testSplitCompressionData(TestCSVCompression.java:147) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runner.JUnitCore.run(JUnitCore.java:157) > at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77) > at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195) > at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) > 2013-10-05 13:21:32,298 INFO [main] v2.ScheduledInputStream (ScheduledInputStream.java:close(348)) - Close:SplitCompression.bz2:0:65301 > 2013-10-05 13:21:32,298 INFO [main] v2.FileScannerV2 (FileScannerV2.java:close(184)) - file:/home/hyunsik/Code/tajo/tajo-4/tajo-core/tajo-core-backend/target/test-data/v2/TestCSVCompression/SplitCompression.bz2:0 closed, totalScanTime=2 > 2013-10-05 13:21:32,299 INFO [FileScanRunner:SplitCompression.bz2:65301:60218_1380946892299] v2.ScheduledInputStream (ScheduledInputStream.java:(81)) - Open:SplitCompression.bz2:65301:60218 > 2013-10-05 13:21:32,301 INFO [FileScanRunner:SplitCompression.bz2:65301:60218_1380946892301] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117)) - EOF:SplitCompression.bz2:65301:60218 > 2013-10-05 13:21:32,302 INFO [FileScanRunner:SplitCompression.bz2:65301:60218_1380946892301] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(121)) - EndOfStream:SplitCompression.bz2:65301:60218 > 2013-10-05 13:21:32,303 INFO [main] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117)) - EOF:SplitCompression.bz2:65301:60218 > 2013-10-05 13:21:32,305 INFO [main] v2.ScheduledInputStream (ScheduledInputStream.java:readNext(117)) - EOF:SplitCompression.bz2:65301:60218 > 2013-10-05 13:21:32,307 INFO [main] v2.ScheduledInputStream (ScheduledInputStream.java:close(348)) - Close:SplitCompression.bz2:65301:60218 > 2013-10-05 13:21:32,307 INFO [main] v2.FileScannerV2 (FileScannerV2.java:close(184)) - file:/home/hyunsik/Code/tajo/tajo-4/tajo-core/tajo-core-backend/target/test-data/v2/TestCSVCompression/SplitCompression.bz2:65301 closed, totalScanTime=2 > java.lang.AssertionError: > Expected :100000 > Actual :0 > > at org.junit.Assert.fail(Assert.java:93) > at org.junit.Assert.failNotEquals(Assert.java:647) > at org.junit.Assert.assertEquals(Assert.java:128) > at org.junit.Assert.assertEquals(Assert.java:472) > at org.junit.Assert.assertEquals(Assert.java:456) > at org.apache.tajo.storage.v2.TestCSVCompression.testSplitCompressionData(TestCSVCompression.java:159) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) > at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) > at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) > at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) > at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) > at org.junit.runners.ParentRunner.run(ParentRunner.java:300) > at org.junit.runner.JUnitCore.run(JUnitCore.java:157) > at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77) > at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195) > at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) > {noformat} -- This message was sent by Atlassian JIRA (v6.1#6144)