From issues-return-44322-archive-asf-public=cust-asf.ponee.io@tez.apache.org Fri Jun 4 09:05:02 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 5083E18066B for ; Fri, 4 Jun 2021 11:05:02 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 797DC402D2 for ; Fri, 4 Jun 2021 09:05:01 +0000 (UTC) Received: (qmail 97472 invoked by uid 500); 4 Jun 2021 09:05:01 -0000 Mailing-List: contact issues-help@tez.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tez.apache.org Delivered-To: mailing list issues@tez.apache.org Received: (qmail 97459 invoked by uid 99); 4 Jun 2021 09:05:01 -0000 Received: from mailrelay1-he-de.apache.org (HELO mailrelay1-he-de.apache.org) (116.203.21.61) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jun 2021 09:05:01 +0000 Received: from jira2-he-de.apache.org (jira2-he-de.apache.org [168.119.33.54]) by mailrelay1-he-de.apache.org (ASF Mail Server at mailrelay1-he-de.apache.org) with ESMTPS id 618F43E8B9 for ; Fri, 4 Jun 2021 09:05:00 +0000 (UTC) Received: from jira2-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira2-he-de.apache.org (ASF Mail Server at jira2-he-de.apache.org) with ESMTP id 235D3C8020D for ; Fri, 4 Jun 2021 09:05:00 +0000 (UTC) Date: Fri, 4 Jun 2021 09:05:00 +0000 (UTC) From: "Tez CI (Jira)" To: issues@tez.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (TEZ-4295) Could not decompress data. Buffer length is too small. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/TEZ-4295?page=3Dcom.atlassian.j= ira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D173571= 98#comment-17357198 ]=20 Tez CI commented on TEZ-4295: ----------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m 47s= {color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0= m 0s{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0= m 0s{color} | {color:green} The patch does not contain any @author tags. {= color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green}= 0m 0s{color} | {color:green} The patch appears to include 1 new or modif= ied test files. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}= 12m 10s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0= m 35s{color} | {color:green} master passed with JDK Ubuntu-11.0.11+9-Ubuntu= -0ubuntu2.20.04 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0= m 27s{color} | {color:green} master passed with JDK Private Build-1.8.0_292= -8u292-b10-0ubuntu1~20.04-b10 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}= 1m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0= m 43s{color} | {color:green} master passed with JDK Ubuntu-11.0.11+9-Ubuntu= -0ubuntu2.20.04 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0= m 29s{color} | {color:green} master passed with JDK Private Build-1.8.0_292= -8u292-b10-0ubuntu1~20.04-b10 {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m 2= 8s{color} | {color:blue} Used deprecated FindBugs config; considering switc= hing to SpotBugs. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} = 1m 25s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}= 0m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0= m 25s{color} | {color:green} the patch passed with JDK Ubuntu-11.0.11+9-Ubu= ntu-0ubuntu2.20.04 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m = 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0= m 20s{color} | {color:green} the patch passed with JDK Private Build-1.8.0_= 292-8u292-b10-0ubuntu1~20.04-b10 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m = 20s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:oran= ge} 0m 16s{color} | {color:orange} tez-runtime-library: The patch generate= d 1 new + 63 unchanged - 0 fixed =3D 64 total (was 63) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green}= 0m 0s{color} | {color:green} The patch has no whitespace issues. {color}= | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0= m 19s{color} | {color:green} the patch passed with JDK Ubuntu-11.0.11+9-Ubu= ntu-0ubuntu2.20.04 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0= m 17s{color} | {color:green} the patch passed with JDK Private Build-1.8.0_= 292-8u292-b10-0ubuntu1~20.04-b10 {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 3s= {color} | {color:red} tez-runtime-library generated 1 new + 0 unchanged - 0= fixed =3D 1 total (was 0) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m 3= 9s{color} | {color:green} tez-runtime-library in the patch passed. {color} = | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green}= 0m 15s{color} | {color:green} The patch does not generate ASF License war= nings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 43m 47s{colo= r} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:tez-runtime-library | | | Redundant nullcheck of bufferSizeProp, which is known to be non-null = in org.apache.tez.runtime.library.utils.CodecUtils.getDecompressedInputStre= amWithBufferSize(CompressionCodec, IFileInputStream, Decompressor, int) Re= dundant null check at CodecUtils.java:is known to be non-null in org.apache= .tez.runtime.library.utils.CodecUtils.getDecompressedInputStreamWithBufferS= ize(CompressionCodec, IFileInputStream, Decompressor, int) Redundant null = check at CodecUtils.java:[line 88] | \\ \\ || Subsystem || Report/Notes || | Docker | ClientAPI=3D1.41 ServerAPI=3D1.41 base: https://ci-hadoop.apache= .org/job/tez-multibranch/job/PR-130/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/tez/pull/130 | | JIRA Issue | TEZ-4295 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs = checkstyle compile | | uname | Linux f9713958cc88 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 2= 3:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/tez.sh | | git revision | master / 0af54dfa5 | | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-= Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0= _292-8u292-b10-0ubuntu1~20.04-b10 | | checkstyle | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-130/= 1/artifact/out/diff-checkstyle-tez-runtime-library.txt | | findbugs | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-130/1/= artifact/out/new-findbugs-tez-runtime-library.html | | Test Results | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-1= 30/1/testReport/ | | Max. process+thread count | 121 (vs. ulimit of 5500) | | modules | C: tez-runtime-library U: tez-runtime-library | | Console output | https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-= 130/1/console | | versions | git=3D2.25.1 maven=3D3.6.3 findbugs=3D3.0.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. > Could not decompress data. Buffer length is too small. > ------------------------------------------------------ > > Key: TEZ-4295 > URL: https://issues.apache.org/jira/browse/TEZ-4295 > Project: Apache Tez > Issue Type: Bug > Affects Versions: 0.10.0 > Reporter: junnan.yang > Assignee: L=C3=A1szl=C3=B3 Bodor > Priority: Major > Fix For: 0.10.1 > > Attachments: TEZ-4295.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > tez =E4=BD=BF=E7=94=A8snappy=E5=8E=8B=E7=BC=A9=E6=96=B9=E5=BC=8F=E6=97=B6= =EF=BC=8C=E4=BC=9A=E6=8A=A5=E9=94=99=E7=BC=93=E5=86=B2=E5=8C=BA=E5=A4=AA=E5= =B0=8F=EF=BC=9A > java.io.IOException: java.lang.InternalError: Could not decompress data. = Buffer length is too small.java.io.IOException: java.lang.InternalError: Co= uld not decompress data. Buffer length is too small. at org.apache.tez.runt= ime.library.common.shuffle.ShuffleUtils.shuffleToMemory(ShuffleUtils.java:1= 37) at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Fetcher= OrderedGrouped.copyMapOutput(FetcherOrderedGrouped.java:550) at org.apache.= tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrderedGrouped.cop= yFromHost(FetcherOrderedGrouped.java:283) at org.apache.tez.runtime.library= .common.shuffle.orderedgrouped.FetcherOrderedGrouped.fetchNext(FetcherOrder= edGrouped.java:182) at org.apache.tez.runtime.library.common.shuffle.ordere= dgrouped.FetcherOrderedGrouped.callInternal(FetcherOrderedGrouped.java:194)= at org.apache.tez.runtime.library.common.shuffle.orderedgrouped.FetcherOrd= eredGrouped.callInternal(FetcherOrderedGrouped.java:57) at org.apache.tez.c= ommon.CallableWithNdc.call(CallableWithNdc.java:36) at com.google.common.ut= il.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.ru= nInterruptibly(TrustedListenableFutureTask.java:111) at com.google.common.u= til.concurrent.InterruptibleTask.run(InterruptibleTask.java:58) at com.goog= le.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenable= FutureTask.java:75) at java.util.concurrent.ThreadPoolExecutor.runWorker(Th= readPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Work= er.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748= )Caused by: java.lang.InternalError: Could not decompress data. Buffer leng= th is too small. at org.apache.hadoop.io.compress.snappy.SnappyDecompressor= .decompressBytesDirect(Native Method) at org.apache.hadoop.io.compress.snap= py.SnappyDecompressor.decompress(SnappyDecompressor.java:238) at org.apache= .hadoop.io.compress.BlockDecompressorStream.decompress(BlockDecompressorStr= eam.java:88) at org.apache.hadoop.io.compress.DecompressorStream.read(Decom= pressorStream.java:105) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.j= ava:210) at org.apache.tez.runtime.library.common.sort.impl.IFile$Reader.re= adToMemory(IFile.java:833) at org.apache.tez.runtime.library.common.shuffle= .ShuffleUtils.shuffleToMemory(ShuffleUtils.java:121) ... 12 more -- This message was sent by Atlassian Jira (v8.3.4#803005)