From dev-return-175220-archive-asf-public=cust-asf.ponee.io@commons.apache.org Wed Oct 14 10:21:54 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mailroute1-lw-us.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 2004118063F for ; Wed, 14 Oct 2020 12:21:54 +0200 (CEST) Received: from mail.apache.org (localhost [127.0.0.1]) by mailroute1-lw-us.apache.org (ASF Mail Server at mailroute1-lw-us.apache.org) with SMTP id 55F22123B1E for ; Wed, 14 Oct 2020 10:21:53 +0000 (UTC) Received: (qmail 15486 invoked by uid 500); 14 Oct 2020 10:21:52 -0000 Mailing-List: contact dev-help@commons.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: "Commons Developers List" Delivered-To: mailing list dev@commons.apache.org Received: (qmail 15470 invoked by uid 99); 14 Oct 2020 10:21:52 -0000 Received: from spamproc1-he-de.apache.org (HELO spamproc1-he-de.apache.org) (116.203.196.100) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 14 Oct 2020 10:21:52 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-de.apache.org (ASF Mail Server at spamproc1-he-de.apache.org) with ESMTP id AAA151FF39D for ; Wed, 14 Oct 2020 10:21:51 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-de.apache.org X-Spam-Flag: NO X-Spam-Score: 1.193 X-Spam-Level: * X-Spam-Status: No, score=1.193 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HK_RANDOM_ENVFROM=0.626, HK_RANDOM_FROM=0.768, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamproc1-he-de.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([116.203.227.195]) by localhost (spamproc1-he-de.apache.org [116.203.196.100]) (amavisd-new, port 10024) with ESMTP id YHHQHkwmInO6 for ; Wed, 14 Oct 2020 10:21:51 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2a00:1450:4864:20::12a; helo=mail-lf1-x12a.google.com; envelope-from=lbrtchx@gmail.com; receiver= Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id DA08E7F9CC for ; Wed, 14 Oct 2020 10:21:50 +0000 (UTC) Received: by mail-lf1-x12a.google.com with SMTP id h6so3106193lfj.3 for ; Wed, 14 Oct 2020 03:21:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=9Ydh+AbMvImPjBbrCRJZdU8wMhRqzum5D8YEIVvvs8U=; b=no9rvd8ouSi3Ed420X8wSCIWk8BNjdosN0Ti4c6eqSV57sF5PmkoL8EqJrF6SQ42sv mwgr6laeWHEetxHyjbJz1C5NtIcW/9TX3ur9Iz+zoTYeXpIkHdnS4wft/bl4xpHo7XDP GgR+RiLL68cIyjTF2M0McbcUvdjADkijDBZdE41KIRtd2w/wmkCb3CN1xbmL1LX3uJSw cXE5EzkmEkHSGQXjzvUdqWverjhRbXpuBEaAl2FdRdQV7LuRB1aSm/s4F14Szk5dp1bW QUoXZutC7/xYJS4xtM+7LVdX168h+gdLkKTzqx5K5Wd0ZEDBISGIAEKzdFR3Umr46Ubh PhNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=9Ydh+AbMvImPjBbrCRJZdU8wMhRqzum5D8YEIVvvs8U=; b=RCZ1ckHRcfeHbCDn5HVFc5I04CWkVibA5yTz6px59h+kcbIXPp3G7ECizN+Vt6zJyq q+bkZi07fGWIxNOS7pzfCnvkqqpLUmtseh6sTTakjtjq9q5lvHuVv4PUqd4hCAFpTD6I riQVsTgNIbAlnEI17Kjy/AfsQoVkWiFAemdLgFREoVMQX5k7ONFqPvXdiJkdJWXU3TeU U+fgKDmwvz//Y7w634KPVa0W2Z1P/6BJ6QCvppPaBF/UjHbTdQQJpW/C/l+KjdyJG78l fQZXxPj4YxMW2T5dBG0C4kqJOc0EeB8cZj6FddYAwNviA9aukgTBhBKedYPQcsluApes /sNA== X-Gm-Message-State: AOAM532p1pPMpeDoY04bdptn2QCegJxtJJ3JPyQLpiAsN0X6eajlIyw5 8ZvmZeeOtZfANBGaQEDUooOL4v8CQbCtSRO74nrwBKPU60Y= X-Google-Smtp-Source: ABdhPJzSRvQ+8Na4vgbHhSimrbcSu0TIDK2/SgHNXUB9HyOXHTFUZbDFRgJ8GZTE166uUJMf/PiGyX3QL2FoP49G1fw= X-Received: by 2002:ac2:52b7:: with SMTP id r23mr1067979lfm.30.1602670909827; Wed, 14 Oct 2020 03:21:49 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:9357:0:0:0:0:0 with HTTP; Wed, 14 Oct 2020 03:21:49 -0700 (PDT) In-Reply-To: References: From: Albretch Mueller Date: Wed, 14 Oct 2020 12:21:49 +0200 Message-ID: Subject: Re: [compress] BZip2CompressorInputStream stops working without rhyme or reason ... To: Commons Developers List Content-Type: text/plain; charset="UTF-8" I don't know what could there apaprently be exactly at byte offset 2848 in some buffer but files reporing to be fine by bzip2 --test can't be processed by BZip2CompressorInputStream: ~ $ _IFL="/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream1.xml-p1p41242.bz2" $ ls -l "${_IFL}" -r--r--r-- 1 lbrtchx lbrtchx 242624781 Sep 22 05:40 /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream1.xml-p1p41242.bz2 $ file --brief "${_IFL}" bzip2 compressed data, block size = 900k $ time bzip2 --test --verbose "${_IFL}" /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream1.xml-p1p41242.bz2: ok real 2m0.650s user 2m0.076s sys 0m0.256s $ _IFL="/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream4.xml-p311330p558391.bz2" $ ls -l "${_IFL}" -r--r--r-- 1 lbrtchx lbrtchx 394001572 Sep 22 05:49 /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream4.xml-p311330p558391.bz2 $ file --brief "${_IFL}" bzip2 compressed data, block size = 900k $ time bzip2 --test --verbose "${_IFL}" /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream4.xml-p311330p558391.bz2: ok real 3m6.249s user 3m5.192s sys 0m0.628s $ _IFL="/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream5.xml-p558392p958045.bz2" $ ls -l "${_IFL}" -r--r--r-- 1 lbrtchx lbrtchx 427323881 Sep 22 05:51 /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream5.xml-p558392p958045.bz2 $ file --brief "${_IFL}" bzip2 compressed data, block size = 900k $ time bzip2 --test --verbose "${_IFL}" /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream5.xml-p558392p958045.bz2: ok real 3m20.861s user 3m19.296s sys 0m0.988s $ _IFL="/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream6.xml-p958046p1483661.bz2" $ ls -l "${_IFL}" -r--r--r-- 1 lbrtchx lbrtchx 458830618 Sep 22 05:52 /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream6.xml-p958046p1483661.bz2 $ file --brief "${_IFL}" bzip2 compressed data, block size = 900k $ time bzip2 --test --verbose "${_IFL}" /home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream6.xml-p958046p1483661.bz2: ok real 3m34.213s user 3m32.636s sys 0m1.056s $ $ _IFL="/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/logs/UnKmprssBZ2_02Test_20201013234903.log" $ tail -n 10 "${_IFL}" // __ Files Context of |4| files containing a total of |1522780852| bytes! // __ [0/4): ...(30.131%) |/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream6.xml-p958046p1483661.bz2| // __ aOFlNm: |/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/REF/enwiki-20200920-pages-articles-multistream6-p958046p1483661.xml| // __ |2848|2848|java.io.IOException: // __ Read bytes and file lenght not the same! lTtlRdByts: |2848| (lTtlRdByts != lFlL), lFlL: |458830618| at UnKmprssBZ2_02Test.main(UnKmprssBZ2_02Test.java:254) real 0m1.759s user 0m2.920s sys 0m0.196s $ _IFL="/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/logs/UnKmprssBZ2_02Test_20201013234826.log" $ tail -n 10 "${_IFL}" // __ Files Context of |4| files containing a total of |1522780852| bytes! // __ [0/4): ...(28.062%) |/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream5.xml-p558392p958045.bz2| // __ aOFlNm: |/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/REF/enwiki-20200920-pages-articles-multistream5-p558392p958045.xml| // __ |2848|2848|java.io.IOException: // __ Read bytes and file lenght not the same! lTtlRdByts: |2848| (lTtlRdByts != lFlL), lFlL: |427323881| at UnKmprssBZ2_02Test.main(UnKmprssBZ2_02Test.java:254) real 0m1.669s user 0m2.720s sys 0m0.220s $ _IFL="/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/logs/UnKmprssBZ2_02Test_20201013234708.log" $ tail -n 10 "${_IFL}" // __ Files Context of |4| files containing a total of |1522780852| bytes! // __ [0/4): ...(25.874%) |/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream4.xml-p311330p558391.bz2| // __ aOFlNm: |/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/REF/enwiki-20200920-pages-articles-multistream4-p311330p558391.xml| // __ |2848|2848|java.io.IOException: // __ Read bytes and file lenght not the same! lTtlRdByts: |2848| (lTtlRdByts != lFlL), lFlL: |394001572| at UnKmprssBZ2_02Test.main(UnKmprssBZ2_02Test.java:254) real 0m1.665s user 0m2.752s sys 0m0.172s $ _IFL="/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/logs/UnKmprssBZ2_02Test_20201013234602.log" $ tail -n 10 "${_IFL}" // __ Files Context of |4| files containing a total of |1522780852| bytes! // __ [0/4): ...(15.933%) |/home/lbrtchx/cmllpz/LklWb/org/wikimedia/dumps/enwiki/20200920/enwiki-20200920-pages-articles-multistream1.xml-p1p41242.bz2| // __ aOFlNm: |/home/lbrtchx/cmllpz/prjx/kd/java/IO/compress/REF/enwiki-20200920-pages-articles-multistream1-p1p41242.xml| // __ |2848|2848|java.io.IOException: // __ Read bytes and file lenght not the same! lTtlRdByts: |2848| (lTtlRdByts != lFlL), lFlL: |242624781| at UnKmprssBZ2_02Test.main(UnKmprssBZ2_02Test.java:254) real 0m1.691s user 0m2.756s sys 0m0.216s $ --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org For additional commands, e-mail: dev-help@commons.apache.org