Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 19 Oct 2017 14:34:40 +0300 (EEST)
From: "r. r." <robert@abv.bg>
To: user@flink.apache.org
Message-ID: <1397770525.2517542.1508412881322.JavaMail.apache@nm73.abv.bg>
Subject: java.lang.NoSuchMethodError and dependencies problem
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
archived-at: Thu, 19 Oct 2017 11:34:56 -0000

Hello
I have a job that runs an Apache Tika pipeline and it fails with "Caused by: java.lang.NoSuchMethodError: org.apache.commons.compress.archivers.ArchiveStreamFactory.detect(Ljava/io/InputStream;)Ljava/lang/String;"

Flink includes commons-compress 1.4.1, while Tika needs 1.14. 
I also have Apache Avro in the project with commons-compress at 1.8.1, so I force 1.14 with 

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.apache.commons</groupId>
                <artifactId>commons-compress</artifactId>
                <version>1.14</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

this seems to work as mvn dependency:tree -Ddetail=true only shows 1.14 and after purge, the local maven repo also only contains 1.14

yet, after i deploy the job and it reads an Avro package from kafka and passes it to Tika, it fails with the error above, which leads me to think it somehow uses commons-compress at a version prior to 1.14, because method 'detect' is not present in older versions

I excluded/included it from the fat-jar
<exclude>org.apache.commons:commons-compress</exclude>
still the same problem

thanks for any hints!