From user-return-78277-archive-asf-public=cust-asf.ponee.io@spark.apache.org Wed Jun 5 07:25:34 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 7D3C218065D for ; Wed, 5 Jun 2019 09:25:34 +0200 (CEST) Received: (qmail 63959 invoked by uid 500); 5 Jun 2019 07:25:28 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@spark.apache.org Received: (qmail 63947 invoked by uid 99); 5 Jun 2019 07:25:28 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Jun 2019 07:25:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 6FB58C2433 for ; Wed, 5 Jun 2019 07:25:27 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.912 X-Spam-Level: * X-Spam-Status: No, score=1.912 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HTML_MESSAGE=2, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.638, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id MxaVgeJuG9gI for ; Wed, 5 Jun 2019 07:25:25 +0000 (UTC) Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 5FF525F35A for ; Wed, 5 Jun 2019 07:25:25 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id n4so15282799wrs.3 for ; Wed, 05 Jun 2019 00:25:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=8sScj5qZrhx6RIvwcqJcjqKgzYbkHa5nlnS3tLXjuMw=; b=oKXc+xWH3IGVey3VtgtTVZVbAaARYLx3nLKp+hFmij/nyKcKw+TiB/zHjQtn4WG4XS pMCnxmDxQUqnyixTaOb7ywfD6l+RzL9giJB1p8wq1xWqb7eUf8mqNk9PRtMfb8IjDgPX +c4DFG/KOP6SxUXj+nalwDjsG2yeD9OCJbDOTDvtjqk3KCIInZ8C6tCGq8Sz0yROLkUo U9No7uY7Nx1MebH5dtjHReAhI1DeQtxklZBxuCLE1Fa7xjts9XevqVo5fEI+evABxz8z YH/fgYj3ECU2UNDyJC59LpZ2fotIgnUZlWt6gO2UHZDIs9cfKp3lIIkXqywgR/7v1wgI xrqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=8sScj5qZrhx6RIvwcqJcjqKgzYbkHa5nlnS3tLXjuMw=; b=rHnjIs1/fSWiIaFOIXqdKjk0jndzBcBTalwq1OxDuTXlKolUT34bBziax2qhpndhBh 1Tyj3NthjMW3/SxoLFOgNWCfbWs9MuWfNDdovjxR7I9MwaeyzID3vYiCVxi8ZQX606Ct 4/7PGmdZ0vTBI355U+0a0sAEUvJpAdi061Stye0i9Iby3xnUfcHtr0BmFNaIvUubhtdg 85D8YQ2m3wUyDoU1P39hC+dmMiiGSrYbZXpFTkfb/2GN89gx9XpJ6qcFhGST0sTgl0G9 OYfl62NbUIWJWwk04H8XVvd3XhZyB+M6ebEaRG2phLtks8DWDaJpOmvyCFjAsjAo5/n4 YQeA== X-Gm-Message-State: APjAAAXib8NHaoUAOHn4nd37dfJt4PNAP1DHQ866nZT9kZ71OgXc2pWj FV8JIkhmk9iPdnGqztsXrF2fG1yn4YazHtxuYhJ69cU1eqwHDw== X-Google-Smtp-Source: APXvYqyN5Jk4zL6tt/yIc2wV7xpVpfXhXcPGXnH0+kNEE1qnjafzAm9cqYnI74wLP4gKZnpzPSbj8sgtWmxp5+BPZ68= X-Received: by 2002:a5d:4dd1:: with SMTP id f17mr1068295wru.43.1559719524083; Wed, 05 Jun 2019 00:25:24 -0700 (PDT) MIME-Version: 1.0 From: Tianhua huang Date: Wed, 5 Jun 2019 15:25:13 +0800 Message-ID: Subject: spark ./build/mvn test failed on aarch64 To: user@spark.apache.org Content-Type: multipart/alternative; boundary="0000000000002eac89058a8e806f" --0000000000002eac89058a8e806f Content-Type: text/plain; charset="UTF-8" Hi all, Recently I run './build/mvn test' of spark on aarch64, and master and branch-2.4 are all failled, the log pieces as below: ...... [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running org.apache.spark.util.kvstore.LevelDBTypeInfoSuite [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.081 s - in org.apache.spark.util.kvstore.LevelDBTypeInfoSuite [INFO] Running org.apache.spark.util.kvstore.InMemoryStoreSuite [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.apache.spark.util.kvstore.InMemoryStoreSuite [INFO] Running org.apache.spark.util.kvstore.InMemoryIteratorSuite [INFO] Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.219 s - in org.apache.spark.util.kvstore.InMemoryIteratorSuite [INFO] Running org.apache.spark.util.kvstore.LevelDBIteratorSuite [ERROR] Tests run: 38, Failures: 0, Errors: 38, Skipped: 0, Time elapsed: 0.23 s <<< FAILURE! - in org.apache.spark.util.kvstore.LevelDBIteratorSuite [ERROR] copyIndexDescendingWithStart(org.apache.spark.util.kvstore.LevelDBIteratorSuite) Time elapsed: 0.2 s <<< ERROR! java.lang.UnsatisfiedLinkError: Could not load library. Reasons: [no leveldbjni64-1.8 in java.library.path, no leveldbjni-1.8 in java.library.path, no leveldbjni in java.library.path, /usr/local/src/spark/common/kvstore/target/tmp/libleveldbjni-64-1-610267671268036503.8: /usr/local/src/spark/common/kvstore/target/tmp/libleveldbjni-64-1-610267671268036503.8: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64-bit .so on a AARCH64-bit platform)] at org.apache.spark.util.kvstore.LevelDBIteratorSuite.createStore(LevelDBIteratorSuite.java:44) ...... There is a dependency of leveldbjni-all , but there is no the native package for aarch64 i in leveldbjni-1.8(all) .jar, I found aarch64 is supported after pr https://github.com/fusesource/leveldbjni/pull/82, but it was not in the 1.8 release, and unfortunately the repo didn't updated almost for two years. So I have a question: does spark support aarch64, and if it is yes, then how to fix this problem, if it is not, what's the plan for it? Thank you all! --0000000000002eac89058a8e806f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi all,
Recently I run './build/mvn test' of s= park on aarch64, and master and branch-2.4 are all failled, the log pieces = as below:

......

[INFO]=C2=A0T E S T S
[INFO]=C2=A0------= -------------------------------------------------
[INFO]=C2=A0Running= org.apache.spark.util.kvstore.LevelDBTypeInfoSuite
[INFO]=C2=A0Tests= run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.081 s - in or= g.apache.spark.util.kvstore.LevelDBTypeInfoSuite
[INFO]=C2=A0Running = org.apache.spark.util.kvstore.InMemoryStoreSuite
[INFO]=C2=A0Tests ru= n: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.ap= ache.spark.util.kvstore.InMemoryStoreSuite
[INFO]=C2=A0Running org.ap= ache.spark.util.kvstore.InMemoryIteratorSuite
[INFO]=C2=A0Tests run: = 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.219 s - in org.apac= he.spark.util.kvstore.InMemoryIteratorSuite
[INFO]=C2=A0Running org.a= pache.spark.util.kvstore.LevelDBIteratorSuite
[ERROR]=C2=A0Tests run:= 38, Failures: 0, Errors: 38, Skipped: 0, Time elapsed: 0.23 s <<<= FAILURE! - in org.apache.spark.util.kvstore.LevelDBIteratorSuite
[ERROR]=C2=A0copyIndexDescendingWithStart(org.apache.spark.util.kvstore.LevelD= BIteratorSuite) Time elapsed: 0.2 s <<< ERROR!
java.lang.Unsati= sfiedLinkError: Could not load library. Reasons:=C2=A0[no leveldbjni64-1.8 in j= ava.library.path, no leveldbjni-1.8 in java.library.path, no leveldbjni in = java.library.path, /usr/local/src/spark/common/kvstore/target/tmp/libleveld= bjni-64-1-610267671268036503.8: /usr/local/src/spark/common/kvstore/target/= tmp/libleveldbjni-64-1-610267671268036503.8: cannot open shared object file= : No such file or directory (Possible cause: can't load AMD 64-bit .so = on a AARCH64-bit platform)]
at org.apache.spark.util.kvstore.Leve= lDBIteratorSuite.createStore(LevelDBIteratorSuite.java:44)

......

There is a dependency of=C2=A0 leveldbjni-all=C2=A0=C2=A0, but there is no the nat= ive package for aarch64 i in=C2=A0leveldbjni-1.8(all) .jar, I found aarch64 is suppor= ted after pr https://github.com/fusesource/leveldbjni/pull/82, but it was not in = the 1.8 release, and unfortunately t= he repo didn't updated almost for

two years.

S= o I have a question: does spark support aarch64, and if it is yes, then how= to fix this problem, if it is not, what's

the plan for it= ? Thank you all!


--0000000000002eac89058a8e806f--