arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@apache.org>
Subject Re: Arrow/Gandiva dependency management in Java
Date Tue, 05 Jan 2021 17:07:27 GMT
If you're up for troubleshooting, do you want to try the near master jar we
use inside of Dremio? I think we're the primary users of the Gandiva Java
bindings and I wonder if the Apache build has an issue with artifact
packaging(?). The jar we use in Dremio is used at hundreds of customers so
it should work without extra items. You can find the reference in the pom
here:

https://github.com/dremio/dremio-oss/blob/master/pom.xml#L41

I think you'll need to add our public repo to get the jar. I think it is
this repo:
https://github.com/dremio/dremio-oss/blob/master/pom.xml#L3243

I'm only suggesting this to see if there is some kind of disconnect in
keeping the Apache artifact complete. If our jar works for you we'll need
to figure out what is different between it and the Apache one. (For
reference, in general our jars are slightly off master to pull forward or
backward certain bugs, etc.)



On Tue, Jan 5, 2021 at 8:55 AM Michael Mior <mmior@apache.org> wrote:

> Yes, I'm using the latest released version (2.0.0).
> --
> Michael Mior
> mmior@apache.org
>
> Le mar. 5 janv. 2021 à 11:45, Jacques Nadeau <jacques@apache.org> a écrit
> :
> >
> > Hey Michael, are you using the released versions of these Arrow
> artifacts? I recall there being complexity to building the
> multi-native-binary jar file...
> >
> > On Tue, Jan 5, 2021 at 8:11 AM Michael Mior <mmior@apache.org> wrote:
> >>
> >> I am working on integrating Arrow and Gandiva with the Apache Calcite
> >> query processing framework so Calcite is able to process SQL queries
> >> using Arrow data sources. The basics are working well but I'm having
> >> trouble getting a reproducible development environment set up.
> >>
> >> For example, on a Linux Travis CI right now, I get the following error:
> >>
> >> java.lang.RuntimeException: libgandiva_jni.so was not found inside JAR.
> >>
> >> I get something similar with AppVeyor on Windows:
> >>
> >> java.lang.RuntimeException: gandiva_jni.dll was not found inside JAR.
> >>
> >> Starting with a fresh clone of the repository, on my macOS machine I
> >> use for development, I get the following which seems to be related to
> >> a protobuf version mismatch:
> >>
> >> java.lang.UnsatisfiedLinkError:
> >>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392:
> >>
> dlopen(/private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392,
> >> 1): Library not loaded:
> >> /usr/local/opt/protobuf/lib/libprotobuf.24.dylib
> >>       Referenced from:
> >>
> /private/var/folders/0h/n0jrz4qn5x37gj25sxx4z2wm0000gp/T/libgandiva_jni.dylib43feded5-f265-4a6d-b1f7-2aa56e94f392
> >>
> >> I have dependencies declared on arrow-memory-netty, arrow-vector, and
> >> arrow-gandiva but this doesn't seem to be sufficient. Any ideas what
> >> I'm missing here?
> >>
> >> --
> >> Michael Mior
> >> mmior@apache.org
>

Mime
View raw message