avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@cloudera.com>
Subject Re: Unable to read data from avro-tools
Date Fri, 10 Apr 2015 14:08:02 GMT
Hurm. So normally this error means that /tmp is mounted noexec.

Let's try treating it the same way, just in case Mint / Ubuntu is making it
noexec some other way.

The default approach when the Xerial Snappy library can't find the native
snappy implementation is to extract a copy of hte needed library into /tmp,
but you can configure where the extraction goes[1].

* If you don't mind changing where all java temporary file usage goes, set
the java.io.tempdir system property to somewhere that you can write that
allows executable loading (a directory you make in your home directory
hopefully will work).
* If you prefer a smaller scope of changes to java's behavior, you can set
the org.xerial.snappy.tempdir system property and Xerial will extract the
snappy native implementation there instead.

e.g. presuming the given directory exists ahead of the command:

 java -Dorg.xerial.snappy.tempdir=/home/dvasthimal/xerial-temp -jar
~/avro-tools-1.7.5.jar tojson ch200out99-r-00098.avro

See if that works.

-Sean

[1]:
https://github.com/xerial/snappy-java/blob/develop/src/main/java/org/xerial/snappy/SnappyLoader.java#L51

On Fri, Apr 10, 2015 at 8:19 AM, Marius <m_diec03@uni-muenster.de> wrote:

>  Not sure what yout meant, /tmp is not explicitly mounted in my fstab,
> however here is my whole fstab (with some comments deleted).
>
>
> # <file system> <mount point>   <type>  <options>       <dump>
 <pass>
> # / was on /dev/sda2 during installation
> UUID=f4fb2441-8735-4fda-8f1e-3d0866702d76 /               ext4
> errors=remount-ro 0       1
> # /boot/efi was on /dev/sda1 during installation
> UUID=30B4-854D  /boot/efi       vfat    defaults        0       1
> # swap was on /dev/sda3 during installation
> UUID=c927862d-b224-4cc2-ab3c-761efd0c86bf none            swap
> sw              0       0
>
>
> Grüße
>
> Marius
>
>
> Am 10.04.2015 um 15:09 schrieb Sean Busbey:
>
> please look in /etc/fstab,  find the line the mounts the /tmp directory
> and paste it here.
>
> On Fri, Apr 10, 2015 at 7:54 AM, Marius <m_diec03@uni-muenster.de> wrote:
>
>>  Sorry i forgot to append that...
>>
>> OS:  Linux Mint 17.1 x64, Cinnamon 2.4.7
>> Kernel: 3.13.0-37-generic
>> Permissions on /tmp are 777 owner and group are root
>>
>>
>>
>> Am 10.04.2015 um 14:49 schrieb Sean Busbey:
>>
>> What OS and version? What are the permissions on /tmp?
>>
>> --
>> Sean
>> On Apr 10, 2015 3:48 AM, "ÐΞ€ρ@Ҝ (๏̯͡๏)" <deepujain@gmail.com>
wrote:
>>
>>>  I tried with 1.7.5 and 1.7.7 and i see same exception. I deleted /tmp/
>>> snappy-1.0.5-libsnappyjava.so but it comes back and i see the same
>>> exception
>>>
>>>  Any suggestions ?
>>>
>>>
>>>
>>>  Logs
>>> ====
>>>  -sh-4.1$ ls -l ch200out99-r-00098.avro
>>> -rw-r--r-- 1 dvasthimal gid-dvasthimal 872131755 Apr 10 01:25
>>> ch200out99-r-00098.avro
>>> -sh-4.1$ java -jar ~/avro-tools-1.7.5.jar tojson ch200out99-r-00098.avro
>>> java.lang.reflect.InvocationTargetException
>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>  at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>  at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>  at java.lang.reflect.Method.invoke(Method.java:606)
>>>  at
>>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:322)
>>>  at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
>>>  at org.xerial.snappy.Snappy.<clinit>(Snappy.java:48)
>>>  at org.apache.avro.file.SnappyCodec.decompress(SnappyCodec.java:58)
>>>  at
>>> org.apache.avro.file.DataFileStream$DataBlock.decompressUsing(DataFileStream.java:343)
>>>  at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:199)
>>>  at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:63)
>>>  at org.apache.avro.tool.Main.run(Main.java:84)
>>>  at org.apache.avro.tool.Main.main(Main.java:73)
>>> Caused by: java.lang.UnsatisfiedLinkError: /tmp/
>>> snappy-1.0.5-libsnappyjava.so: /tmp/snappy-1.0.5-libsnappyjava.so:
>>> failed to map segment from shared object: Operation not permitted
>>>  at java.lang.ClassLoader$NativeLibrary.load(Native Method)
>>>  at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
>>>  at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
>>>  at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
>>>  at java.lang.Runtime.load0(Runtime.java:795)
>>>  at java.lang.System.load(System.java:1062)
>>>  at
>>> org.xerial.snappy.SnappyNativeLoader.load(SnappyNativeLoader.java:39)
>>>  ... 13 more
>>> Exception in thread "main" org.xerial.snappy.SnappyError:
>>> [FAILED_TO_LOAD_NATIVE_LIBRARY] null
>>>  at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:239)
>>>  at org.xerial.snappy.Snappy.<clinit>(Snappy.java:48)
>>>  at org.apache.avro.file.SnappyCodec.decompress(SnappyCodec.java:58)
>>>  at
>>> org.apache.avro.file.DataFileStream$DataBlock.decompressUsing(DataFileStream.java:343)
>>>  at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:199)
>>>  at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:63)
>>>  at org.apache.avro.tool.Main.run(Main.java:84)
>>>  at org.apache.avro.tool.Main.main(Main.java:73)
>>> -sh-4.1$ java -jar  ~/avro-tools-1.7.7.jar tojson
>>> ch200out99-r-00098.avro
>>> java.lang.reflect.InvocationTargetException
>>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>  at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>  at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>  at java.lang.reflect.Method.invoke(Method.java:606)
>>>  at
>>> org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:322)
>>>  at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:229)
>>>  at org.xerial.snappy.Snappy.<clinit>(Snappy.java:48)
>>>  at org.apache.avro.file.SnappyCodec.decompress(SnappyCodec.java:58)
>>>  at
>>> org.apache.avro.file.DataFileStream$DataBlock.decompressUsing(DataFileStream.java:352)
>>>  at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:199)
>>>  at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:77)
>>>  at org.apache.avro.tool.Main.run(Main.java:84)
>>>  at org.apache.avro.tool.Main.main(Main.java:73)
>>> Caused by: java.lang.UnsatisfiedLinkError: /tmp/
>>> snappy-1.0.5-libsnappyjava.so: /tmp/snappy-1.0.5-libsnappyjava.so:
>>> failed to map segment from shared object: Operation not permitted
>>>  at java.lang.ClassLoader$NativeLibrary.load(Native Method)
>>>  at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
>>>  at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
>>>  at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
>>>  at java.lang.Runtime.load0(Runtime.java:795)
>>>  at java.lang.System.load(System.java:1062)
>>>  at
>>> org.xerial.snappy.SnappyNativeLoader.load(SnappyNativeLoader.java:39)
>>>  ... 13 more
>>> Exception in thread "main" org.xerial.snappy.SnappyError:
>>> [FAILED_TO_LOAD_NATIVE_LIBRARY] null
>>>  at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:239)
>>>  at org.xerial.snappy.Snappy.<clinit>(Snappy.java:48)
>>>  at org.apache.avro.file.SnappyCodec.decompress(SnappyCodec.java:58)
>>>  at
>>> org.apache.avro.file.DataFileStream$DataBlock.decompressUsing(DataFileStream.java:352)
>>>  at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:199)
>>>  at org.apache.avro.tool.DataFileReadTool.run(DataFileReadTool.java:77)
>>>  at org.apache.avro.tool.Main.run(Main.java:84)
>>>  at org.apache.avro.tool.Main.main(Main.java:73)
>>> -sh-4.1$ java -version
>>> java version "1.7.0_60"
>>> Java(TM) SE Runtime Environment (build 1.7.0_60-b19)
>>> Java HotSpot(TM) 64-Bit Server VM (build 24.60-b09, mixed mode)
>>> -sh-4.1$
>>>
>>>  --
>>>  Deepak
>>>
>>>
>>
>
>
>  --
>  Sean
>
>
>


-- 
Sean

Mime
View raw message