hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Demeter Sztanko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6347) ZeroCopy read path for ORC RecordReader
Date Wed, 20 Jan 2016 21:03:40 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15109401#comment-15109401
] 

Demeter Sztanko commented on HIVE-6347:
---------------------------------------

Thanks, [~gopalv],
Seems like tasks are not able to locate the libhadoop, indeed. However, they were able to
do that when zerocopy was disabled (I am using snappy compression in my orc tables) and perform
well. libhadoop.so and libsnappy are there and I never had any problem with those. This makes
me think that there is a correlation between finding native libraries and zerocopy.


> ZeroCopy read path for ORC RecordReader
> ---------------------------------------
>
>                 Key: HIVE-6347
>                 URL: https://issues.apache.org/jira/browse/HIVE-6347
>             Project: Hive
>          Issue Type: Bug
>          Components: File Formats
>    Affects Versions: tez-branch
>            Reporter: Gopal V
>            Assignee: Gopal V
>             Fix For: tez-branch
>
>         Attachments: HIVE-6347.1.patch, HIVE-6347.2-tez.patch, HIVE-6347.3-tez.patch,
HIVE-6347.4-tez.patch, HIVE-6347.5-tez.patch
>
>
> ORC can use the new HDFS Caching APIs and the ZeroCopy readers to avoid extra data copies
into memory while scanning files.
> Implement ORC zcr codepath and a hive.orc.zerocopy flag.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message