orc-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From István <lecc...@gmail.com>
Subject Re: Writing ORC files without HDFS
Date Wed, 13 Jan 2016 13:48:31 GMT
Ravi,

one final question: which package should I import into my project?

I found this as the best candidate:

<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>1.2.0</version>
</dependency

Thanks,
Istvan

On Fri, Jan 8, 2016 at 8:38 AM, Ravi Tatapudi <ravi_tatapudi@in.ibm.com>
wrote:

> Hello,
>
> You can write ORC-files on local-filesystem, by getting the local
> "FileSystem" object, as "FileSystem.getLocal(conf)". Pl. find below the
> simple-example given below & see if it works for your requirement.
>
> =================================================
> public class orcw {
>
>         private static Configuration conf = new Configuration();
>         public static Writer writer ;
>
> public static class OrcRow
> {
>         int col1 ;
>         String col2 ;
>         String col3 ;
>
>         OrcRow(int a, String b, String c) {
>         this.col1  = a ;
>         this.col2  = b ;
>         this.col3  = c ;
>         }
> }
>
> public static void main(String[] args) throws IOException,
>
>     InterruptedException, ClassNotFoundException {
>
>             String path = "/tmp/orcfile1";
>
>             try {
>
>             conf = new Configuration();
>         FileSystem fs = FileSystem.getLocal(conf);
>
>             ObjectInspector ObjInspector =
> ObjectInspectorFactory.getReflectionObjectInspector(OrcRow.class,
> ObjectInspectorFactory.ObjectInspectorOptions.JAVA);
>             writer = OrcFile.createWriter(new Path(path),
> OrcFile.writerOptions(conf).inspector(ObjInspector).stripeSize(100000).bufferSize(10000).compress(CompressionKind.ZLIB).version(OrcFile.Version.V_0_12));
>
>         writer.addRow(new OrcRow(1,"hello","orcFile")) ;
>         writer.addRow(new OrcRow(2,"hello2","orcFile2")) ;
>
>             writer.close();
>             }
>             catch (Exception e)
>             {
>                     e.printStackTrace();
>             }
>     }
> }
> =================================================
>
> Thanks,
>  Ravi
>
>
>
> From:        István <leccine@gmail.com>
> To:        user@orc.apache.org
> Date:        01/08/2016 01:53 AM
> Subject:        Writing ORC files without HDFS
> ------------------------------
>
>
>
> Hi all,
>
> I am working on a project that requires me to write ORC files locally on a
> non-HDFS location. I was wondering if there is any project doing something
> similar, but I guess there is none, after spending some time on Google.
>
> I think what needs to get done is to re-implement the ORC Writer, sort of
> similar to the following but leaving out Hadoop:
>
>
> *https://github.com/apache/hive/blob/master/orc/src/java/org/apache/orc/impl/WriterImpl.java*
> <https://github.com/apache/hive/blob/master/orc/src/java/org/apache/orc/impl/WriterImpl.java>
>
> Am I on the right track implementing this?
>
> Let me know if you have any suggestions or links in the subject.
>
> Thank you very much,
> Istvan
>
> --
> the sun shines for all
>
>
>
>


-- 
the sun shines for all

Mime
View raw message