orc-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <omal...@apache.org>
Subject Re: Setting Null correctly
Date Sat, 25 Feb 2017 21:02:21 GMT
On Sat, Feb 25, 2017 at 9:39 AM, Telco Phone <telco5@yahoo.com> wrote:

> Give the code here I am trying to find the correct way to set null to
> various vectors
>
> In the case of Long or Bytes vectors, how do you correctly set nulls ?
>
> Lines in question are
>
> col1.isNull[4] = Boolean.TRUE;    <--- does not set to null but sets to 0
> in output
> col2.isNull[4] = Boolean.TRUE;   <--- throws error on write
>

It is easier to use "true" instead of "Boolean.TRUE":
col1.isNull[4] = true;
col2.isNull[4] = true;

You also need to set ColumnVector.noNulls
http://orc.apache.org/api/hive-storage-api/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.html#noNulls
to false:

col1.noNulls = false;
col2.noNulls = false;

.. Owen


>
>
> Thanks in advance
>
> void example() {
>
> String s = "struct<_col0:bigint,_col1:string>";
>
>         TypeDescription schema = TypeDescription.fromString(s);
>
>
>         // Build col0
>
>         LongColumnVector col1 = new LongColumnVector(5);
>         col1.init();
>         col1.vector[0] = 9L;
>         col1.vector[1] = 9L;
>         col1.vector[2] = 9L;
>         col1.vector[3] = 9L;
>         col1.isNull[4] = Boolean.TRUE;
>
>
>         // Build col1
>
>         BytesColumnVector col2 = new BytesColumnVector();
>         col2.init();
>         col2.initBuffer();
>
>         byte[] byteString = "Test0".getBytes();
>         col2.setVal(0, byteString, 0, byteString.length);
>
>         byteString = "Test1".getBytes();
>         col2.setVal(1, byteString, 0, byteString.length);
>
>         byteString = "Test2".getBytes();
>         col2.setVal(2, byteString, 0, byteString.length);
>
>         byteString = "Test3".getBytes();
>         col2.setVal(3, byteString, 0, byteString.length);
>
>         byteString = null;
>
>         col2.isNull[4] = Boolean.TRUE;
>
>
>         VectorizedRowBatch batch = schema.createRowBatch();
>         batch.cols[0] = col1;
>         batch.cols[1] = col2;
>
>         batch.size=5;
>
>
>         try {
>             File f = new File("/tmp/my-file.orc");
>             f.delete();
>
>             Configuration conf = new Configuration();
>             Writer writer = OrcFile.createWriter(new
> Path("/tmp/my-file.orc"), OrcFile.writerOptions(conf).setSchema(schema));
>             writer.addRowBatch(batch);
>             writer.close();
>
>
>
>         } catch (Exception e) {
>             e.printStackTrace();
>         }
> }
>

Mime
View raw message