drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5919) Add non-numeric support for JSON processing
Date Fri, 10 Nov 2017 23:49:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16248187#comment-16248187
] 

ASF GitHub Bot commented on DRILL-5919:
---------------------------------------

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1026#discussion_r150359625
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/conv/JsonConvertTo.java
---
    @@ -90,7 +91,71 @@ public void eval(){
     
           java.io.ByteArrayOutputStream stream = new java.io.ByteArrayOutputStream();
           try {
    -        org.apache.drill.exec.vector.complex.fn.JsonWriter jsonWriter = new org.apache.drill.exec.vector.complex.fn.JsonWriter(stream,
true, true);
    +        org.apache.drill.exec.vector.complex.fn.JsonWriter jsonWriter = new org.apache.drill.exec.vector.complex.fn.JsonWriter(stream,
true, true, false);
    +
    +        jsonWriter.write(input);
    +      } catch (Exception e) {
    +        throw new RuntimeException(e);
    +      }
    +
    +      byte [] bytea = stream.toByteArray();
    +
    +      out.buffer = buffer = buffer.reallocIfNeeded(bytea.length);
    +      out.buffer.setBytes(0, bytea);
    +      out.end = bytea.length;
    +    }
    +  }
    +
    +  @FunctionTemplate(names = { "convert_toJSON", "convert_toSIMPLEJSON" } , scope = FunctionScope.SIMPLE,
nulls = NullHandling.NULL_IF_NULL)
    +  public static class ConvertToJsonNonNumeric implements DrillSimpleFunc{
    +
    +    @Param FieldReader input;
    +    @Param BitHolder nonNumeric;
    +    @Output VarBinaryHolder out;
    +    @Inject DrillBuf buffer;
    +
    +    public void setup(){
    +    }
    +
    +    public void eval(){
    +      out.start = 0;
    +
    +      java.io.ByteArrayOutputStream stream = new java.io.ByteArrayOutputStream();
    +      try {
    +        org.apache.drill.exec.vector.complex.fn.JsonWriter jsonWriter = new org.apache.drill.exec.vector.complex.fn.JsonWriter(stream,
true, false,
    --- End diff --
    
    More copies.


> Add non-numeric support for JSON processing
> -------------------------------------------
>
>                 Key: DRILL-5919
>                 URL: https://issues.apache.org/jira/browse/DRILL-5919
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - JSON
>    Affects Versions: 1.11.0
>            Reporter: Volodymyr Tkach
>            Assignee: Volodymyr Tkach
>              Labels: doc-impacting
>             Fix For: Future
>
>
> Add session options to allow drill working with non standard json strings number literals
like: NaN, Infinity, -Infinity. By default these options will be switched off, the user will
be able to toggle them during working session.
> *For documentation*
> 1. Added two session options {{store.json.reader.non_numeric_numbers}} and {{store.json.reader.non_numeric_numbers}}
that allow to read/write NaN and Infinity as numbers. By default these options are set to
false.
> 2. Extended signature of {{convert_toJSON}} and {{convert_fromJSON}} functions by adding
second optional parameter that enables read/write NaN and Infinity.
> For example:
> {noformat}
> select convert_fromJSON('{"key": NaN}') from (values(1)); will result with JsonParseException,
but
> select convert_fromJSON('{"key": NaN}', true) from (values(1)); will parse NaN as a number.
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message