drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha" <asi...@maprtech.com>
Subject Re: Review Request 32223: DRILL-2193: implement fast count / skip-all semantics for JSON reader
Date Thu, 19 Mar 2015 07:40:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32223/#review77027
-----------------------------------------------------------



exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonProcessor.java
<https://reviews.apache.org/r/32223/#comment124816>

    Is it necessary to have this method in the interface ?  The name suggests that implementors
should ensure at least 1 field/column but the counting reader does not actually do that.



exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/CountingJsonReader.java
<https://reviews.apache.org/r/32223/#comment124819>

    The default JsonReader (which is used when skip-all is false) has a initial while loop
to iterate over the tokens; is that not needed here because you are expecting to be either
at end-of-stream or at the beginning of a record ? I am wondering what happens where a single
large record (with either many fields or a large string field) spans across batch boundary.
(I am actually not completely sure if that is allowed, so let me know if that situation is
not going to occur).


- Aman Sinha


On March 18, 2015, 11:34 p.m., Hanifi Gunes wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32223/
> -----------------------------------------------------------
> 
> (Updated March 18, 2015, 11:34 p.m.)
> 
> 
> Review request for drill, Aman Sinha and Parth Chandra.
> 
> 
> Bugs: DRILL-2193
>     https://issues.apache.org/jira/browse/DRILL-2193
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> DRILL-2193: implement fast count / skip-all semantics for JSON reader
> 
> This patch introduces an abstraction for JSON processing and implements a efficient counting
JSON reader if query is in skip-all mode(see DRILL-2358).
> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
c343177a719b5f36f51bcb2f84d68518ba1ae02f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonProcessor.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/BaseJsonProcessor.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/CountingJsonReader.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
cc5c8af63c6383eb8d2e28a409a3c055bf5cc737 
> 
> Diff: https://reviews.apache.org/r/32223/diff/
> 
> 
> Testing
> -------
> 
> unit + regression
> 
> 
> Thanks,
> 
> Hanifi Gunes
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message