drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hanifi Gunes" <hgu...@maprtech.com>
Subject Re: Review Request 31809: DRILL-2193: Implement skip-all semantics for JSON reader
Date Fri, 06 Mar 2015 21:13:50 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31809/#review75545
-----------------------------------------------------------



exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
<https://reviews.apache.org/r/31809/#comment122718>

    this is not needed. will get rid of this.


- Hanifi Gunes


On March 6, 2015, 7:44 p.m., Jason Altekruse wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31809/
> -----------------------------------------------------------
> 
> (Updated March 6, 2015, 7:44 p.m.)
> 
> 
> Review request for drill and Jason Altekruse.
> 
> 
> Bugs: DRILL-2193
>     https://issues.apache.org/jira/browse/DRILL-2193
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> Currently all records will be read if we run a count(*) query. To avoid copying a lot
of data, a new concept of a skip-all has been added in DRILL-2358. In this mode of operation,
a reader will do the minimal amount of work to determine the number of records in the data
source. This patch implements this functionality for the JSON reader.While it is processing
raw text without meta-data, the whole file must be read from disk and analyzed to find the
number of records, but there is no need to copy any data into Drill's Value Vectors.
> 
> Patch authored by Hanifi Gunes.
> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/ColumnList.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
c343177 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonProcessor.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/BaseJsonProcessor.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/CountingJsonReader.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
cc5c8af 
> 
> Diff: https://reviews.apache.org/r/31809/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Jason Altekruse
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message