drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neeraja (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-2783) Abbreviated Flatten syntax for multi-level nested elements
Date Tue, 14 Apr 2015 18:08:13 GMT
Neeraja created DRILL-2783:

             Summary: Abbreviated Flatten syntax for multi-level nested elements
                 Key: DRILL-2783
                 URL: https://issues.apache.org/jira/browse/DRILL-2783
             Project: Apache Drill
          Issue Type: Improvement
          Components: Storage - JSON
    Affects Versions: 0.7.0
            Reporter: Neeraja
            Assignee: Steven Phillips
             Fix For: Future

The flatten function in Drill works great for any level of nesting, however the applying nested
flatten can complicate query syntax as the # of nested levels increase due to the need to
use subqueries, especially when the structure contains maps of repeated lists.

Consider JSON below

Below is a query we need to write to flatten the 3 levels.

select count(t2.td.id3) from 
         (select flatten(t1.tp.level3) as td from 
                  (select flatten(t.ts.level2) as tp from 
                      (select flatten(level1) as ts from    dfs.`default`.`/Users/nrentachintala/Downloads/flattenex.json`)
as t) as t1) as t2;

An abbreviated/intuitive syntax to apply flatten on maps with arrays would make working with
this kind of data much easier. 
For ex: 
The above query could potentially be rewritten as below.
select flatten(flatten(flatten(level1).level2).level3) as ts from    dfs.`default`.`/Users/nrentachintala/Downloads/flattenex.json`

This message was sent by Atlassian JIRA

View raw message