incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varadharajan M <varad...@thoughtworks.com>
Subject Re: Usage of 'store' operation in Logical Queries
Date Thu, 24 Oct 2013 08:01:35 GMT
Hi Jason,

Thanks for your reply*. *
I indeed want to contribute to Drill. If there are any recommendations on
which i could work on (Especially low hanging fruits that would let me gain
better understanding of the system), let me know. I'm currently just
exploring the source tree of Drill.

Regards,
Varadha


On Sun, Oct 20, 2013 at 9:54 AM, Jason Altekruse
<altekrusejason@gmail.com>wrote:

> Hello Varadha,
>
> I am actually working on the writer interface for Drill right now.
> Unfortunately we currently only have reading implementations for JSON and
> Parquet and neither of those storage engines can export data.
>
> I am working on designing a unified reader/writer interface that will
> enable us to add more formats faster, while simultaneously adding read and
> write support with each implementation.
>
> I will be posting my patches to the list for review soon, I invite you to
> take a look if you have an interest in Drill from a development
> perspective.
>
> Regards,
> Jason Altekruse
>
>
> On Sat, Oct 19, 2013 at 1:06 PM, Varadharajan M
> <varadham@thoughtworks.com>wrote:
>
> > Hi All,
> >
> > I've a logical plan, thats intended to read from a json file and write
> the
> > output to another file. I was under the impression that the write
> operation
> > could be done with the help of 'store' operation. Here is my logical
> plan:
> >
> > {
> >   "head" : {
> >     "type" : "APACHE_DRILL_LOGICAL",
> >     "version" : "1",
> >     "generator" : {
> >     "type" : "optiq",
> >     "info" : "na"
> >     }
> >   },
> >
> >   "storage" : {
> >     "jsonl" : {
> >       "type" : "json",
> >       "dfsName" : "file:///"
> >     },
> >     "filel" : {
> >       "type" : "fs",
> >       "root" : "file:///"
> >     }
> >   },
> >
> >   "query" : [ {
> >     "op" : "scan",
> >     "memo" : "initial_scan",
> >     "ref" : "_MAP",
> >     "storageengine" : "jsonl",
> >     "selection" : [ {
> >       "path" : "/tmp/students.json"
> >     } ],
> >     "@id" : 1
> >   },
> >
> >   {
> >     "op" : "store",
> >     "input" : 1,
> >     "memo" : "output sink",
> >     "storageengine": "filel",
> >     "@id" : 2,
> >      target: {
> >         file: "file:///tmp/do.json"
> >       }
> >   } ]
> > }
> >
> >
> > Now, if i try to run this logical plan with the submit_plan shell
> script, i
> > get the output to the console rather than to the specified file (That is,
> > there is no data written to the file). Am i missing anything here?
> >
> > - Varadha
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message