incubator-drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Querying wide rows with Drill
Date Tue, 11 Nov 2014 20:37:52 GMT
On Tue, Nov 11, 2014 at 1:46 PM, Steven Phillips <sphillips@maprtech.com>
wrote:

> For this to really work well in your case, I think we need to be able to
> push the "mappify" operation into the scan. In other words, we need the
> hbase scan to ouptut the records in the desired key/value format.
> Currently, hbase scan will output in the normal, sparse column schema, and
> then a separate operator would convert it.
>
> One way to do this would be to write a new HBaseRecordReader that outputs
> in the key/value mode, and then have a System/session option to set which
> mode to use.
>

Actually, I think that what you suggest would be plenty fast even without
any special support in the DB layer.  The key limitation is rows per second
retrieved from the DB, not rows per second processed by drill.

THis is *very* exciting.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message