drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sudip Mukherjee <smukher...@commvault.com>
Subject RE: Drill Sql Max row size
Date Tue, 08 Sep 2015 14:53:52 GMT
Yes, there is no problem in that . Was just wondering if there is a mechanism of getting data
by batches from the record reader. 
Actually , I wrote a basic plugin to solr and was thinking if I paginate or use cursormark
params in the recordreader.

Thanks,
Sudip

-----Original Message-----
From: Tomer Shiran [mailto:tshiran@dremio.com] 
Sent: 08 September 2015 PM 08:16
To: dev@drill.apache.org
Subject: Re: Drill Sql Max row size

That won't be a problem. There's actually no limit on how many records/rows you can have.
 

> On Sep 8, 2015, at 2:00 AM, Sudip Mukherjee <smukherjee@commvault.com> wrote:
> 
> Hi,
> I have somewhere around a million records. But columns are less in 
> numbers (up to 10)
> 
> Thanks,
> Sudip
> 
> -----Original Message-----
> From: Jacques Nadeau [mailto:jacques@dremio.com]
> Sent: 08 September 2015 AM 06:58
> To: dev@drill.apache.org
> Subject: Re: Drill Sql Max row size
> 
> Generally, no. That being said, Drill will probably struggle if you start reading records
where one or more cells is greater than a few hundred kilobytes (or mbs) or more than several
hundred columns/fields. What size records are you working with?
> 
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
> 
> On Mon, Sep 7, 2015 at 4:49 AM, Sudip Mukherjee 
> <smukherjee@commvault.com>
> wrote:
> 
>> Hi Devs,
>> 
>> Is there a max row limit which I can configure while pulling data 
>> from underlying datasource? If there is a large data-set would drill 
>> fetch like page by page?
>> 
>> Thanks,
>> Sudip
>> 
>> 
>> 
>> ***************************Legal 
>> Disclaimer***************************
>> "This communication may contain confidential and privileged material 
>> for the sole use of the intended recipient. Any unauthorized review, 
>> use or distribution by others is strictly prohibited. If you have 
>> received the message by mistake, please advise the sender by reply 
>> email and delete the message. Thank you."
>> *********************************************************************
>> *
> 
> 
> 
> ***************************Legal Disclaimer***************************
> "This communication may contain confidential and privileged material 
> for the sole use of the intended recipient. Any unauthorized review, 
> use or distribution by others is strictly prohibited. If you have 
> received the message by mistake, please advise the sender by reply email and delete the
message. Thank you."
> **********************************************************************



***************************Legal Disclaimer***************************
"This communication may contain confidential and privileged material for the
sole use of the intended recipient. Any unauthorized review, use or distribution
by others is strictly prohibited. If you have received the message by mistake,
please advise the sender by reply email and delete the message. Thank you."
**********************************************************************

Mime
View raw message