kudu-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: kudu insert select
Date Tue, 26 Apr 2016 05:55:25 GMT
On Mon, Apr 25, 2016 at 5:44 PM, 李书明 <18769721812@163.com> wrote:

>
> Hi,
>
> You said 112 hash bucket, but the amount of data is one day the increment
> of about 100 million, to about two years of data, the total data volume of
> about 7 100 million, so query speed too slow.
>

Why is the query speed slow? What tests did you perform? Having more than
10-20 tablets per server should not show a big speed improvement.


> There is an asynchronous insert is always lost data, synchronous insertion
> is no problem, the log has not been reported to the wrong.
>

Can you share the code you are using for async insert? Maybe you are not
properly checking errors?

-Todd


> Thanks.
>
>
>
>
>
>
>
> Hi, in particular you may have too many hash buckets.  Try creating the
> table with more like 112 hash buckets, and see if insert performance
> improves.
>
> - Dan
>
> On Mon, Apr 25, 2016 at 10:19 AM, Dan Burkert <dan@cloudera.com> wrote:
>
> > Hi,
> >
> > On Mon, Apr 25, 2016 at 10:14 AM, Misty Stanley-Jones <
> > mstanleyjones@cloudera.com> wrote:
> >
> >> This is one of our most frequently asked questions. Make sure that your
> >> table is created with a schema that will spread the data evenly among
> >> tablets and make sure that you have a number of tablets that is a multiple
> >> of the number of tablet servers. See
> >> http://getkudu.io/docs/schema_design.html and
> >> http://getkudu.io/docs/kudu_impala_integration.html#kudu_impala_create_table
> >> .
> >>
> >> Thanks,
> >> Misty
> >>
> >> On Sun, Apr 24, 2016 at 11:38 PM, 李书明 <18769721812@163.com> wrote:
> >>
> >>> HI
> >>> 14 nodes with API Java insertion speed is only 3000 per second, how to
> >>> improve the insertion rate?
> >>>
> >>> Create table use DISTRIBUTE BY HASH (id) INTO 5000 BUCKETS, error is “kuduRpc
> method=IsCreateTableDone timeout=10000”,How to solve
> >>>
> >>>
> >>> thanks!
>
>
>
>
>
>
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message