phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3778) Local Index - Writes get exponentially slower as batch size increases
Date Mon, 24 Apr 2017 18:02:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15981591#comment-15981591
] 

James Taylor commented on PHOENIX-3778:
---------------------------------------

Duplicate of PHOENIX-3796. If writes are still exponentially slower, please reopen, [~mujtabachohan].

> Local Index - Writes get exponentially slower as batch size increases
> ---------------------------------------------------------------------
>
>                 Key: PHOENIX-3778
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3778
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.10.0
>            Reporter: Mujtaba Chohan
>
> {noformat}
> CREATE TABLE IF NOT EXISTS T (
>     PK_PART_A CHAR(15) NOT NULL,
>     PK_PART_B CHAR(3) NOT NULL,
>     PK_PART_C CHAR(15) NOT NULL,
>     PK_PART_DATE DATE NOT NULL,
>     PK_PART_LAST CHAR(15) NOT NULL,
>     VAL_ID CHAR(15), 
>     CONSTRAINT PK PRIMARY KEY 
>     (
>         PK_PART_A, 
>         PK_PART_B,
>         PK_PART_C,
>         PK_PART_DATE,
>         PK_PART_LAST
>     )
> ) IMMUTABLE_ROWS=true
> CREATE LOCAL INDEX IF NOT EXISTS T_IDX ON T (PK_PART_DATE)
> {noformat}
> With HBase 0.98.23 and head of 4.x. writes to local index gets exponentially slower as
batch size increases compared to global index when data is upserted to base table with index.

> Following is the average time per batch comparing local vs global index. Tested on cluster
with 12GB heap per region server and also on single node machine with 4GB heap. Memory usage
was within limits with minimal GC activity on client and server. [~samarthjain] and I looked
for anything obvious for CPU in jvisualvm but nothing popped up.
> \\
> | *Batch Size* | *Local Index* time in ms per batch | *Global Index* time in ms per batch
|
> | 10         | 15                                 | 11                              
   |
> | 100        | 175                                | 15                              
   |
> | 500        | 3500                               | 50                              
   |
> | 1000       | 150000                             | 95                              
   |
> \\
> [~rajeshbabu]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message