phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-5082) Observations about the Phoenix-Omid integration
Date Thu, 27 Dec 2018 11:59:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729548#comment-16729548
] 

Lars Hofhansl edited comment on PHOENIX-5082 at 12/27/18 11:58 AM:
-------------------------------------------------------------------

Hi [~yonigo], thanks for looking.

Here's how I am testing:
 # Single node setup. I run a single Namenode, Datanode, HMaster, HRegionServer, ZK QuorumPeer,
and TSO on a single machine (since it's almost only the region server using CPU and IOPS this
should be a valid setup).
 # Set up a table in Phoenix like this: {{create table test (pk1 integer not null, pk2 integer
not null, pk3 integer not null, v1 float, v2 float, v3 integer CONSTRAINT pk PRIMARY KEY (pk1,
pk2, pk3)) DISABLE_WAL=true, TRANSACTIONAL=true;}}
 # Seed the table: {{upsert into test values(rand()*10000000, rand()*10000000, rand()*10000000,
rand(), rand(), rand()*1000000);}}
 # The repeatedly: {{upsert into test select rand()*10000000, rand()*10000000, rand()*10000000,
rand(), rand(), rand()*1000000 from test;}}
 # Config for the TSO: {{waitStrategy: LOW_CPU}}

I filed OMID-122, OMID-123, OMID-124. Feel like having a look? :)

In order to isolate this to the actual commit, I turned autocommit off in SQLLine (!autocommit
off). Then issued a manual commit, it seems the time is spent writing the shadow columns,
but I am not 100% sure.

The behavior is interesting. It takes 1.2s to upsert 256 rows, and 2.4s to upsert 512 rows,
so something does not scale right in the number of rows in Phoenix/Omid.

Oh, profiling... I used jvisualvm ([https://visualvm.github.io/] if your JVM doesn't have
it.)


was (Author: lhofhansl):
Hi [~yonigo], thanks for looking.

Here's how I am testing:
 # Single node setup. I run a single Namenode, Datanode, HMaster, HRegionServer, ZK QuorumPeer,
and TSO on a single machine (since it's almost only the region server using CPU and IOPS this
should be a valid setup).
 # Set up a table in Phoenix like this: {{create table test (pk1 integer not null, pk2 integer
not null, pk3 integer not null, v1 float, v2 float, v3 integer CONSTRAINT pk PRIMARY KEY (pk1,
pk2, pk3)) DISABLE_WAL=true, TRANSACTIONAL=true;}}
 # Seed the table: {{upsert into test values(rand()*10000000, rand()*10000000, rand()*10000000,
rand(), rand(), rand()*1000000);}}
 # The repeatedly: {{upsert into test select rand()*10000000, rand()*10000000, rand()*10000000,
rand(), rand(), rand()*1000000 from test;}}
 # Config for the TSO: {{waitStrategy: LOW_CPU}}

I filed OMID-122, OMID-123, OMID-124. Feel like having a look? :)

In order to isolate this to the actual commit, I turned autocommit off in SQLLine (!autocommit
off). Then issued a manual commit, it seems the time is spent writing the shadow columns,
but I am not 100% sure.

The behavior is interesting. It takes 1.2s to upsert 256 rows, and 2.4s to upsert 512 rows,
so something does not scale right in the number of rows in Phoenix/Omid.

> Observations about the Phoenix-Omid integration
> -----------------------------------------------
>
>                 Key: PHOENIX-5082
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5082
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: Lars Hofhansl
>            Priority: Major
>         Attachments: Omid-profile.png, Omid-sample.png
>
>
> Filing this issue to track my observations and perhaps some fixes.
> [~ohads]
> [~jamestaylor]
> [~yonigo]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message