phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5090) Discuss: Allow transactional writes without buffering the entire transaction on the client.
Date Tue, 09 Apr 2019 09:08:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813154#comment-16813154
] 

Lars Hofhansl commented on PHOENIX-5090:
----------------------------------------

Hah. The same happens with default Phoenix (without this patch too).
 Steps to reproduce:
 # {{!autocommit off}}
 # {{create table test (pk1 integer not null, pk2 integer not null, pk3 integer not null,
v1 float, v2 float, v3 integer CONSTRAINT pk PRIMARY KEY (pk1, pk2, pk3)) DISABLE_WAL=true,
TRANSACTIONAL=true;}}
 # {{upsert into test values(rand()*10000000, rand()*10000000, rand()*10000000, rand(), rand(),
rand()*1000000);}}
 # {{upsert into test select rand()*10000000, rand()*10000000, rand()*10000000, rand(), rand(),
rand()*1000000 from test;}}
 # {{select count\(*) from test; – this will cause uncommitted data to sent to the server.}}
 # Goto #4 a few time (until you inserted 131072 rows)
 # {{!commit}}

In a separate sqlline session just repeat after the commit was issued in the other session.
* {{select count\(*) from test;}}

You'll see that number will change until it finally settles.

> Discuss: Allow transactional writes without buffering the entire transaction on the client.
> -------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5090
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5090
>             Project: Phoenix
>          Issue Type: Wish
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Major
>         Attachments: 5090-looksee.txt, 5090-v1.txt, 5090-v2.txt, 5090-v3.txt
>
>
> Currently it is not possible execute transactions in Phoenix that are too large to be
buffered entirely on the client.
> Both Tephra and Omid support writing uncommitted data to HBase immediately and at full
speed. The client still needs to keep tracks of the rows changes for:
> # Conflict detection
> # (for Omid) writing the shadow cells
> I'd like to do some brainstorming here.
> * It should *always* be enough to only hold on to the changed rows (and columns?) only
for _conflict resolution_ and free the rest from the client as soon as the uncommitted data
is written to HBase.
> * For the shadows cells we need only keep the rows changed, right?
> * There are situations where we can avoid the client site buffering entirely (perhaps
only for Tephra) when we declare a table or upsert not to participate in conflict resolution.
> [~tdsilva], [~ohads], [~yonigo], [~jamestaylor], [~vincentpoon], more, better ideas?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message