phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5090) Discuss: Allow transactional writes without buffering the entire transaction on the client.
Date Sat, 05 Jan 2019 18:58:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734981#comment-16734981
] 

Lars Hofhansl commented on PHOENIX-5090:
----------------------------------------

Hmm... I forgot to mention that part. Thanks for pointing that out.

My thought is that we can (perhaps optionally) start with an optimistic assumption that the
transaction won't conflict or be rolled back. If now there is a conflict or rollback it'd
need to handled the same way as a client crashing before it can cleanup any partial writes.

Hmm again... Maybe it's not worth the effort. It just seems ... unfortunate ... that we have
a scalable database in the background but need to limit the size of any given transaction
to what can be buffered at the client.

Perhaps we need another setting to set the transactional batch-size...? 2MB are near perfect
for most networks. So we should keep that for non-transactional mutations. For transactions
we should allow increasing that so that larger transactions can be handled (without compromising
the non-transactional case)


> Discuss: Allow transactional writes without buffering the entire transaction on the client.
> -------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-5090
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5090
>             Project: Phoenix
>          Issue Type: Wish
>            Reporter: Lars Hofhansl
>            Priority: Major
>
> Currently it is not possible execute transactions in Phoenix that are too large to be
buffered entirely on the client.
> Both Tephra and Omid support writing uncommitted data to HBase immediately and at full
speed. The client still needs to keep tracks of the rows changes for:
> # Conflict detection
> # (for Omid) writing the shadow cells
> I'd like to do some brainstorming here.
> * It should *always* be enough to only hold on to the changed rows (and columns?) only
for _conflict resolution_ and free the rest from the client as soon as the uncommitted data
is written to HBase.
> * For the shadows cells we need only keep the rows changed, right?
> * There are situations where we can avoid the client site buffering entirely (perhaps
only for Tephra) when we declare a table or upsert not to participate in conflict resolution.
> [~tdsilva], [~ohads], [~yonigo], [~jamestaylor], [~vincentpoon], more, better ideas?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message