phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-5082) Observations about the Phoenix-Omid integration
Date Thu, 27 Dec 2018 16:47:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729617#comment-16729617
] 

Lars Hofhansl edited comment on PHOENIX-5082 at 12/27/18 4:46 PM:
------------------------------------------------------------------

In UpsertCompiler I see this:
{code}
                // Commit a batch if auto commit is true and we're at our batch size
                if (isAutoCommit && rowCount % batchSize == 0) {
                    MutationState state = new MutationState(tableRef, mutation, 0, maxSize,
maxSizeBytes, connection);
                    connection.getMutationState().join(state);
                    connection.getMutationState().send();
                    mutation.clear();
                }
{code}

This is interesting now with transactions. The meaning of commit is different now.
With transaction I mostly want to have autocommit off, yet I'd want to send changes to HBase
as soon as a batch has filled up... When we have a separate transaction processor edits should
be send to HBase immediately when a batch has filled.
I don't know whether rollback would still work in that case... Need to think about this.

Update: I tried that. In that case commit does not work. We really need to rethink this! The
value of this is supposed to be that data can be written at full speed to HBase and does not
have to be buffered on the client, and just the commit stamp needs to be written. (I guess
at least we need to retain the information about the rows written so that we can write the
shadow cells).


was (Author: lhofhansl):
In UpsertCompiler I see this:
{code}
                // Commit a batch if auto commit is true and we're at our batch size
                if (isAutoCommit && rowCount % batchSize == 0) {
                    MutationState state = new MutationState(tableRef, mutation, 0, maxSize,
maxSizeBytes, connection);
                    connection.getMutationState().join(state);
                    connection.getMutationState().send();
                    mutation.clear();
                }
{code}

This is interesting now with transactions. The meaning of commit is different now.
With transaction I mostly want to have autocommit off, yet I'd want to send changes to HBase
as soon as a batch has filled up... When we have a separate transaction processor edits should
be send to HBase immediately when a batch has filled.
I don't know whether rollback would still work in that case... Need to think about this.


> Observations about the Phoenix-Omid integration
> -----------------------------------------------
>
>                 Key: PHOENIX-5082
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5082
>             Project: Phoenix
>          Issue Type: Task
>            Reporter: Lars Hofhansl
>            Assignee: Yonatan Gottesman
>            Priority: Major
>         Attachments: CommitTS-Sample.png, Omid-profile.png, Omid-sample.png
>
>
> Filing this issue to track my observations and perhaps some fixes.
> [~ohads]
> [~jamestaylor]
> [~yonigo]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message