phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4130) Avoid server retries for mutable indexes
Date Fri, 26 Jan 2018 21:29:01 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341621#comment-16341621
] 

ASF GitHub Bot commented on PHOENIX-4130:
-----------------------------------------

Github user vincentpoon commented on a diff in the pull request:

    https://github.com/apache/phoenix/pull/290#discussion_r164226421
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/IndexWriterUtils.java
---
    @@ -70,13 +70,13 @@
        public static final String HTABLE_KEEP_ALIVE_KEY = "hbase.htable.threads.keepalivetime";
     
        public static final String INDEX_WRITER_RPC_RETRIES_NUMBER = "phoenix.index.writes.rpc.retries.number";
    -   /**
    -    * Based on the logic in HBase's AsyncProcess, a default of 11 retries with a pause
of 100ms
    -    * approximates 48 sec total retry time (factoring in backoffs).  The total time should
be less
    -    * than HBase's rpc timeout (default of 60 sec) or else the client will retry before
receiving
    -    * the response
    -    */
    -   public static final int DEFAULT_INDEX_WRITER_RPC_RETRIES_NUMBER = 11;
    +    /**
    +     * Retry server-server index write rpc only once, and let the client retry the data
write
    +     * instead to avoid typing up the handler
    +     */
    +   // note in HBase 2+, numTries = numRetries + 1
    +   // in prior versions, numTries = numRetries
    +   public static final int DEFAULT_INDEX_WRITER_RPC_RETRIES_NUMBER = 1;
    --- End diff --
    
    I don't think we need any code change if we're relying on the operator.  As it is, we
check the config INDEX_WRITER_RPC_RETRIES_NUMBER, and only if it's not there, we use the DEFAULT_INDEX_WRITER_RPC_RETRIES_NUMBER,
which is now 1.  So we assume the operator will have the property set to 11 until all clients
are upgraded, at which point the remove the property and it uses the default of 1.


> Avoid server retries for mutable indexes
> ----------------------------------------
>
>                 Key: PHOENIX-4130
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4130
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Lars Hofhansl
>            Assignee: Vincent Poon
>            Priority: Major
>             Fix For: 4.14.0
>
>         Attachments: PHOENIX-4130.v1.master.patch, PHOENIX-4130.v2.master.patch, PHOENIX-4130.v3.master.patch
>
>
> Had some discussions with [~jamestaylor], [~samarthjain], and [~vincentpoon], during
which I suggested that we can possibly eliminate retry loops happening at the server that
cause the handler threads to be stuck potentially for quite a while (at least multiple seconds
to ride over common scenarios like splits).
> Instead we can do the retries at the Phoenix client that.
> So:
> # The index updates are not retried on the server. (retries = 0)
> # A failed index update would set the failed index timestamp but leave the index enabled.
> # Now the handler thread is done, it throws an appropriate exception back to the client.
> # The Phoenix client can now retry. When those retries fail the index is disabled (if
the policy dictates that) and throw the exception back to its caller.
> So no more waiting is needed on the server, handler threads are freed immediately.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message