phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Samarth Jain (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3994) Index RPC priority still depends on the controller factory property in hbase-site.xml
Date Thu, 06 Jul 2017 20:47:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077183#comment-16077183
] 

Samarth Jain commented on PHOENIX-3994:
---------------------------------------

bq. Index updates that happen to be local bypass the RPC stack entirely.

I think we detect that ourselves and go through the Hregion api. 

See Indexer#doPostWithExceptions where we collect local updates:

{code}
for (Pair<Mutation, byte[]> mutation : indexUpdates) {
        			  if (Bytes.toString(mutation.getSecond()).equals(
        					  environment.getRegion().getTableDesc().getNameAsString())) {
        				  localUpdates.add(mutation);
        			  } else {
                          remoteUpdates.add(mutation);
        			  }
        		  }
{code}

And then in ParallelWriterIndexCommitter, we apply these local updates through IndexUtil.writeLocalUpdates():

{code}

try {
                                throwFailureIfDone();
                                IndexUtil.writeLocalUpdates(env.getRegion(), mutations, true);
                                return null;
                            } catch (IOException ignord) {
                                // when it's failed we fall back to the standard & slow
way
                                if (LOG.isDebugEnabled()) {
                                    LOG.debug("indexRegion.batchMutate failed and fall back
to HTable.batch(). Got error="
                                              ignord);
                                }
                            }
{code}

IndexUtil#writeLocalUpdates looks like this:

{code}
public static void writeLocalUpdates(HRegion region, final List<Mutation> mutations,
boolean skipWAL) throws IOException {
        if(skipWAL) {
            for (Mutation m : mutations) {
                m.setDurability(Durability.SKIP_WAL);
            }
        }
        region.batchMutate(
            mutations.toArray(new Mutation[mutations.size()]),
            HConstants.NO_NONCE, HConstants.NO_NONCE);
    }
{code}


> Index RPC priority still depends on the controller factory property in hbase-site.xml
> -------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3994
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3994
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.11.0
>            Reporter: Sergey Soldatov
>            Priority: Critical
>         Attachments: PHOENIX-3994.patch
>
>
> During PHOENIX-3360 we tried to remove dependency on hbase.rpc.controllerfactory.class
property in hbase-site.xml since it cause problems on the client side (if client is using
server side configuration, all client request may go using index priority). Committed solution
is using setting the controller factory programmatically for coprocessor environment in Indexer
class, but it comes that this solution doesn't work because the environment configuration
is not used for the coprocessor connection creation. We need to provide a better solution
since this issue may cause accidental locks and failures that hard to identify and avoid.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message