phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chenglei (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-3199) ServerCacheClient would send cache to all regions unnecessarily once it should send cache to the first region
Date Thu, 03 Nov 2016 12:40:58 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

chenglei updated PHOENIX-3199:
------------------------------
    Description: 
The issue is caused by the htable's coprocessorService method,the third parameter endKey is
inclusive,not exclusive.When both
startKey and endKey is HConstants.EMPTY_START_ROW,the coprocessorService method may send callable
to all regions:

{code:borderStyle=solid} 
coprocessorService(final Class<T> service,byte[] startKey, byte[] endKey, final Batch.Call<T,R>
callable)


In the addServerCache method of  org.apache.phoenix.cache.ServerCacheClient  class, once the
first region can pass the if test in line 174, because the startKey of the first region is
HConstants.EMPTY_START_ROW, so the key  is also  HConstants.EMPTY_START_ROW in line 180. When
we invoke the htable's coprocessorService method in line 189, the startKey and endKey(Inclusive)
parameters are both HConstants.EMPTY_START_ROW,and  the htable's coprocessorService method
internally  uses getKeysAndRegionsInRange method to locate 
 [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so cache would be
sent to all regions :

{code:borderStyle=solid} 
170  for (HRegionLocation entry : locations) {
171               // Keep track of servers we've sent to and only send once
172                byte[] regionStartKey = entry.getRegionInfo().getStartKey();
173                byte[] regionEndKey = entry.getRegionInfo().getEndKey();
174               if ( ! servers.contains(entry) && 
175                        keyRanges.intersectRegion(regionStartKey, regionEndKey,
176                               cacheUsingTable.getIndexType() == IndexType.LOCAL)) {  
177                   // Call RPC once per server
178                   servers.add(entry);
179                    if (LOG.isDebugEnabled()) {LOG.debug(addCustomAnnotations("Adding cache
entry to be sent for " + entry, connection));}
180                    final byte[] key = entry.getRegionInfo().getStartKey();
181                    final HTableInterface htable = services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
182                    closeables.add(htable);
183                    futures.add(executor.submit(new JobCallable<Boolean>() {
184                      
185                        @Override
186                        public Boolean call() throws Exception {
187                            final Map<byte[], AddServerCacheResponse> results;
188                            try {
189                                results = htable.coprocessorService(ServerCachingService.class,
key, key, 
190                                            new Batch.Call<ServerCachingService, AddServerCacheResponse>()
{
{code}

  was:
In the addServerCache method of  org.apache.phoenix.cache.ServerCacheClient  class, once the
first region can pass the if test in line 174, because the startKey of the first region is
HConstants.EMPTY_START_ROW, so the key  is also  HConstants.EMPTY_START_ROW in line 180. When
we invoke the htable's coprocessorService method in line 189, the startKey and endKey(Inclusive)
parameters are both HConstants.EMPTY_START_ROW,and  the htable's coprocessorService method
internally  uses getKeysAndRegionsInRange method to locate 
 [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so cache would be
sent to all regions :

{code:borderStyle=solid} 
170  for (HRegionLocation entry : locations) {
171               // Keep track of servers we've sent to and only send once
172                byte[] regionStartKey = entry.getRegionInfo().getStartKey();
173                byte[] regionEndKey = entry.getRegionInfo().getEndKey();
174               if ( ! servers.contains(entry) && 
175                        keyRanges.intersectRegion(regionStartKey, regionEndKey,
176                               cacheUsingTable.getIndexType() == IndexType.LOCAL)) {  
177                   // Call RPC once per server
178                   servers.add(entry);
179                    if (LOG.isDebugEnabled()) {LOG.debug(addCustomAnnotations("Adding cache
entry to be sent for " + entry, connection));}
180                    final byte[] key = entry.getRegionInfo().getStartKey();
181                    final HTableInterface htable = services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
182                    closeables.add(htable);
183                    futures.add(executor.submit(new JobCallable<Boolean>() {
184                      
185                        @Override
186                        public Boolean call() throws Exception {
187                            final Map<byte[], AddServerCacheResponse> results;
188                            try {
189                                results = htable.coprocessorService(ServerCachingService.class,
key, key, 
190                                            new Batch.Call<ServerCachingService, AddServerCacheResponse>()
{
{code}


> ServerCacheClient would send cache to all regions unnecessarily once it should send cache
to the first region
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3199
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3199
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.8.0
>            Reporter: chenglei
>         Attachments: PHOENIX-3199_v3.patch
>
>
> The issue is caused by the htable's coprocessorService method,the third parameter endKey
is inclusive,not exclusive.When both
> startKey and endKey is HConstants.EMPTY_START_ROW,the coprocessorService method may send
callable to all regions:
> {code:borderStyle=solid} 
> coprocessorService(final Class<T> service,byte[] startKey, byte[] endKey, final
Batch.Call<T,R> callable)
> In the addServerCache method of  org.apache.phoenix.cache.ServerCacheClient  class, once
the first region can pass the if test in line 174, because the startKey of the first region
is HConstants.EMPTY_START_ROW, so the key  is also  HConstants.EMPTY_START_ROW in line 180.
When we invoke the htable's coprocessorService method in line 189, the startKey and endKey(Inclusive)
parameters are both HConstants.EMPTY_START_ROW,and  the htable's coprocessorService method
internally  uses getKeysAndRegionsInRange method to locate 
>  [HConstants.EMPTY_START_ROW,HConstants.EMPTY_START_ROW] to all regions, so cache would
be sent to all regions :
> {code:borderStyle=solid} 
> 170  for (HRegionLocation entry : locations) {
> 171               // Keep track of servers we've sent to and only send once
> 172                byte[] regionStartKey = entry.getRegionInfo().getStartKey();
> 173                byte[] regionEndKey = entry.getRegionInfo().getEndKey();
> 174               if ( ! servers.contains(entry) && 
> 175                        keyRanges.intersectRegion(regionStartKey, regionEndKey,
> 176                               cacheUsingTable.getIndexType() == IndexType.LOCAL))
{  
> 177                   // Call RPC once per server
> 178                   servers.add(entry);
> 179                    if (LOG.isDebugEnabled()) {LOG.debug(addCustomAnnotations("Adding
cache entry to be sent for " + entry, connection));}
> 180                    final byte[] key = entry.getRegionInfo().getStartKey();
> 181                    final HTableInterface htable = services.getTable(cacheUsingTableRef.getTable().getPhysicalName().getBytes());
> 182                    closeables.add(htable);
> 183                    futures.add(executor.submit(new JobCallable<Boolean>() {
> 184                      
> 185                        @Override
> 186                        public Boolean call() throws Exception {
> 187                            final Map<byte[], AddServerCacheResponse> results;
> 188                            try {
> 189                                results = htable.coprocessorService(ServerCachingService.class,
key, key, 
> 190                                            new Batch.Call<ServerCachingService,
AddServerCacheResponse>() {
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message