Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@phoenix.apache.org
Date: Thu, 13 Jul 2017 12:16:02 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: dev@phoenix.apache.org
Message-ID: <JIRA.13086276.1499783222000.238842.1499948162497@Atlassian.JIRA>
In-Reply-To: <JIRA.13086276.1499783222000@Atlassian.JIRA>
References: <JIRA.13086276.1499783222000@Atlassian.JIRA> <JIRA.13086276.1499783222000@jira-lw-us.apache.org>
Subject: [jira] [Commented] (PHOENIX-4010) Hash Join cache may not be send
 to all regionservers when we have stale HBase meta cache
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 13 Jul 2017 12:16:11 -0000


    [ https://issues.apache.org/jira/browse/PHOENIX-4010?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D16=
085620#comment-16085620 ]=20

Hadoop QA commented on PHOENIX-4010:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest a=
ttachment=20
  http://issues.apache.org/jira/secure/attachment/12876644/PHOENIX-4010.pat=
ch
  against master branch at commit f2eac858eab64fda3eacf7f6e1b2ab9656bf4cfa.
  ATTACHMENT ID: 12876644

    {color:green}+1 @author{color}.  The patch does not contain any @author=
 tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to inclu=
de any new or modified tests.
                        Please justify why no new tests are needed for this=
 patch.
                        Also please list what manual steps were performed t=
o verify this patch.

    {color:green}+1 javac{color}.  The applied patch does not increase the =
total number of javac compiler warnings.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generat=
ed 52 warning messages.

    {color:green}+1 release audit{color}.  The applied patch does not incre=
ase the total number of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces the following l=
ines longer than 100:
    +    private static final Map<Integer, TableRef> cacheUsingTableRefMap =
=3D new ConcurrentHashMap<Integer, TableRef>();
+                            return addServerCache(htable, key, cacheUsingT=
able, cacheId, cachePtr, cacheFactory, txState);
+             * Allow for the possibility that the region we based where to=
 send our cache has split and been relocated
+             * to another region server *after* we sent it, but before we =
removed it. To accommodate this, we iterate
+             * through the current metadata boundaries and remove the cach=
e once for each server that we originally sent
+                LOG.debug(addCustomAnnotations("Removing Cache " + cacheId=
 + " from servers.", connection));
+                        || (keys !=3D null && ByteUtil.contains(keys,entry=
.getRegionInfo().getStartKey()))) {=20
+                                    public RemoveServerCacheResponse call(=
ServerCachingService instance)
+                                        BlockingRpcCallback<RemoveServerCa=
cheResponse> rpcCallback =3D new BlockingRpcCallback<RemoveServerCacheRespo=
nse>();
+                                        RemoveServerCacheRequest.Builder b=
uilder =3D RemoveServerCacheRequest

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end=
.GroupByIT

     {color:red}-1 core zombie tests{color}.  There are 1 zombie test(s): =
=09at org.apache.cxf.transport.jms.RequestResponseTest.testRequestTopicResp=
onseStaticQueue(RequestResponseTest.java:90)

Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1207//t=
estReport/
Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/120=
7//artifact/patchprocess/patchJavadocWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1207/=
/console

This message is automatically generated.

> Hash Join cache may not be send to all regionservers when we have stale H=
Base meta cache
> -------------------------------------------------------------------------=
---------------
>
>                 Key: PHOENIX-4010
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4010
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Ankit Singhal
>            Assignee: Ankit Singhal
>             Fix For: 4.12.0
>
>         Attachments: PHOENIX-4010.patch
>
>
>  If the region locations changed and our HBase meta cache is not updated =
then we might not be sending hash join cache to all region servers hosting =
the regions.
> ConnectionQueryServicesImpl#getAllTableRegions
> {code}
> boolean reload =3Dfalse;
>         while (true) {
>             try {
>                 // We could surface the package projected HConnectionImpl=
ementation.getNumberOfCachedRegionLocations
>                 // to get the sizing info we need, but this would require=
 a new class in the same package and a cast
>                 // to this implementation class, so it's probably not wor=
th it.
>                 List<HRegionLocation> locations =3D Lists.newArrayList();
>                 byte[] currentKey =3D HConstants.EMPTY_START_ROW;
>                 do {
>                     HRegionLocation regionLocation =3D connection.getRegi=
onLocation(
>                             TableName.valueOf(tableName), currentKey, rel=
oad);
>                     locations.add(regionLocation);
>                     currentKey =3D regionLocation.getRegionInfo().getEndK=
ey();
>                 } while (!Bytes.equals(currentKey, HConstants.EMPTY_END_R=
OW));
>                 return locations;
> {code}
> Skipping duplicate servers in ServerCacheClient#addServerCache
> {code}
> List<HRegionLocation> locations =3D services.getAllTableRegions(cacheUsin=
gTable.getPhysicalName().getBytes());
>             int nRegions =3D locations.size();
>            =20
> .....
>  if ( ! servers.contains(entry) &&=20
>                         keyRanges.intersectRegion(regionStartKey, regionE=
ndKey,
>                                 cacheUsingTable.getIndexType() =3D=3D Ind=
exType.LOCAL)) { =20
>                     // Call RPC once per server
>                     servers.add(entry);
> {code}
> For eg:- Table =E2=80=99T=E2=80=99 has two regions R1 and R2 originally h=
osted on regionserver RS1.=20
> while Phoenix/Hbase connection is still active, R2 is transitioned to RS2=
 ,  but stale meta cache will still give old region locations i.e R1 and R2=
 on RS1 and when we start copying hash table, we copy for R1 and skip R2 as=
 they are hosted on same regionserver. so, the query on a table will fail a=
s it will unable to find hash table cache on RS2 for processing regions R2.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)