Return-Path: X-Original-To: apmail-phoenix-dev-archive@minotaur.apache.org Delivered-To: apmail-phoenix-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E976817493 for ; Fri, 17 Apr 2015 17:22:31 +0000 (UTC) Received: (qmail 30563 invoked by uid 500); 17 Apr 2015 17:22:31 -0000 Delivered-To: apmail-phoenix-dev-archive@phoenix.apache.org Received: (qmail 30494 invoked by uid 500); 17 Apr 2015 17:22:31 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 30443 invoked by uid 99); 17 Apr 2015 17:22:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Apr 2015 17:22:31 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: message received from 54.164.171.186 which is an MX secondary for dev@phoenix.apache.org) Received: from [54.164.171.186] (HELO mx1-us-east.apache.org) (54.164.171.186) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Apr 2015 17:22:26 +0000 Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with SMTP id 697BC453F3 for ; Fri, 17 Apr 2015 17:22:05 +0000 (UTC) Received: (qmail 19830 invoked by uid 99); 17 Apr 2015 17:21:59 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Apr 2015 17:21:59 +0000 Date: Fri, 17 Apr 2015 17:21:59 +0000 (UTC) From: "Shuxiong Ye (JIRA)" To: dev@phoenix.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (PHOENIX-1870) Fix NPE occurring during regex processing when joni library not used MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/PHOENIX-1870?page=3Dcom.atlassi= an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D14= 500220#comment-14500220 ]=20 Shuxiong Ye edited comment on PHOENIX-1870 at 4/17/15 5:21 PM: --------------------------------------------------------------- [~jamestaylor] Several questions: 1. What tuple is used for? 2. In some functions, for example RegexpReplaceFunction, there are two argu= ments, source strings and replace strings. How should the replace interface= be designed? 2.1 replace(srcPtr, replacePtr): RegexpReplaceFunction has a class member r= eplacePtr, and if is used in evaluation, and will be set to empty bytes after evaluation. 2.2 replace(ptr, srcExpression, replaceExpression): Pass the expression to = Regex Engine, and srcStr and replaceStr use the same ptr. Which one is better? 3. for a function, REGEXP_REPLACE(PatternStr, SrcStr, ReplaceStr), if some = of the three arguments is Determinism.ALWAYS, so they can be pre-compute in init(). Why not try to cache the value? Fox example, fun1( arg1, fun2(arg2)), if fun2(arg2) is Determinism.ALWAYS, = and in every evaluation of fun1, we don't have to compute fun2 again, because we can pre= -compute in fun1.init(). was (Author: shuxi0ng): [~jamestaylor] Several questions: 1. What tuple is used for? 2. In some functions, for example RegexpReplaceFunction, there are two argu= ments, source strings and replace strings. 2.1 replace(srcPtr, replacePtr): RegexpReplaceFunction has a class member r= eplacePtr, and if is used in evaluation, and will be set to empty bytes after evaluation. 2.2 replace(ptr, srcExpression, replaceExpression): Pass the expression to = Regex Engine, and srcStr and replaceStr use the same ptr. Which one is better? 3. for a function, REGEXP_REPLACE(PatternStr, SrcStr, ReplaceStr), if some = of the three arguments is Determinism.ALWAYS, so they can be pre-compute in init(). Why not try to cache the value? Fox example, fun1( arg1, fun2(arg2)), if fun2(arg2) is Determinism.ALWAYS, = and in every evaluation of fun1, we don't have to compute fun2 again, because we can pre= -compute in fun1.init(). > Fix NPE occurring during regex processing when joni library not used > -------------------------------------------------------------------- > > Key: PHOENIX-1870 > URL: https://issues.apache.org/jira/browse/PHOENIX-1870 > Project: Phoenix > Issue Type: Sub-task > Reporter: James Taylor > Assignee: Shuxiong Ye > Fix For: 5.0.0, 4.4.0 > > Attachments: 0001-PHOENIX-1870-Fix-NPE-occurring-during-regex-pro= cessi.patch, PHOENIX-1870_v2.patch > > > Tests in error: > {code} > IndexExpressionIT.testImmutableCaseSensitiveFunctionIndex:1277->helpTes= tCaseSensitiveFunctionIndex:1321 =C2=BB Commit > IndexExpressionIT.testImmutableLocalCaseSensitiveFunctionIndex:1282->he= lpTestCaseSensitiveFunctionIndex:1321 =C2=BB Commit > {code} > Stack trace: > {code} > testImmutableCaseSensitiveFunctionIndex(org.apache.phoenix.end2end.index.= IndexExpressionIT) Time elapsed: 0.723 sec <<< ERROR! > org.apache.phoenix.execute.CommitException: org.apache.hadoop.hbase.clien= t.RetriesExhaustedWithDetailsException: Failed 2 actions: org.apache.phoeni= x.hbase.index.builder.IndexBuildingFailureException: Failed to build index = for unexpected reason! > at org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethro= wIndexingException(IndexManagementUtil.java:180) > at org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.= java:206) > at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$35.= call(RegionCoprocessorHost.java:989) > at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$Reg= ionOperation.call(RegionCoprocessorHost.java:1671) > at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.exe= cOperation(RegionCoprocessorHost.java:1746) > at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.exe= cOperation(RegionCoprocessorHost.java:1703) > at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.pre= BatchMutate(RegionCoprocessorHost.java:985) > at org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutati= on(HRegion.java:2697) > at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegi= on.java:2478) > at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegi= on.java:2432) > at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegi= on.java:2436) > at org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(R= SRpcServices.java:641) > at org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomic= RegionMutation(RSRpcServices.java:605) > at org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpc= Services.java:1822) > at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Client= Service$2.callBlockingMethod(ClientProtos.java:31451) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031= ) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107= ) > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecut= or.java:130) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java= :107) > at java.lang.Thread.run(Thread.java:724) > Caused by: java.lang.NullPointerException > at org.apache.phoenix.expression.util.regex.JavaPattern.substr(Ja= vaPattern.java:83) > at org.apache.phoenix.expression.function.RegexpSubstrFunction.ev= aluate(RegexpSubstrFunction.java:118) > at org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMain= tainer.java:453) > at org.apache.phoenix.index.IndexMaintainer.buildDeleteMutation(I= ndexMaintainer.java:842) > at org.apache.phoenix.index.PhoenixIndexCodec.getIndexUpdates(Pho= enixIndexCodec.java:173) > at org.apache.phoenix.index.PhoenixIndexCodec.getIndexDeletes(Pho= enixIndexCodec.java:119) > at org.apache.phoenix.hbase.index.covered.CoveredColumnsIndexBuil= der.addDeleteUpdatesToMap(CoveredColumnsIndexBuilder.java:403) > at org.apache.phoenix.hbase.index.covered.CoveredColumnsIndexBuil= der.addCleanupForCurrentBatch(CoveredColumnsIndexBuilder.java:287) > at org.apache.phoenix.hbase.index.covered.CoveredColumnsIndexBuil= der.addMutationsForBatch(CoveredColumnsIndexBuilder.java:239) > at org.apache.phoenix.hbase.index.covered.CoveredColumnsIndexBuil= der.batchMutationAndAddUpdates(CoveredColumnsIndexBuilder.java:136) > at org.apache.phoenix.hbase.index.covered.CoveredColumnsIndexBuil= der.getIndexUpdate(CoveredColumnsIndexBuilder.java:99) > at org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.cal= l(IndexBuildManager.java:133) > at org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.cal= l(IndexBuildManager.java:129) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:= 334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolEx= ecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolE= xecutor.java:615) > ... 1 more > : 2 times, > at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeEx= ception(AsyncProcess.java:227) > at org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access= $1700(AsyncProcess.java:207) > at org.apache.hadoop.hbase.client.AsyncProcess$AsyncRequestFuture= Impl.getErrors(AsyncProcess.java:1563) > at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:928) > at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:942) > at org.apache.phoenix.execute.MutationState.commit(MutationState.= java:422) > at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnec= tion.java:439) > at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnec= tion.java:436) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnec= tion.java:436) > at org.apache.phoenix.end2end.index.IndexExpressionIT.helpTestCas= eSensitiveFunctionIndex(IndexExpressionIT.java:1321) > at org.apache.phoenix.end2end.index.IndexExpressionIT.testImmutab= leCaseSensitiveFunctionIndex(IndexExpressionIT.java:1277) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)