pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-965) PERFORMANCE: optimize common case in matches (PORegex)
Date Tue, 15 Dec 2009 00:44:18 GMT

    [ https://issues.apache.org/jira/browse/PIG-965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790492#action_12790492

Thejas M Nair commented on PIG-965:

Reviewed the latest patch.
Comments :
* RegexInit.java, in determineBestRegexMethod Line 85 - 120
There are while loops where we are testing only for preceding '\'
The handling of preceding escapes could be done in a separate function, since the logic is
used at multiple places

* RegexInit.java lines 61,147
// This is the case when an old number of escapes         
I believe you meant "odd"

* RegexImpl.java - following comments are not relevant anymore
+// LHSXXXX means LHS is constantExpression and RHS varies with each Tuple
+// RHSXXXX means RHS is constantExpression and LHS varies with each Tuple

* NonConstantRegex , line 34-35
|| rhs.length() != oldString.length()
|| rhs.compareTo(oldString) != 0
could be simplified as -
|| !rhs.equals(oldString)
Did you chose the former because it might be faster ? That can be the case in this situation,
because equals has a additional check of - "instanceOf String" .  So I think the existing
code is fine. A comment there might be useful.

Can you also publish your numbers for the comparison of dk.brics.automaton.RunAutomaton and
optimization 2 (Use string comparisons for simple common regexes ) in the jira ?

> PERFORMANCE: optimize common case in matches (PORegex)
> ------------------------------------------------------
>                 Key: PIG-965
>                 URL: https://issues.apache.org/jira/browse/PIG-965
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>            Reporter: Thejas M Nair
>            Assignee: Ankit Modi
>         Attachments: automaton.jar, poregex2.patch
> Some frequently seen use cases of 'matches' comparison operator have follow properties
> 1. The rhs is a constant string . eg "c1 matches 'abc%' "
> 2. Regexes such that look for matching prefix , suffix etc are very common. eg - "abc%',
"%abc", '%abc%' 
> To optimize for these common cases , PORegex.java can be changed to -
> 1. Compile the pattern (rhs of matches) re-use it if the pattern string has not changed.

> 2. Use string comparisons for simple common regexes (in 2 above).
> The implementation of Hive like clause uses similar optimizations.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message