Mailing-List: contact dev-help@atlas.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@atlas.incubator.apache.org
Date: Tue, 10 Jan 2017 23:53:58 +0000 (UTC)
From: "Suma Shivaprasad (JIRA)" <jira@apache.org>
To: dev@atlas.incubator.apache.org
Message-ID: <JIRA.13027563.1481572123000.702843.1484092438431@Atlassian.JIRA>
In-Reply-To: <JIRA.13027563.1481572123000@Atlassian.JIRA>
References: <JIRA.13027563.1481572123000@Atlassian.JIRA> <JIRA.13027563.1481572123136@arcas>
Subject: [jira] [Commented] (ATLAS-1369) Optimize Gremlin created by DSL
 Translator
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 10 Jan 2017 23:54:50 -0000


    [ https://issues.apache.org/jira/browse/ATLAS-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816571#comment-15816571 ] 

Suma Shivaprasad commented on ATLAS-1369:
-----------------------------------------

Hi [~jnhagelb]

Since getting this to work with TP2 was taking more time, I fixed it in an easier way with the code on top of 0.7-incubating release - https://issues.apache.org/jira/browse/ATLAS-1403 which we are planning to release in 0.7.1-incubating. Did a couple of minor issues with this patch while testing

1. changed java.util.Function to guava Function
2. Enabled GremlinQueryOptimizer for TP2
3. Changed GraphTraversal to GremlinPipeline 

After these changes there were a bunch of failures as I remember correctly with the groupBy, order by expressions related queries which I couldnt get time to debug further. I guess there needs to be some changes done on top of this patch to make it work for TP2 as you mentioned which needs to be addressed before committing. 

Please take a look at https://issues.apache.org/jira/browse/ATLAS-1403 and see if it would be helpful with this optimization effort as well. Let me know if you have any questions.

> Optimize Gremlin created by DSL Translator
> ------------------------------------------
>
>                 Key: ATLAS-1369
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1369
>             Project: Atlas
>          Issue Type: Improvement
>            Reporter: Jeffrey Hagelberg
>            Assignee: Jeffrey Hagelberg
>         Attachments: rb54680.patch
>
>
> The way the queries are generated now, in many cases the indices are not used when executing queries.  The main problem is that gremlin cannot currently use the graph indices to optimize the lookup for "has" expressions within an "and" or an "or".  This JIRA introduces a GremlinQueryOptimizer that attempts to extract "has" expressions out of "and" expressions and turn "or" expressions into unions.
> The code being submitted has only been tested with TP 3 at this point.  There are some issues that need to be fixed before this can be used with TP 2:
>   - GremlinQueryOptimizer is currently disabled when using TP 2.  The condition in AtlasRepositoryConfiguration.isGremlinOptimizerEnabled needs to be updated.
>   - FunctionGenerator - generates functions that take GraphTraversal as an argument.  That should be changed to GreminPipeline when using TP 2.  The return type should probably come from a new method in GremlinExpressionFactory
>    - loop support - for TP2, we will need to add logic to make sure that the loop call is not separated from the alias it references
>   - alias handling - the alias save/restore logic in ExpandOrsOptimization.getBaseResultExpression() may be need to adjusted for TP2
>   - possibly other issues, TBD


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)