atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suma Shivaprasad (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-1369) Optimize Gremlin created by DSL Translator
Date Tue, 10 Jan 2017 23:53:58 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816571#comment-15816571
] 

Suma Shivaprasad commented on ATLAS-1369:
-----------------------------------------

Hi [~jnhagelb]

Since getting this to work with TP2 was taking more time, I fixed it in an easier way with
the code on top of 0.7-incubating release - https://issues.apache.org/jira/browse/ATLAS-1403
which we are planning to release in 0.7.1-incubating. Did a couple of minor issues with this
patch while testing

1. changed java.util.Function to guava Function
2. Enabled GremlinQueryOptimizer for TP2
3. Changed GraphTraversal to GremlinPipeline 

After these changes there were a bunch of failures as I remember correctly with the groupBy,
order by expressions related queries which I couldnt get time to debug further. I guess there
needs to be some changes done on top of this patch to make it work for TP2 as you mentioned
which needs to be addressed before committing. 

Please take a look at https://issues.apache.org/jira/browse/ATLAS-1403 and see if it would
be helpful with this optimization effort as well. Let me know if you have any questions.

> Optimize Gremlin created by DSL Translator
> ------------------------------------------
>
>                 Key: ATLAS-1369
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1369
>             Project: Atlas
>          Issue Type: Improvement
>            Reporter: Jeffrey Hagelberg
>            Assignee: Jeffrey Hagelberg
>         Attachments: rb54680.patch
>
>
> The way the queries are generated now, in many cases the indices are not used when executing
queries.  The main problem is that gremlin cannot currently use the graph indices to optimize
the lookup for "has" expressions within an "and" or an "or".  This JIRA introduces a GremlinQueryOptimizer
that attempts to extract "has" expressions out of "and" expressions and turn "or" expressions
into unions.
> The code being submitted has only been tested with TP 3 at this point.  There are some
issues that need to be fixed before this can be used with TP 2:
>   - GremlinQueryOptimizer is currently disabled when using TP 2.  The condition in AtlasRepositoryConfiguration.isGremlinOptimizerEnabled
needs to be updated.
>   - FunctionGenerator - generates functions that take GraphTraversal as an argument.
 That should be changed to GreminPipeline when using TP 2.  The return type should probably
come from a new method in GremlinExpressionFactory
>    - loop support - for TP2, we will need to add logic to make sure that the loop call
is not separated from the alias it references
>   - alias handling - the alias save/restore logic in ExpandOrsOptimization.getBaseResultExpression()
may be need to adjusted for TP2
>   - possibly other issues, TBD



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message