atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 59378: Basic Search Performance Improvement
Date Fri, 19 May 2017 04:17:43 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59378/#review175467
-----------------------------------------------------------




repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 392 (patched)
<https://reviews.apache.org/r/59378/#comment248910>

    Shouldn't this list all subTypes of the given classification, similar to line #402 below
for entities?
    
    Consider replacing this method with:
    
    private String getClassificationFilter(String classificationName) {
    
      AtlasClassificationType classification = typeRegistry.getClassificationTypeByName(classificationName);
    
      if (classification != null) {
        return String.format("AND (%s)", StringUtils.join(classification.getTypeAndAllSubTypes(),
" ")); // Solr doesn't require OR seperator between type-names
      }
      
      return "";
    }
    
    Similarly consider replacing method getSubTypesForType() as well.


- Madhan Neethiraj


On May 19, 2017, 2:52 a.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59378/
> -----------------------------------------------------------
> 
> (Updated May 19, 2017, 2:52 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1818
>     https://issues.apache.org/jira/browse/ATLAS-1818
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Background
> ==========
> Please refer to [ATLAS-1818](https://issues.apache.org/jira/browse/ATLAS-1818) for background,
analysis and solution.
> 
> Implementation
> ==============
> 
> * _Solrconfig.xml_ updated.
> * _atlas-application.properties_: New property _atlas.graph.index.search.max-result-set-size_=150.
> * _EntityDiscoveryService.searchUsingBasicQuery_ updated to pass type and classification
as part of search query passed to Solr.
> * Added overload to _AtlasGraph.indexQuery_ to include offset.
> 
> **CURL**
> ```
> curl -s -u 'admin:admin' -H 'Content-type: application/json' "http://localhost:21000/api/atlas/v2/search/basic?typeName=hive_table&query=testtable_*&classification=tag0"

> ```
> 
> References
> ==========
> * _Solr Cookbook 3rd Ed by Rafal Kuc_
> 
> 
> Diffs
> -----
> 
>   distro/src/conf/atlas-application.properties b2b8e745 
>   distro/src/conf/solr/solrconfig.xml ce2e20bb 
>   graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java a3a27bfd

>   graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java
9624c99f 
>   graphdb/titan0/src/test/resources/atlas-application.properties 636a9ff3 
>   graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Graph.java
6a610755 
>   repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java e6a06c37

>   repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
PRE-CREATION 
>   typesystem/src/test/resources/atlas-application.properties 5ffde5e5 
> 
> 
> Diff: https://reviews.apache.org/r/59378/diff/5/
> 
> 
> Testing
> -------
> 
> **Unit tests**
> *EntityDiscoveryServiceTest* covers new code added during this exercise.
> 
> **Performance tests**
> Performance testing suite in performance benchmarking environment.
> 
> |-----------------------------|-----------|----------|
> | Test                        | Existing  |  New     |
> |-----------------------------|-----------|----------|
> |basic search:                |           |          |
> |type specified               |           |          |
> |classification not specified |   ~75 secs|    ~1 sec|
> |search string: testtable     |           |          | 
> |-----------------------------|-----------|----------|
> |basic search:                |           |          |
> |type specified               |           |          |
> |classification specified     |   ~75 secs|    ~1 sec|
> |search string: testtable     |           |          | 
> |-----------------------------|-----------|----------|
> |basic search:                |           |          |
> |type specified               |           |          |
> |classification specified with|           |          |
> | sub-types                   |   ~75 secs|    ~1 sec|
> |search string: testtable     |           |          | 
> |-----------------------------|-----------|----------|
> 
> **Functional tests**
> Execute the following API:
> ```
> /v2/search/basic?typeName=hive_table&query=testtable_*&classification=tag0" 
> ```
> 
> Page navigation.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message