atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shwetha GS <sshivalingamur...@hortonworks.com>
Subject Re: Review Request 49625: ATLAS-347 Atlas search APIs should allow pagination of results
Date Tue, 19 Jul 2016 16:24:32 GMT


> On July 12, 2016, 6:07 p.m., Neeru Gupta wrote:
> > common/src/main/java/org/apache/atlas/AtlasProperties.java, line 43
> > <https://reviews.apache.org/r/49625/diff/2/?file=1442108#file1442108line43>
> >
> >     You might consider bumping up these default values. Otherwise it will break
existing clients who rely on getting all the data from db. Now by default they will only get
100 records unless they configure these properties themselves.

Getting all the data is not possible as it can lead to out of memory and hence bring down
the server. I think the default heap size is 1GB, will check whats the data size it can accommodate
and set the values accordingly.


- Shwetha


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49625/#review141933
-----------------------------------------------------------


On July 12, 2016, 6:06 a.m., Shwetha GS wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49625/
> -----------------------------------------------------------
> 
> (Updated July 12, 2016, 6:06 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-347
>     https://issues.apache.org/jira/browse/ATLAS-347
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> 1. orderby doesn't require fully qualified field name anymore. For example, 'hive_table
orderby name' resolves name to Asset.name
> 2. Added limit and offset to search by DSL API.
> 3. DSL query adds API's limit and offset to the specified DSL query. For eaxmple, 'hive_table
orderby name' maps to 'hive_table orderby name limit 100 offset 0'. 'hive_table orderby name
limit 200 offset 10' maps to 'hive_table orderby name limit 100 offset 10'. 100 and 0 are
API's limit and offset respectively here
> 4. All existing tests work
> 
> Pending:
> 1. Add more tests 
> 2. Add limit and offset for full text search. I don't think the index search that we
do supports pagination. May have to filter results on atlas server
> 
> 
> Diffs
> -----
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
cd0e964 
>   addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java
9f7f6b0 
>   addons/sqoop-bridge/src/test/java/org/apache/atlas/sqoop/hook/SqoopHookIT.java 72d9201

>   addons/storm-bridge/src/test/java/org/apache/atlas/storm/hook/StormAtlasHookIT.java
b33bb5f 
>   client/src/main/java/org/apache/atlas/AtlasClient.java d3af6ad 
>   common/src/main/java/org/apache/atlas/AtlasProperties.java PRE-CREATION 
>   common/src/main/java/org/apache/atlas/utils/ParamChecker.java edfe355 
>   docs/src/site/twiki/Search.twiki 58c9238 
>   repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java c6790de

>   repository/src/main/java/org/apache/atlas/discovery/DiscoveryService.java PRE-CREATION

>   repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
8ddbe1b 
>   repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java
f97b83d 
>   repository/src/main/java/org/apache/atlas/repository/Constants.java 893f1b6 
>   repository/src/main/scala/org/apache/atlas/query/Expressions.scala 2405750 
>   repository/src/main/scala/org/apache/atlas/query/GremlinQuery.scala 14c42b0 
>   repository/src/main/scala/org/apache/atlas/query/QueryParser.scala 60b57d9 
>   repository/src/main/scala/org/apache/atlas/query/Resolver.scala 5fc9400 
>   repository/src/main/scala/org/apache/atlas/query/TypeUtils.scala 5a64c53 
>   repository/src/test/java/org/apache/atlas/discovery/DataSetLineageServiceTest.java
89e9b41 
>   repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
a911c49 
>   repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
97dfbcd 
>   server-api/src/main/java/org/apache/atlas/discovery/DiscoveryService.java e347c2c 
>   webapp/src/main/java/org/apache/atlas/LocalAtlasClient.java 7173d4d 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStart.java 42bccec 
>   webapp/src/main/java/org/apache/atlas/web/resources/MetadataDiscoveryResource.java
3f22217 
>   webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerIT.java
72f403e 
>   webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 498db92 
>   webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java d41041c

>   webapp/src/test/java/org/apache/atlas/web/resources/MetadataDiscoveryJerseyResourceIT.java
a8c1213 
> 
> Diff: https://reviews.apache.org/r/49625/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message