atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shwetha GS <sshivalingamur...@hortonworks.com>
Subject Re: Review Request 45558: ATLAS-540 API to retrieve entity version events
Date Sat, 09 Apr 2016 18:16:03 GMT


> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java,
line 110
> > <https://reviews.apache.org/r/45558/diff/2/?file=1323995#file1323995line110>
> >
> >     Can we create an AtlasClient here with the endpoint? The advantage is that we
would not need to store the endpoint as state. Also, we would keep chaining constructors,
so that there would be only one initialization point in the end.

As part of AtlasClient creation, it writes config file and createss objects. Want to avoid
all this in hive hook as hive hook doesn't require atlas client. Have cleaned up HiveMetaStoreBridge
constructor calls though


> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java,
line 161
> > <https://reviews.apache.org/r/45558/diff/2/?file=1324007#file1324007line161>
> >
> >     If the last byte has value 0x7F, this overflows to a negative number. In that
case, the byte comparison would be different from expected, no?

The byte array here is for the entity id string which is UUID. So, don't expect the overflow


> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java,
line 164
> > <https://reviews.apache.org/r/45558/diff/2/?file=1324007#file1324007line164>
> >
> >     Start rows seem to be inclusive in scans: http://stackoverflow.com/questions/17558547/hbase-easy-how-to-perform-range-prefix-scan-in-hbase-shell.
Also, the javadoc for setStartRow says this: Note: In order to make startRow exclusive add
a trailing 0 byte.
> >     
> >     I feel it will be easier for clients to just pass the last key in the result
set and we add this trailing byte 0 to get results from the next row. Thoughts?

Start row by intuition, should be included in the results.

The thing is, the keys are ordered in the increasing order of timestamp in hbase. But while
retrieving entries, we do reverse scan and return the results. So, if the clients want pagination
with 20 results, they should retrieve 21 results and use the last one's key as startKey for
next call. Yes, this is easier than clients having to do manual key modification to figure
out the startkey. Will update the documentation


> On April 4, 2016, 1:33 p.m., Hemanth Yamijala wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java,
line 176
> > <https://reviews.apache.org/r/45558/diff/2/?file=1324007#file1324007line176>
> >
> >     Shouldn't we break here? Once an entityId is crossed, we would not get any results
that would apply for this entityId anymore, right?

With reverse scan, this should be continue


- Shwetha


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45558/#review126818
-----------------------------------------------------------


On April 4, 2016, 8:36 a.m., Shwetha GS wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45558/
> -----------------------------------------------------------
> 
> (Updated April 4, 2016, 8:36 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-540
>     https://issues.apache.org/jira/browse/ATLAS-540
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Added REST API to get audit events. 
> 
> Pending: User info is no coming correctly. Each REST API call makes two calls - OPTIONS
call and then actual call. OPTIONS call contains the actual user in httprequest, but the actual
call contains the user in the queryparam doAs. Will debug this as part of ATLAS-585
> 
> 
> Diffs
> -----
> 
>   addons/falcon-bridge/pom.xml 03cb11e 
>   addons/falcon-bridge/src/test/java/org/apache/atlas/falcon/hook/FalconHookIT.java 4249a8f

>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
50a5311 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 68e32ff 
>   addons/sqoop-bridge/src/test/java/org/apache/atlas/sqoop/hook/SqoopHookIT.java 0e4658a

>   client/src/main/java/org/apache/atlas/AtlasClient.java 0bb5264 
>   client/src/main/java/org/apache/atlas/EntityAuditEvent.java PRE-CREATION 
>   common/src/main/java/org/apache/atlas/utils/ParamChecker.java 91adfaf 
>   notification/src/main/java/org/apache/atlas/notification/NotificationModule.java e8ae177

>   pom.xml 226529d 
>   repository/pom.xml eca087a 
>   repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java ce1bdfb 
>   repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
0c5c551 
>   repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
d41c4da 
>   repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
c4329a5 
>   repository/src/main/java/org/apache/atlas/repository/audit/InMemoryEntityAuditRepository.java
df75290 
>   repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java cd1161a

>   repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
9c193f7 
>   repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java aa00417

>   server-api/pom.xml d3e84c4 
>   server-api/src/main/java/org/apache/atlas/services/MetadataService.java ab402d7 
>   webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java 6bfd780

>   webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 53c503b 
>   webapp/src/main/java/org/apache/atlas/web/resources/TypesResource.java 8b0d0e9 
>   webapp/src/main/java/org/apache/atlas/web/service/ActiveInstanceElectorModule.java
065666d 
>   webapp/src/main/java/org/apache/atlas/web/service/ServiceModule.java PRE-CREATION 
>   webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java a8cd1ef

> 
> Diff: https://reviews.apache.org/r/45558/diff/
> 
> 
> Testing
> -------
> 
> UTs, ITs and end to end test
> 
> 
> Thanks,
> 
> Shwetha GS
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message