atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "t oo (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ATLAS-2889) S3 object tag import hook
Date Tue, 25 Sep 2018 00:01:00 GMT
t oo created ATLAS-2889:
---------------------------

             Summary: S3 object tag import hook
                 Key: ATLAS-2889
                 URL: https://issues.apache.org/jira/browse/ATLAS-2889
             Project: Atlas
          Issue Type: New Feature
          Components:  atlas-core, atlas-intg
    Affects Versions: 1.1.0
            Reporter: t oo


https://issues.apache.org/jira/browse/ATLAS-2708 introduced support for s3 type.

 

>From the comments in the jira, there are some external steps needed to import object
tags. This Jira is about creating the E2E functionality within Atlas. ie functionality in
UI or an API that when supplied with argument of an s3 bucket/slice then Atlas automatically
imports all the object names/tags recursively from under that s3 bucket/slice.

 

"It doesn't do it automatically through a listener like the hive hook.  We do it via lambda
functions, triggered, say, on the creation of S3 object or pseudodirectory or bucket.  We
package up the info into AtlasEntities and then publish to the ATLAS_HOOK kafka topic.

You have to create your own Lambda code that creates AtlasEntities on the fly as trigerred
by Lambda Function(on changes made to s3 object) and then push to Kafka Queue. This particular
functionality is not part of Atlas tool as of now."

cc: [~barbara] [~ayushmnnit]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message