atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Mestry (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ATLAS-2064) Compressed HiveHook Messages
Date Fri, 18 Aug 2017 22:53:01 GMT
Ashutosh Mestry created ATLAS-2064:
--------------------------------------

             Summary: Compressed HiveHook Messages
                 Key: ATLAS-2064
                 URL: https://issues.apache.org/jira/browse/ATLAS-2064
             Project: Atlas
          Issue Type: Improvement
          Components: atlas-intg
    Affects Versions: 0.8-incubating
            Reporter: Ashutosh Mestry
            Assignee: Ashutosh Mestry
             Fix For: trunk


*Background*
Messages posted by hooks to Atlas Kafka topics are JSON format. 

Kafka imposes a 1MB limit on the message size.

Occasionally, depending on operations performed, this threshold is reached. This results in
messages being dropped.

The entities are thus not reflected in Atlas.

*Solution*
Applying compression to these message will alleviate the problem.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message