atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-2729) Audit logging fails for large entities
Date Mon, 04 Jun 2018 06:31:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-2729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499798#comment-16499798
] 

ASF subversion and git services commented on ATLAS-2729:
--------------------------------------------------------

Commit c542deb6d6bbc47a806525aa456b8ea522bd97f4 in atlas's branch refs/heads/branch-1.0 from
[~madhan.neethiraj]
[ https://git-wip-us.apache.org/repos/asf?p=atlas.git;h=c542deb ]

ATLAS-2729: fix - audit logging fails for large entities

(cherry picked from commit 92f3a91ec56538305043ba0980d3f7a8b477ec96)


> Audit logging fails for large entities
> --------------------------------------
>
>                 Key: ATLAS-2729
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2729
>             Project: Atlas
>          Issue Type: Bug
>          Components:  atlas-core
>    Affects Versions: 1.0.0
>            Reporter: Ayub Pathan
>            Assignee: Madhan Neethiraj
>            Priority: Critical
>         Attachments: ATLAS-2729.patch
>
>
> Created a table with 4000 columns. Notification message from Hive hooks is about 5mb;
and  the hook sends the message in compressed format. However, processing of this notification
fails in Atlas server, due to the following error:
> {noformat}
> 2018-05-28 06:07:37,817 WARN  - [NotificationHookConsumer thread-0:] ~ audit record too
long: entityType=hive_table, guid=4124f580-b98d-473b-ba51-12e2ca2c6cec, size=2891084; maxSize=1048576.
entity attribute values not stored in audit (EntityAuditListenerV2:189)
> 2018-05-28 06:07:37,899 ERROR - [NotificationHookConsumer thread-0:] ~ graph rollback
due to exception  (GraphTransactionInterceptor:154)
> java.lang.IllegalArgumentException: KeyValue size too large
>         at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:952)
>         at org.apache.hadoop.hbase.client.HTable.validatePut(HTable.java:940)
>         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:564)
>         at org.apache.atlas.repository.audit.HBaseBasedAuditRepository.putEve
> {noformat}
> This issue was addressed in earlier releases via ATLAS-1463. The fix needs to be reviewed/updated
for recent additions in master (i.e. addition of relationshipAttributes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message