hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11139) Emit more lineage information
Date Sun, 19 Jul 2015 20:53:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632922#comment-14632922
] 

Prasanth Jayachandran commented on HIVE-11139:
----------------------------------------------

[~jxiang] Yes you are right. Setting maxBackupIndex to 0 will truncate the file. If the aim
of the appender is to not delete files, then how about using the normal FileAppender which
should never delete/roll the files? If rolling up is desired, then recommendation from log4j
is to use maxBackupIndex to <10 (for performance reasons) and use a high value for maxFileSize
(in the order of GBs). Similar discussion is here https://jazz.net/forum/questions/150960/orgapachelog4jrollingfileappender-maxbackupindex-limit

> Emit more lineage information
> -----------------------------
>
>                 Key: HIVE-11139
>                 URL: https://issues.apache.org/jira/browse/HIVE-11139
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Jimmy Xiang
>            Assignee: Jimmy Xiang
>             Fix For: 1.3.0, 2.0.0
>
>         Attachments: HIVE-11139.1.patch, HIVE-11139.2.patch, HIVE-11139.3.patch
>
>
> HIVE-1131 emits some column lineage info. But it doesn't support INSERT statements, or
CTAS statements. It doesn't emit the predicate information either.
> We can enhance and use the dependency information created in HIVE-1131, generate more
complete lineage info.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message