pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-5316) Initialize mapred.task.id property for PoS jobs
Date Tue, 28 Nov 2017 18:42:00 GMT

    [ https://issues.apache.org/jira/browse/PIG-5316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16269241#comment-16269241
] 

Xuefu Zhang commented on PIG-5316:
----------------------------------

[~nkollar], Sorry for the late reply.
{quote}
Though MRConfiguration is not intended for public use in Pig, should Hive use MRConfiguration#TASK_ID
instead of referring to the taskId as a string?
{quote}
Your concern is valid. However, Hive has a lot of legacy MR1 code (more than just mapred.task.id).
It probably takes a lot of effort to clean up this. Before that happens, Yes, the risk is
going to be there.

> Initialize mapred.task.id property for PoS jobs
> -----------------------------------------------
>
>                 Key: PIG-5316
>                 URL: https://issues.apache.org/jira/browse/PIG-5316
>             Project: Pig
>          Issue Type: Improvement
>          Components: spark
>            Reporter: Adam Szita
>            Assignee: Nandor Kollar
>             Fix For: 0.18.0
>
>         Attachments: PIG-5316_1.patch, PIG-5316_2.patch
>
>
> Some downstream systems may require the presence of {{mapred.task.id}} property (e.g.
HCatalog). This is currently not set when Pig On Spark jobs are started. Let's initialise
it.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message