pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandeep Samdaria (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4554) Compress pig.script before encoding
Date Fri, 18 Sep 2015 20:21:04 GMT

    [ https://issues.apache.org/jira/browse/PIG-4554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876305#comment-14876305
] 

Sandeep Samdaria commented on PIG-4554:
---------------------------------------

I have added a new variable called {{truncatedScript}} and storing the truncated script in
that variable. 
I have one question. How can I test and verify that the Tez DAG UI is showing the truncated
script?
I will upload the patch, once I am able to verify the above.

> Compress pig.script before encoding
> -----------------------------------
>
>                 Key: PIG-4554
>                 URL: https://issues.apache.org/jira/browse/PIG-4554
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: 0.14.0
>            Reporter: Rohini Palaniswamy
>              Labels: newbie
>             Fix For: 0.16.0
>
>         Attachments: PIG-4554.patch
>
>
>   Currently we truncate the pig script (maxScriptSize = 10240) and base64 encode it and
store in config. We should remove the truncation and store the full script by compressing
and then doing base64 encoding. We already do that for udfcontext, etc. It will save space
as it will compress really well and will also give the full pig script while debugging.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message