pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "shenxianqiang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-4849) pig on tez will cause tez-ui to crash,because the content from timeline server is too long.
Date Thu, 24 Mar 2016 10:10:25 GMT

     [ https://issues.apache.org/jira/browse/PIG-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

shenxianqiang updated PIG-4849:
-------------------------------
    Attachment: PIG-4849.patch

> pig on tez will cause tez-ui to crash,because the content from timeline server is too
long. 
> --------------------------------------------------------------------------------------------
>
>                 Key: PIG-4849
>                 URL: https://issues.apache.org/jira/browse/PIG-4849
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.15.0
>         Environment: pig:0.15,tez:0.6.2,hadoop:2.5.0
>            Reporter: shenxianqiang
>            Assignee: shenxianqiang
>         Attachments: PIG-4849.patch
>
>
> After running several Pig Job,the tez-ui server crashed,because the content from timeline
server exceed 80MB.
> When I input :
> http://timeline:48188/ws/v1/timeline/TEZ_DAG_ID?limit=51
> The content is too long...
> Like this :
> {quote}
> {"vertexName":"scope-18","processorClass":"org.apache.pig.backend.hadoop.executionengine.tez.runtime.PigProcessor","userPayloadAsText":"{\"desc\":\"wordcount[4,12]
(GROUP_BY)\",\"config\":{\"dfs.datanode.data.dir\":\"file:\\/\\/\\/search\\/hadoop\\/dfs_data,\",\"dfs.namenode.checkpoint.txns\":\"1000000\",\"s3.replication\":\"3\",\"mapreduce.output.fileoutputformat.compress.type\":\"RECORD\",\"mapreduce.jobtracker.jobhistory.lru.cache.size\":\"5\",\"hadoop.http.filter.initializers\":\"org.apache.hadoop.http.lib.StaticUserWebFilter\",\"yarn.nodemanager.keytab\":\"\\/etc\\/krb5.keytab\",\"nfs.mountd.port\":\"4242\",\"yarn.resourcemanager.zk-acl\":\"world:anyone:rwcda\",\"dfs.https.server.keystore.resource\":\"ssl-server.xml\",\"mapr......
> {quote}
> I am surprised that each vertex has a "userPayloadAsText",and "config" information is
particularly large.
> When I run a more complex pig on tez job, the tez-ui server is easy to crash.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message