pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (PIG-4457) Error is thrown by JobStats.getOutputSize() when storing to a MySql table
Date Tue, 24 Mar 2015 09:33:53 GMT

     [ https://issues.apache.org/jira/browse/PIG-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Rohini Palaniswamy reopened PIG-4457:
-------------------------------------
      Assignee: Rohini Palaniswamy

Reopening to have pig.stats.output.size.reader.unsupported=org.apache.hcatalog.pig.HCatStorer,org.apache.hive.hcatalog.pig.HCatStorer,org.apache.pig.piggybank.storage.DBStorage

added to pig-default.properties.

> Error is thrown by JobStats.getOutputSize() when storing to a MySql table
> -------------------------------------------------------------------------
>
>                 Key: PIG-4457
>                 URL: https://issues.apache.org/jira/browse/PIG-4457
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Kunal Kumar
>            Assignee: Rohini Palaniswamy
>
> Here is an example of stack trace printed to console output. Actually, this is a warning
message and does not make the job fail. The data is getting stored to mysql table, but i have
no idea why pig is looking to store output on hdfs. I am using PIg along with Tez.
> using output size reader: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.FileBasedOutputSizeReader
> unable to find the output file
> java.io.FileNotFoundException: File hdfs://pts0021.persistent.co.in:9000/user/shareinsights/filtered_stock_data
does not exist.
>         at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:647)
>         at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:101)
>         at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:705)
>         at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:701)
>         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>         at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:701)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.FileBasedOutputSizeReader.getOutputSize(FileBasedOutputSizeReader.java:81)
>         at org.apache.pig.tools.pigstats.JobStats.getOutputSize(JobStats.java:351)
>         at org.apache.pig.tools.pigstats.tez.TezVertexStats.addOutputStatistics(TezVertexStats.java:270)
>         at org.apache.pig.tools.pigstats.tez.TezVertexStats.accumulateStats(TezVertexStats.java:188)
>         at org.apache.pig.tools.pigstats.tez.TezDAGStats.accumulateStats(TezDAGStats.java:209)
>         at org.apache.pig.tools.pigstats.tez.TezPigScriptStats.accumulateStats(TezPigScriptStats.java:180)
>         at org.apache.pig.backend.hadoop.executionengine.tez.TezJob.run(TezJob.java:194)
>         at org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher$1.run(TezLauncher.java:167)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message