hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-180) BinaryStorage is implicitly used where it should not be
Date Thu, 03 Apr 2008 01:24:24 GMT

     [ https://issues.apache.org/jira/browse/PIG-180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arun C Murthy updated PIG-180:
------------------------------

    Attachment: PIG-180_0_20080402.patch

This patch fixes a bug in the {Load|Store}Optimizer where the input/output {Load|Store}Func
of the StreamingCommand wasn't being set correctly in some cases when it got flipped to BinaryStorage.


> BinaryStorage is implicitly used where it should not be
> -------------------------------------------------------
>
>                 Key: PIG-180
>                 URL: https://issues.apache.org/jira/browse/PIG-180
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Xu Zhang
>            Assignee: Arun C Murthy
>         Attachments: PIG-180_0_20080402.patch, script.pig, streaming.pl
>
>
> When I ran the attached Pig script, I got unexpected resulting output data.  After a
closer at Pig's console output, it appeared BinaryStorage is incorrectly used:
> {noformat}
> I can't find HOD configuration for piglet, hopefully you weren't planning on using HOD.
> 2008-04-02 15:39:11,668 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
- Connecting to hadoop file system at: wilbur11.labs.corp.sp1.yahoo.com:8020
> 2008-04-02 15:39:12,478 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- ----- MapReduce Job -----
> 2008-04-02 15:39:12,478 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Input: [/user/pig/tests/data/singlefile/studenttab10k:org.apache.pig.builtin.PigStorage()]
> 2008-04-02 15:39:12,478 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Map: [[*]->org.apache.pig.impl.eval.StreamSpec@181edf4]
> 2008-04-02 15:39:12,479 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Group: null
> 2008-04-02 15:39:12,479 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Combine: null
> 2008-04-02 15:39:12,479 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Reduce: null
> 2008-04-02 15:39:12,479 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Output: results_7:org.apache.pig.builtin.BinaryStorage
> 2008-04-02 15:39:12,479 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Split: null
> 2008-04-02 15:39:12,480 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Map parallelism: -1
> 2008-04-02 15:39:12,480 [main] INFO  org.apache.pig.backend.hadoop.executionengine.POMapreduce
- Reduce parallelism: -1
> 219190 hdfs://wilbur11.labs.corp.sp1.yahoo.com:8020/user/pig/tests/data/singlefile/studenttab10k
> 2008-04-02 15:39:15,308 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapreduceExec.MapReduceLauncher
- Pig progress = 0%
> 2008-04-02 15:39:20,329 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapreduceExec.MapReduceLauncher
- Pig progress = 50%
> 2008-04-02 15:39:26,355 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapreduceExec.MapReduceLauncher
- Pig progress = 60%
> 2008-04-02 15:39:28,395 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapreduceExec.MapReduceLauncher
- Pig progress = 80%
> 2008-04-02 15:39:30,407 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapreduceExec.MapReduceLauncher
- Pig progress = 90%
> 2008-04-02 15:39:32,415 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapreduceExec.MapReduceLauncher
- Pig progress = 100%
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message