datafu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Hayes <matthew.terence.ha...@gmail.com>
Subject Re: [jira] [Commented] (DATAFU-45) RFE: CartesianProduct
Date Wed, 30 Apr 2014 17:58:08 GMT
I can't tell from this what the problem is from this.  I would try creating
a test case that reproduces the issue.  If this is a Pig problem we can
file a JIRA and get it fixed.


On Wed, Apr 30, 2014 at 10:54 AM, Sam Steingold (JIRA) <jira@apache.org>wrote:

>
>     [
> https://issues.apache.org/jira/browse/DATAFU-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985834#comment-13985834]
>
> Sam Steingold commented on DATAFU-45:
> -------------------------------------
>
> same error with your code:
> {code}
> 2014-04-30 13:50:38,556 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - job job_201404290704_3659 has failed! Stop running all dependent jobs
> 2014-04-30 13:50:38,561 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - 100% complete
> 2014-04-30 13:50:38,905 [main] ERROR
> org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed!
> 2014-04-30 13:50:38,905 [main] INFO
>  org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics:
>
> HadoopVersion   PigVersion      UserId  StartedAt       FinishedAt
>  Features
> 2.0.0-cdh4.4.0  0.11.0-cdh4.4.0 sam     2014-04-30 13:49:11     2014-04-30
> 13:50:38     UNKNOWN
>
> Failed!
>
> Failed Jobs:
> JobId   Alias   Feature Message Outputs
> job_201404290704_3659   my_in,my_stage1,my_stage2       MAP_ONLY
>  Message: Job failed!    hdfs://
> hnn001.dev.smq.mgnt.cc:8020/tmp/temp63655099/tmp-776443090,
>
> Input(s):
> Failed to read data from "/user/sam/rollup"
>
> Output(s):
> Failed to produce result in "hdfs://
> hnn001.dev.smq.mgnt.cc:8020/tmp/temp63655099/tmp-776443090"
>
> Counters:
> Total records written : 0
> Total bytes written : 0
> Spillable Memory Manager spill count : 0
> Total bags proactively spilled: 0
> Total records proactively spilled: 0
>
> Job DAG:
> job_201404290704_3659
>
>
> 2014-04-30 13:50:38,905 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
> - Failed!
> 2014-04-30 13:50:38,933 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1066: Unable to open iterator for alias my_stage2
> {code}
>
> > RFE: CartesianProduct
> > ---------------------
> >
> >                 Key: DATAFU-45
> >                 URL: https://issues.apache.org/jira/browse/DATAFU-45
> >             Project: DataFu
> >          Issue Type: New Feature
> >            Reporter: Sam Steingold
> >
> > Given two bags, produce their [Cartesian product|
> http://en.wikipedia.org/wiki/Cartesian_product]:
> > {code}
> > B1: bag{T1}
> > B2: bag{T2}
> > CartesianProduct(B1,B2): bag{(T1,T2)}
> > {code}
> > Use case:
> > {code}
> > toks = TOKENIZE((charray)$0,',');
> > kwds = CartesianProduct(toks, {1.0/(double)SIZE(toks)});
> > {code}
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message