Mailing-List: contact dev-help@datafu.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@datafu.incubator.apache.org
Date: Sun, 2 Mar 2014 12:37:19 +0000 (UTC)
From: "jian wang (JIRA)" <jira@apache.org>
To: dev@datafu.incubator.apache.org
Message-ID: <JIRA.12697086.1393313393749.131948.1393763839766@arcas>
In-Reply-To: <JIRA.12697086.1393313393749@arcas>
References: <JIRA.12697086.1393313393749@arcas>
Subject: [jira] [Commented] (DATAFU-33) Can't instantiate AliasableEvalFunc
 for unit-testing
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/DATAFU-33?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13917387#comment-13917387 ] 

jian wang commented on DATAFU-33:
---------------------------------

In the test() method, please change this line:
    udf.getOutputSchema(schemaBag)
to
    udf.outputSchema(schemaBag).
and try again.

The outputSchema() function calls storeFieldAliases() to set up the alias map.

I have also created a unit test of the AliasEvalFunc based on your use case, please check: https://reviews.apache.org/r/18670/


> Can't instantiate AliasableEvalFunc for unit-testing
> ----------------------------------------------------
>
>                 Key: DATAFU-33
>                 URL: https://issues.apache.org/jira/browse/DATAFU-33
>             Project: DataFu
>          Issue Type: Bug
>    Affects Versions: 1.3.0
>         Environment: CDH 4.4
>            Reporter: Sergey
>
> Hi, I can't make it work for 1.2 version too. Works perfectly inside script, can't instantiate in for unit-testing.
> {code}
>     @Test
>     void test(){
>         def udf = new ReportBuilder('/any/path/to/2013/10/21', getBasePath('Partition_dict.csv'))
>         udf.setUDFContextSignature('test')
>         def schemaTuple = new Schema([
>                 new Schema.FieldSchema('msisdn', DataType.LONG),
>                 new Schema.FieldSchema('ts', DataType.INTEGER),
>                 new Schema.FieldSchema('center_lon', DataType.DOUBLE),
>                 new Schema.FieldSchema('center_lat', DataType.DOUBLE),
>         ])
>         def schemaBag = new Schema(new Schema.FieldSchema('orderedRoutes', schemaTuple, DataType.BAG))
>         udf.getOutputSchema(schemaBag)
>         udf.exec(TupleFactory.instance.newTuple([71230000000L, 1382351612,	10.697D,	20.713D]))
>     }
>     /** A stub for testing files taken from distcache */
>     def getBasePath(String fileName){
>         FileUtils.toFile(this.class.classLoader.getResource(fileName)).parentFile.absolutePath
>     }
> {code}
> Script usage:
> {code}
>              orderedRoutes = ORDER routes BY ts;
>             GENERATE FLATTEN(ReportBuilder(orderedRoutes)) as (list_of_fields:anytype);
> {code}
> fails here:
> {code}
> @Override
>     DataBag exec(Tuple input) throws IOException {
>         def pivots = getBag(input, ORDERED_ROUTES).toList() //error happens here
>         def outputBag = BagFactory.instance.newDefaultBag()
>         //some code goes here
>         outputBag
>     }
> {code}
> {code}
> 14/02/24 17:55:40 ERROR udf.ReportBuilder: Class: class pig.udf.ReportBuilder
> 14/02/24 17:55:40 ERROR udf.ReportBuilder: Instance name: test
> 14/02/24 17:55:40 ERROR udf.ReportBuilder: Properties: {test={}}
> java.lang.RuntimeException: Could not retrieve aliases from properties using aliasMap
> 	at datafu.pig.util.AliasableEvalFunc.getFieldAliases(AliasableEvalFunc.java:164)
> 	at datafu.pig.util.AliasableEvalFunc.getPosition(AliasableEvalFunc.java:171)
> 	at datafu.pig.util.AliasableEvalFunc.getBag(AliasableEvalFunc.java:253)
> 	at datafu.pig.util.AliasableEvalFunc$getBag.callCurrent(Unknown Source)
> 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:145)
> 	at udf.ReportBuilder.exec(ReportBuilder.groovy:26)
> 	at pig.udf.ReportBuilder$exec.call(Unknown Source)
> 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
> 	at udf.ReportBuilderTest.test(ReportBuilderTest.groovy:28)
> {code}


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)