datafu-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jian wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DATAFU-33) Can't instantiate AliasableEvalFunc for unit-testing
Date Mon, 03 Mar 2014 00:45:20 GMT

     [ https://issues.apache.org/jira/browse/DATAFU-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

jian wang updated DATAFU-33:
----------------------------

    Attachment: 0001-add-unit-test-for-alias-eval-func-udf.patch

> Can't instantiate AliasableEvalFunc for unit-testing
> ----------------------------------------------------
>
>                 Key: DATAFU-33
>                 URL: https://issues.apache.org/jira/browse/DATAFU-33
>             Project: DataFu
>          Issue Type: Bug
>    Affects Versions: 1.3.0
>         Environment: CDH 4.4
>            Reporter: Sergey
>         Attachments: 0001-add-unit-test-for-alias-eval-func-udf.patch
>
>
> Hi, I can't make it work for 1.2 version too. Works perfectly inside script, can't instantiate
in for unit-testing.
> {code}
>     @Test
>     void test(){
>         def udf = new ReportBuilder('/any/path/to/2013/10/21', getBasePath('Partition_dict.csv'))
>         udf.setUDFContextSignature('test')
>         def schemaTuple = new Schema([
>                 new Schema.FieldSchema('msisdn', DataType.LONG),
>                 new Schema.FieldSchema('ts', DataType.INTEGER),
>                 new Schema.FieldSchema('center_lon', DataType.DOUBLE),
>                 new Schema.FieldSchema('center_lat', DataType.DOUBLE),
>         ])
>         def schemaBag = new Schema(new Schema.FieldSchema('orderedRoutes', schemaTuple,
DataType.BAG))
>         udf.getOutputSchema(schemaBag)
>         udf.exec(TupleFactory.instance.newTuple([71230000000L, 1382351612,	10.697D,	20.713D]))
>     }
>     /** A stub for testing files taken from distcache */
>     def getBasePath(String fileName){
>         FileUtils.toFile(this.class.classLoader.getResource(fileName)).parentFile.absolutePath
>     }
> {code}
> Script usage:
> {code}
>              orderedRoutes = ORDER routes BY ts;
>             GENERATE FLATTEN(ReportBuilder(orderedRoutes)) as (list_of_fields:anytype);
> {code}
> fails here:
> {code}
> @Override
>     DataBag exec(Tuple input) throws IOException {
>         def pivots = getBag(input, ORDERED_ROUTES).toList() //error happens here
>         def outputBag = BagFactory.instance.newDefaultBag()
>         //some code goes here
>         outputBag
>     }
> {code}
> {code}
> 14/02/24 17:55:40 ERROR udf.ReportBuilder: Class: class pig.udf.ReportBuilder
> 14/02/24 17:55:40 ERROR udf.ReportBuilder: Instance name: test
> 14/02/24 17:55:40 ERROR udf.ReportBuilder: Properties: {test={}}
> java.lang.RuntimeException: Could not retrieve aliases from properties using aliasMap
> 	at datafu.pig.util.AliasableEvalFunc.getFieldAliases(AliasableEvalFunc.java:164)
> 	at datafu.pig.util.AliasableEvalFunc.getPosition(AliasableEvalFunc.java:171)
> 	at datafu.pig.util.AliasableEvalFunc.getBag(AliasableEvalFunc.java:253)
> 	at datafu.pig.util.AliasableEvalFunc$getBag.callCurrent(Unknown Source)
> 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:145)
> 	at udf.ReportBuilder.exec(ReportBuilder.groovy:26)
> 	at pig.udf.ReportBuilder$exec.call(Unknown Source)
> 	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
> 	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
> 	at udf.ReportBuilderTest.test(ReportBuilderTest.groovy:28)
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message