crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lucy Chen <lucychen2014f...@gmail.com>
Subject reporter of a Crunch job?
Date Tue, 07 Apr 2015 22:50:04 GMT
Hi,

     I implemented a MapFn ClassA, and in the map function I need call a
function from third party's library. But its function needs an input
parameters including a reporter of type Reporter, which is also a built-in
class of their library. That's why I have a child class CrunchReporter. But
I don't know when I create an instance of CurnchReporter, what's the
reporter and log that I can use as the inputs.

      I used Pig before. Its class EvalFunc has reporter and log as
built-in field, org.apache.pig.EvalFunc.reporter and
org.apache.pig.EvalFunc.log. I passed those fields for the input to new a
PigReporter. For Crunch jobs, like DoFn or MapFn, how can I specify the
inputs for the reporter and log, this.params = Params(params_input, ?
reporter, ?log).

   Thanks!

Lucy

public class *ClassA* extends MapFn< T, T>{

 public transient Params params;

 public ClassA(String params_input)

{

this.params = Params(params_input, ?reporter, ?log);

}

@Override

public map()

{

.....

}


}


public class Params implements java.io.Serializable{


    private CrunchReporter reporter = null;

    .....


      public LSRParams(String input, Progressable crunch_reporter, Log log)

{

             ..........

             this.reporter = new CrunchReporter(crunch_reporter, log);


        }

}


public class CrunchReporter extends Reporter {

 Progressable crunchReporter = null;

PrintStream out = null;

Log log = null;

 public CrunchReporter(Progressable crunchReporter, Log log)

{

super(LogLevel.DEBUG);

this.crunchReporter = crunchReporter;

this.log = log;

 }

}

Mime
View raw message