hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Hbase and Hadoop Config to run in Standalone mode
Date Thu, 23 Jul 2009 15:36:51 GMT
What you wrote is a bit confusing to me, sorry.

The usual way to debug MR jobs is to define a logger and post with
either info or debug level, not sysout like you did. I'm not even sure
where the standard output is logged when using a local job. Also since
this is local you won't see anything in your host:50030 web UI. So use
apache common logging and you should see your output.

J-D

On Thu, Jul 23, 2009 at 11:13 AM, bharath
vissapragada<bharathvissapragada1990@gmail.com> wrote:
> Thanks for ur reply J-D ... Im pasting some part of the code ...
>
> Im doing it frm the command line .. Iam pasting some part of the code here
> ....
>
>  public void mapp(ImmutableBytesWritable row, RowResult value,
> OutputCollector<Text, Text> output, Reporter reporter) throws IOException {
>                System.out.println(row);
> }
>
> public JobConf createSubmittableJob(String[] args) throws IOException {
>                JobConf c = new JobConf(getConf(), MR_DS_Scan_Case1.class);
>                c.set("col.name", args[1]);
>                c.set("operator.name",args[2]);
>                c.set("val.name",args[3]);
>                IdentityTableMap.initJob(args[0], args[1], this.getClass(),
> c);
>                c.setOutputFormat(NullOutputFormat.class);
>                 return c
> }
>
> As u can see ... im just printing the value of row in the map .. i can't see
> in the terminal .....
> I only wan't the map phase ... so i didn't write any reduce phase .. is my
> jobConf correct??
>
> Also as i have already asked how to check the job logs and web interface
> like "localhost:<port>/jobTracker.jsp"... since im running in local mode ...
>
> On Thu, Jul 23, 2009 at 6:32 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:
>
>> What output do you need exactly? I see that you have 8 output records
>> in your reduce task so if you take a look in your output folder or
>> table (I don't know which sink you used) you should see them.
>>
>> Also did you run your MR inside Eclipse or in command line?
>>
>> Thx,
>>
>> J-D
>>
>> On Thu, Jul 23, 2009 at 8:30 AM, bharath
>> vissapragada<bharat_v@students.iiit.ac.in> wrote:
>> > This is the output i go t.. seems everything is fine ..but no output!!
>> >
>> > 09/07/23 23:25:36 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> > processName=JobTracker, sessionId=
>> > 09/07/23 23:25:36 WARN mapred.JobClient: No job jar file set.  User
>> classes
>> > may not be found. See JobConf(Class) or JobConf#setJar(String).
>> > 09/07/23 23:25:36 INFO mapred.TableInputFormatBase: split:
>> > 0->localhost.localdomain:,
>> > 09/07/23 23:25:37 INFO mapred.JobClient: Running job: job_local_0001
>> > 09/07/23 23:25:37 INFO mapred.TableInputFormatBase: split:
>> > 0->localhost.localdomain:,
>> > 09/07/23 23:25:37 INFO mapred.MapTask: numReduceTasks: 1
>> > 09/07/23 23:25:37 INFO mapred.MapTask: io.sort.mb = 100
>> > 09/07/23 23:25:37 INFO mapred.MapTask: data buffer = 79691776/99614720
>> > 09/07/23 23:25:37 INFO mapred.MapTask: record buffer = 262144/327680
>> > 09/07/23 23:25:37 INFO mapred.MapTask: Starting flush of map output
>> > 09/07/23 23:25:37 INFO mapred.MapTask: Finished spill 0
>> > 09/07/23 23:25:37 INFO mapred.TaskRunner:
>> Task:attempt_local_0001_m_000000_0
>> > is done. And is in the process of commiting
>> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner:
>> > 09/07/23 23:25:37 INFO mapred.TaskRunner: Task
>> > 'attempt_local_0001_m_000000_0' done.
>> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner:
>> > 09/07/23 23:25:37 INFO mapred.Merger: Merging 1 sorted segments
>> > 09/07/23 23:25:37 INFO mapred.Merger: Down to the last merge-pass, with 1
>> > segments left of total size: 333 bytes
>> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner:
>> > 09/07/23 23:25:37 INFO mapred.TaskRunner:
>> Task:attempt_local_0001_r_000000_0
>> > is done. And is in the process of commiting
>> > 09/07/23 23:25:37 INFO mapred.LocalJobRunner: reduce > reduce
>> > 09/07/23 23:25:37 INFO mapred.TaskRunner: Task
>> > 'attempt_local_0001_r_000000_0' done.
>> > 09/07/23 23:25:38 INFO mapred.JobClient: Job complete: job_local_0001
>> > 09/07/23 23:25:38 INFO mapred.JobClient: Counters: 11
>> > 09/07/23 23:25:38 INFO mapred.JobClient:   File Systems
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Local bytes read=38949
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Local bytes written=78378
>> > 09/07/23 23:25:38 INFO mapred.JobClient:   Map-Reduce Framework
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Reduce input groups=8
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Combine output records=0
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Map input records=8
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Reduce output records=8
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Map output bytes=315
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Map input bytes=0
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Combine input records=0
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Map output records=8
>> > 09/07/23 23:25:38 INFO mapred.JobClient:     Reduce input records=8
>> >
>> >
>> > On Thu, Jul 23, 2009 at 12:17 PM, bharath vissapragada <
>> > bharat_v@students.iiit.ac.in> wrote:
>> >
>> >> since i haven;t started the cluster .. i can even see the details in
>> >> "localhost:<port>/jobTracker.jsp" ..  i didn't even add anything
to
>> >> hadoop/conf/hadoop-site.xml
>> >>
>> >>
>> >> On Thu, Jul 23, 2009 at 12:16 PM, bharath vissapragada <
>> >> bharat_v@students.iiit.ac.in> wrote:
>> >>
>> >>> Hi all ,
>> >>>
>> >>> I wanted to run HBase in standalone mode to check my Hbase MR programs
>> ...
>> >>> I have dl a built version of hbase-0.20. and i have hadoop 0.19.3
>> >>>
>> >>> "I have set JAVA_HOME in both of them" .. then i started hbase and
>> >>> inserted some tables using JAVA API .. Now i have written some MR
>> programs
>> >>> onHBase and when i run them on Hbase it runs perfectly without any
>> errors
>> >>> and all the Map -reduce statistics are displayed correctly but  i get
>> no
>> >>> output .
>> >>>
>> >>> I have one doubt now .. how do HBase recognize hadoop in stand alone
>> >>> mode(i haven;t started my hadoop even) .. Even simple print statements
>> donot
>> >>> work .. no output is displayed on the screen ... I doubt my config ....
>> >>>
>> >>> Do i need to add some config to run them ... Please reply ...
>> >>>
>> >>
>> >>
>> >
>>
>

Mime
View raw message