hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From velo0001 <velo0...@myid.me>
Subject Re: Exception in rowcount program
Date Thu, 13 Aug 2009 14:23:42 GMT

Copying the hbase-0.20.0.jar and the hbase-site.xml files into the Hadoop
deployment location DID solve the above class not found issue. 

Then it showed up that I also needed the zookeeper jar file in HBase
available to Hadoop as well. 

This time modifying the HADOOP_CLASSPATH in the *env.sh DID allow running
Hadoop to resolve zoo keeper classes, as it did NOT with HBase classes. To
repeat, I did NOT need to copy the zoo keeper jar file into Hadoop location,
as I did have to with the HBase jar.

So, FWIW, the above is my experience. And it seems to mirror the original
posters experience.

The MapReduce java code process runs fine now:

DEBUG 10:02:52.449 org.apache.hadoop.mapred.JobClient (JobClient.java:776) -
Creating splits at
hdfs://devdkvstore.headquarters.socketware.com/tmp/hadoop-root/mapred/system/job_200908130817_0001/job.split
 INFO 10:02:52.465 org.apache.hadoop.mapred.FileInputFormat
(FileInputFormat.java:192) - Total input paths to process : 1
DEBUG 10:02:52.480 org.apache.hadoop.mapred.FileInputFormat
(FileInputFormat.java:248) - Total # of splits: 2
 INFO 10:02:53.073 org.apache.hadoop.mapred.JobClient (JobClient.java:1278)
- Running job: job_200908130817_0001
 INFO 10:02:54.088 org.apache.hadoop.mapred.JobClient (JobClient.java:1291)
-  map 0% reduce 0%
 INFO 10:03:16.459 org.apache.hadoop.mapred.JobClient (JobClient.java:1291)
-  map 100% reduce 0%
 INFO 10:03:24.779 org.apache.hadoop.mapred.JobClient (JobClient.java:1291)
-  map 100% reduce 100%
 INFO 10:03:26.793 org.apache.hadoop.mapred.JobClient (JobClient.java:1346)
- Job complete: job_200908130817_0001
DEBUG 10:03:26.809 org.apache.hadoop.mapred.Counters (Counters.java:151) -
Creating group org.apache.hadoop.mapred.JobInProgress$Counter with bundle
DEBUG 10:03:26.809 org.apache.hadoop.mapred.Counters (Counters.java:151) -
Creating group FileSystemCounters with nothing
DEBUG 10:03:26.809 org.apache.hadoop.mapred.Counters (Counters.java:151) -
Creating group org.apache.hadoop.mapred.Task$Counter with bundle
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:514) -
Counters: 17
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:516) -  
Job Counters 
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Launched reduce tasks=1
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Launched map tasks=2
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Data-local map tasks=2
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:516) -  
FileSystemCounters
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
FILE_BYTES_READ=4387
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
HDFS_BYTES_READ=5073
 INFO 10:03:26.809 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
FILE_BYTES_WRITTEN=8844
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:516) -  
Map-Reduce Framework
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Reduce input groups=100
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Combine output records=0
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Map input records=100
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Reduce shuffle bytes=4393
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Reduce output records=100
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Spilled Records=200
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Map output bytes=4181
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Map input bytes=3381
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Combine input records=0
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Map output records=100
 INFO 10:03:26.824 org.apache.hadoop.mapred.JobClient (Counters.java:518) -    
Reduce input records=100


Now, onward to learning about and using PIG....

_mc

-- 
View this message in context: http://www.nabble.com/Exception-in-rowcount-program-tp23195921p24954786.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message