hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Ross <ericdross_2...@yahoo.com>
Subject Re: importtsv
Date Wed, 27 Apr 2011 19:04:17 GMT
I'm not running it on a cluster but on my local machine in pseudo distributed mode.

The jobtracker address in mapred-site.xml is set to localhost and changing it to my system's
ip didn't make any difference.

Do you have suggestions for any other features/options that I should check?


--- On Mon, 4/25/11, Todd Lipcon <todd@cloudera.com> wrote:

> From: Todd Lipcon <todd@cloudera.com>
> Subject: Re: importtsv
> To: user@hbase.apache.org, ericdross_2000@yahoo.com
> Date: Monday, April 25, 2011, 12:42 PM
> Hi Eric,
> 
> Unfortunately, the LocalJobRunner is missing a feature that
> is causing the
> bulk load option to fail.
> 
> Are you running a MapReduce cluster? Make sure that you've
> configured the
> jobtracker address in your mapred-site.xml if so.
> 
> -Todd
> 
> On Fri, Apr 22, 2011 at 11:09 AM, Eric Ross <ericdross_2000@yahoo.com>wrote:
> 
> > Hi all,
> >
> > I'm having some trouble running the importtsv tool on
> CDH3B4 configured in
> > pseudo distributed mode.
> > The tool works fine unless I add the option
> importtsv.bulk.output.
> >
> > Does importtsv with the option importtsv.bulk.output
> work in pseudo
> > distributed mode or do I maybe have something
> configured incorrectly?
> >
> > Here is some info on the error:
> >
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.LocalJobRunner$Job <init>
> > WARNING: LocalJobRunner does not support symlinking
> into current working
> > dir.
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.TaskRunner symlink
> > INFO: Creating symlink:
> >
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> > <-
> /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.TaskRunner symlink
> > WARNING: Failed to create symlink:
> >
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> > <-
> /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.JobClient
> > monitorAndPrintJob
> > INFO: Running job: job_local_0001
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > <init>
> > INFO: io.sort.mb = 100
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > <init>
> > INFO: data buffer = 79691776/99614720
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > <init>
> > INFO: record buffer = 262144/327680
> > Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.mapred.LocalJobRunner$Job run
> > WARNING: job_local_0001
> > java.lang.IllegalArgumentException: Can't read
> partitions file
> >        at
> >
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:111)
> >        at
> >
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> >        at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> >        at
> >
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:559)
> >        at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638)
> >        at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
> >        at
> >
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> > Caused by: java.io.FileNotFoundException: File
> _partition.lst does not
> > exist.
> >        at
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
> >        at
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> >        at
> org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776)
> >        at
> >
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
> >        at
> >
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
> >        at
> >
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.readPartitions(TotalOrderPartitioner.java:296)
> >        at
> >
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:82)
> >        ... 6 more
> > Apr 22, 2011 9:35:40 AM
> >
> org.apache.hadoop.filecache.TrackerDistributedCacheManager
> deleteLocalPath
> > INFO: Deleted path
> >
> /tmp/hadoop-hadoop/mapred/local/archive/-8364002144339543919_194806607_1507402918/file/home/hadoop/test/java/lib/guava-r06.jar
> > Apr 22, 2011 9:35:40 AM
> >
> org.apache.hadoop.filecache.TrackerDistributedCacheManager
> deleteLocalPath
> > INFO: Deleted path
> >
> /tmp/hadoop-hadoop/mapred/local/archive/-7608337350018775429_-1267154261_925648918/file/home/hadoop/test/java/lib/hadoop-core-0.20.2-CDH3B4.jar
> > Apr 22, 2011 9:35:40 AM
> >
> org.apache.hadoop.filecache.TrackerDistributedCacheManager
> deleteLocalPath
> > INFO: Deleted path
> >
> /tmp/hadoop-hadoop/mapred/local/archive/6475934364733173115_-1837084859_925493918/file/home/hadoop/test/java/lib/hbase-0.90.1-CDH3B4.jar
> > Apr 22, 2011 9:35:40 AM
> >
> org.apache.hadoop.filecache.TrackerDistributedCacheManager
> deleteLocalPath
> > INFO: Deleted path
> >
> /tmp/hadoop-hadoop/mapred/local/archive/-5268899720351360254_-17093236_1440710918/file/home/hadoop/test/java/lib/zookeeper-3.3.2-CDH3B4.jar
> > Apr 22, 2011 9:35:40 AM
> >
> org.apache.hadoop.filecache.TrackerDistributedCacheManager
> deleteLocalPath
> > INFO: Deleted path
> >
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> > Apr 22, 2011 9:35:41 AM
> org.apache.hadoop.mapred.JobClient
> > monitorAndPrintJob
> > INFO:  map 0% reduce 0%
> > Apr 22, 2011 9:35:41 AM
> org.apache.hadoop.mapred.JobClient
> > monitorAndPrintJob
> > INFO: Job complete: job_local_0001
> > Apr 22, 2011 9:35:41 AM
> org.apache.hadoop.mapred.Counters log
> > INFO: Counters: 0
> >
> > Thanks,
> > Eric
> >
> >
> 
> 
> -- 
> Todd Lipcon
> Software Engineer, Cloudera
> 

Mime
View raw message