hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: importtsv
Date Mon, 25 Apr 2011 19:42:04 GMT
Hi Eric,

Unfortunately, the LocalJobRunner is missing a feature that is causing the
bulk load option to fail.

Are you running a MapReduce cluster? Make sure that you've configured the
jobtracker address in your mapred-site.xml if so.

-Todd

On Fri, Apr 22, 2011 at 11:09 AM, Eric Ross <ericdross_2000@yahoo.com>wrote:

> Hi all,
>
> I'm having some trouble running the importtsv tool on CDH3B4 configured in
> pseudo distributed mode.
> The tool works fine unless I add the option importtsv.bulk.output.
>
> Does importtsv with the option importtsv.bulk.output work in pseudo
> distributed mode or do I maybe have something configured incorrectly?
>
> Here is some info on the error:
>
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job <init>
> WARNING: LocalJobRunner does not support symlinking into current working
> dir.
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink
> INFO: Creating symlink:
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> <- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink
> WARNING: Failed to create symlink:
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> <- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.JobClient
> monitorAndPrintJob
> INFO: Running job: job_local_0001
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> <init>
> INFO: io.sort.mb = 100
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> <init>
> INFO: data buffer = 79691776/99614720
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> <init>
> INFO: record buffer = 262144/327680
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job run
> WARNING: job_local_0001
> java.lang.IllegalArgumentException: Can't read partitions file
>        at
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:111)
>        at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
>        at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>        at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:559)
>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
>        at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> Caused by: java.io.FileNotFoundException: File _partition.lst does not
> exist.
>        at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
>        at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>        at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776)
>        at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
>        at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
>        at
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.readPartitions(TotalOrderPartitioner.java:296)
>        at
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:82)
>        ... 6 more
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/-8364002144339543919_194806607_1507402918/file/home/hadoop/test/java/lib/guava-r06.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/-7608337350018775429_-1267154261_925648918/file/home/hadoop/test/java/lib/hadoop-core-0.20.2-CDH3B4.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/6475934364733173115_-1837084859_925493918/file/home/hadoop/test/java/lib/hbase-0.90.1-CDH3B4.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/-5268899720351360254_-17093236_1440710918/file/home/hadoop/test/java/lib/zookeeper-3.3.2-CDH3B4.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient
> monitorAndPrintJob
> INFO:  map 0% reduce 0%
> Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient
> monitorAndPrintJob
> INFO: Job complete: job_local_0001
> Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.Counters log
> INFO: Counters: 0
>
> Thanks,
> Eric
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message