Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of todd@cloudera.com designates
 209.85.214.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <81117.94398.qm@web80013.mail.sp1.yahoo.com>
References: <81117.94398.qm@web80013.mail.sp1.yahoo.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Mon, 25 Apr 2011 12:42:04 -0700
Message-ID: <BANLkTimU+RMnC0r==Rnva1zb3rGTvTcQwQ@mail.gmail.com>
Subject: Re: importtsv
To: user@hbase.apache.org, ericdross_2000@yahoo.com
Content-Type: multipart/alternative; boundary=0015175cd5b81ba68704a1c36847

--0015175cd5b81ba68704a1c36847
Content-Type: text/plain; charset=ISO-8859-1

Hi Eric,

Unfortunately, the LocalJobRunner is missing a feature that is causing the
bulk load option to fail.

Are you running a MapReduce cluster? Make sure that you've configured the
jobtracker address in your mapred-site.xml if so.

-Todd

On Fri, Apr 22, 2011 at 11:09 AM, Eric Ross <ericdross_2000@yahoo.com>wrote:

> Hi all,
>
> I'm having some trouble running the importtsv tool on CDH3B4 configured in
> pseudo distributed mode.
> The tool works fine unless I add the option importtsv.bulk.output.
>
> Does importtsv with the option importtsv.bulk.output work in pseudo
> distributed mode or do I maybe have something configured incorrectly?
>
> Here is some info on the error:
>
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job <init>
> WARNING: LocalJobRunner does not support symlinking into current working
> dir.
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink
> INFO: Creating symlink:
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> <- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.TaskRunner symlink
> WARNING: Failed to create symlink:
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> <- /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.JobClient
> monitorAndPrintJob
> INFO: Running job: job_local_0001
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> <init>
> INFO: io.sort.mb = 100
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> <init>
> INFO: data buffer = 79691776/99614720
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> <init>
> INFO: record buffer = 262144/327680
> Apr 22, 2011 9:35:40 AM org.apache.hadoop.mapred.LocalJobRunner$Job run
> WARNING: job_local_0001
> java.lang.IllegalArgumentException: Can't read partitions file
>        at
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:111)
>        at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
>        at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>        at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:559)
>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
>        at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> Caused by: java.io.FileNotFoundException: File _partition.lst does not
> exist.
>        at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
>        at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>        at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776)
>        at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
>        at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
>        at
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.readPartitions(TotalOrderPartitioner.java:296)
>        at
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:82)
>        ... 6 more
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/-8364002144339543919_194806607_1507402918/file/home/hadoop/test/java/lib/guava-r06.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/-7608337350018775429_-1267154261_925648918/file/home/hadoop/test/java/lib/hadoop-core-0.20.2-CDH3B4.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/6475934364733173115_-1837084859_925493918/file/home/hadoop/test/java/lib/hbase-0.90.1-CDH3B4.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/-5268899720351360254_-17093236_1440710918/file/home/hadoop/test/java/lib/zookeeper-3.3.2-CDH3B4.jar
> Apr 22, 2011 9:35:40 AM
> org.apache.hadoop.filecache.TrackerDistributedCacheManager deleteLocalPath
> INFO: Deleted path
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient
> monitorAndPrintJob
> INFO:  map 0% reduce 0%
> Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.JobClient
> monitorAndPrintJob
> INFO: Job complete: job_local_0001
> Apr 22, 2011 9:35:41 AM org.apache.hadoop.mapred.Counters log
> INFO: Counters: 0
>
> Thanks,
> Eric
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

--0015175cd5b81ba68704a1c36847--