Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of todd@cloudera.com designates
 209.85.214.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <296778.81048.qm@web80015.mail.sp1.yahoo.com>
References: <BANLkTimU+RMnC0r==Rnva1zb3rGTvTcQwQ@mail.gmail.com>
 <296778.81048.qm@web80015.mail.sp1.yahoo.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Wed, 27 Apr 2011 14:52:52 -0700
Message-ID: <BANLkTi=k+0kwwhvm-Ff+L3x_aFF-CANd-g@mail.gmail.com>
Subject: Re: importtsv
To: user@hbase.apache.org, ericdross_2000@yahoo.com
Content-Type: multipart/alternative; boundary=0016e6d7e906915bcb04a1ed77e7

--0016e6d7e906915bcb04a1ed77e7
Content-Type: text/plain; charset=ISO-8859-1

On Wed, Apr 27, 2011 at 12:04 PM, Eric Ross <ericdross_2000@yahoo.com>wrote:

> I'm not running it on a cluster but on my local machine in pseudo
> distributed mode.
>
> The jobtracker address in mapred-site.xml is set to localhost and changing
> it to my system's ip didn't make any difference.
>

The importtsv program doesn't appear to be picking up mapred-site.xml, then.
Are you sure it's valid XML? You can try "xmllint" to verify. Perhaps attach
it here?

-Todd


>
> Do you have suggestions for any other features/options that I should check?
>
>
> --- On Mon, 4/25/11, Todd Lipcon <todd@cloudera.com> wrote:
>
> > From: Todd Lipcon <todd@cloudera.com>
> > Subject: Re: importtsv
> > To: user@hbase.apache.org, ericdross_2000@yahoo.com
> > Date: Monday, April 25, 2011, 12:42 PM
> > Hi Eric,
> >
> > Unfortunately, the LocalJobRunner is missing a feature that
> > is causing the
> > bulk load option to fail.
> >
> > Are you running a MapReduce cluster? Make sure that you've
> > configured the
> > jobtracker address in your mapred-site.xml if so.
> >
> > -Todd
> >
> > On Fri, Apr 22, 2011 at 11:09 AM, Eric Ross <ericdross_2000@yahoo.com
> >wrote:
> >
> > > Hi all,
> > >
> > > I'm having some trouble running the importtsv tool on
> > CDH3B4 configured in
> > > pseudo distributed mode.
> > > The tool works fine unless I add the option
> > importtsv.bulk.output.
> > >
> > > Does importtsv with the option importtsv.bulk.output
> > work in pseudo
> > > distributed mode or do I maybe have something
> > configured incorrectly?
> > >
> > > Here is some info on the error:
> > >
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.LocalJobRunner$Job <init>
> > > WARNING: LocalJobRunner does not support symlinking
> > into current working
> > > dir.
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.TaskRunner symlink
> > > INFO: Creating symlink:
> > >
> >
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> > > <-
> > /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.TaskRunner symlink
> > > WARNING: Failed to create symlink:
> > >
> >
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> > > <-
> > /tmp/hadoop-hadoop/mapred/local/localRunner/_partition.lst
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.JobClient
> > > monitorAndPrintJob
> > > INFO: Running job: job_local_0001
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > > <init>
> > > INFO: io.sort.mb = 100
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > > <init>
> > > INFO: data buffer = 79691776/99614720
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.MapTask$MapOutputBuffer
> > > <init>
> > > INFO: record buffer = 262144/327680
> > > Apr 22, 2011 9:35:40 AM
> > org.apache.hadoop.mapred.LocalJobRunner$Job run
> > > WARNING: job_local_0001
> > > java.lang.IllegalArgumentException: Can't read
> > partitions file
> > >        at
> > >
> >
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:111)
> > >        at
> > >
> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
> > >        at
> > >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> > >        at
> > >
> >
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:559)
> > >        at
> > org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:638)
> > >        at
> > org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
> > >        at
> > >
> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
> > > Caused by: java.io.FileNotFoundException: File
> > _partition.lst does not
> > > exist.
> > >        at
> > >
> >
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
> > >        at
> > >
> >
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> > >        at
> > org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776)
> > >        at
> > >
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
> > >        at
> > >
> > org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
> > >        at
> > >
> >
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.readPartitions(TotalOrderPartitioner.java:296)
> > >        at
> > >
> >
> org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:82)
> > >        ... 6 more
> > > Apr 22, 2011 9:35:40 AM
> > >
> > org.apache.hadoop.filecache.TrackerDistributedCacheManager
> > deleteLocalPath
> > > INFO: Deleted path
> > >
> >
> /tmp/hadoop-hadoop/mapred/local/archive/-8364002144339543919_194806607_1507402918/file/home/hadoop/test/java/lib/guava-r06.jar
> > > Apr 22, 2011 9:35:40 AM
> > >
> > org.apache.hadoop.filecache.TrackerDistributedCacheManager
> > deleteLocalPath
> > > INFO: Deleted path
> > >
> >
> /tmp/hadoop-hadoop/mapred/local/archive/-7608337350018775429_-1267154261_925648918/file/home/hadoop/test/java/lib/hadoop-core-0.20.2-CDH3B4.jar
> > > Apr 22, 2011 9:35:40 AM
> > >
> > org.apache.hadoop.filecache.TrackerDistributedCacheManager
> > deleteLocalPath
> > > INFO: Deleted path
> > >
> >
> /tmp/hadoop-hadoop/mapred/local/archive/6475934364733173115_-1837084859_925493918/file/home/hadoop/test/java/lib/hbase-0.90.1-CDH3B4.jar
> > > Apr 22, 2011 9:35:40 AM
> > >
> > org.apache.hadoop.filecache.TrackerDistributedCacheManager
> > deleteLocalPath
> > > INFO: Deleted path
> > >
> >
> /tmp/hadoop-hadoop/mapred/local/archive/-5268899720351360254_-17093236_1440710918/file/home/hadoop/test/java/lib/zookeeper-3.3.2-CDH3B4.jar
> > > Apr 22, 2011 9:35:40 AM
> > >
> > org.apache.hadoop.filecache.TrackerDistributedCacheManager
> > deleteLocalPath
> > > INFO: Deleted path
> > >
> >
> /tmp/hadoop-hadoop/mapred/local/archive/953502662101888516_-198765657_2115049918/file/home/hadoop/test/java/partitions_1303490140287
> > > Apr 22, 2011 9:35:41 AM
> > org.apache.hadoop.mapred.JobClient
> > > monitorAndPrintJob
> > > INFO:  map 0% reduce 0%
> > > Apr 22, 2011 9:35:41 AM
> > org.apache.hadoop.mapred.JobClient
> > > monitorAndPrintJob
> > > INFO: Job complete: job_local_0001
> > > Apr 22, 2011 9:35:41 AM
> > org.apache.hadoop.mapred.Counters log
> > > INFO: Counters: 0
> > >
> > > Thanks,
> > > Eric
> > >
> > >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>


-- 
Todd Lipcon
Software Engineer, Cloudera

--0016e6d7e906915bcb04a1ed77e7--