hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Kellerman <...@powerset.com>
Subject RE: NoSuchElementException when creating a table
Date Wed, 07 Nov 2007 17:18:22 GMT
+1

---
Jim Kellerman, Senior Engineer; Powerset
jim@powerset.com


> -----Original Message-----
> From: Holger Stenzhorn [mailto:holger.stenzhorn@gmail.com]
> Sent: Wednesday, November 07, 2007 4:42 AM
> To: hadoop-user@lucene.apache.org
> Subject: Re: NoSuchElementException when creating a table
>
> Ok, here is my first shot at it:
>
> --- SNIP ---
>
> 3.1. Set the hadoop tmp directory property, hadoop.tmp.dir
> (For Windows/Cygwin users: If you do not include this
> variable in the site file then the value
> /tmp/hadoop-${user.name} from hadoop-default.xml is used. But
> since this does not work properly for Windows/Cygwin you have
> to specify a Windows-formatted path for hadoop.tmp.dir such
> as c:\tmp\hbase)
>
> --- SNIP ---
>
> Cheers,
> Holger
>
> Michael Stack wrote:
> > Holger Stenzhorn wrote:
> >> Perhaps you could add to the "10 minutes" page at point 3.1 an
> >> example for Unix and Windows/Cygwin to make the different path
> >> handling a bit clearer.
> >>
> > Sounds like a good idea to me Holger.  Would you mind
> taking a first
> > cut at it since you've had first hand-experience of where
> the doc. is
> > lacking?
> >
> > St.Ack
> >
> >> Cheers,
> >> Holger
> >>
> >> Michael Stack wrote:
> >>> Holger Stenzhorn wrote:
> >>>>>>> Hadoop RunJar looks to contain the only use of hadoop.tmp.dir
> >>>>>>> other than test and contrib code.  It does the
> following: 'new
> >>>>>>> File(new Configuration().get("hadoop.tmp.dir"));' using
> >>>>>>> java.io.File.  In hbase, its all Path and hadoop
> FileSystem. So,
> >>>>>>> yes, content of hadoop.tmp.dir is treated differently.
> >>>>>> Could that be changed? ...meaning, wouldn't it be sensible to
> >>>>>> change HBase (to use Path etc.) so it behaves like Hadoop?
> >>>>>
> >>>>> You mean, can hadoop be changed?  (Its hbase that's drunk the
> >>>>> kool-aid; its the one that goes via hadoop FileSystem for all
> >>>>> filesystem accesses whereas the hadoop use of
> hadoop.tmp.dir goes
> >>>>> via java.io.File).
> >>>> Nope. I mean the other way around: Can you change the
> HBase code so
> >>>> it goes via the Hadoop filesystem? ...or would this imply a
> >>>> performance drop for HBase?
> >>> It already does (See my comments above).
> >>>
> >>>
> >>>>> What happens if you clean up your local directory removing
> >>>>> ${hadoop.tmp.dir} and retry?
> >>>> Again, if I use "c:\tmp\hadoop-holste" as value for
> >>>> "${hadoop.tmp.dir}" it works: The log reflects just the
> situation
> >>>> when this variable is not set in "hbase-site.xml" - then it does
> >>>> not work.
> >>>> I also forgot to mention in my last my that it is not just
> >>>> "${hadoop.tmp.dir}" set to "/tmp/hadoop-${user.name}"
> that causes
> >>>> the problem but also if it is set to "/tmp/hadoop-holste".
> >>>
> >>> Did you try my suggestion of removing anything that might
> once have
> >>> been a value for hadoop.tmp.dir?
> >>>
> >>> Testing here on cygwin using local filesystem and all
> defaults, it
> >>> all just works fine.  I'm disturbed by the state you seem
> to be in
> >>> where the .META. table is not being deployed AND there is no
> >>> exception saying why.   The only think I can think is
> that previous
> >>> runs have left a horked -ROOT- region on the filesystem.
> >>>
> >>> Good on you Holger,
> >>> St.Ack
> >>>
> >>>>>
> >>>>> Thanks,
> >>>>> St.Ack
> >>>>>
> >>>>>
> >>>>>>>
> >>>>>>> Did you just change the value of hadoop.tmp.dir from
> >>>>>>> /tmp/hadoop-${user.name} to /tmp/hadoop-holste on
> cygwin and it
> >>>>>>> then worked?
> >>>>>> No. I changed it to changed the value "c:\tmp\hadoop-holste".
> >>>>>>>
> >>>>>>>
> >>>>>>>> Also, I tried to do the very same (as described in
> my intitial
> >>>>>>>> mail) on Ubuntu also *without* specifying "hadoop.tmp.dir".
> >>>>>>>> There it works wihtout problems...
> >>>>>>>
> >>>>>>> Yeah.  I know things work out-of-the-box on
> linux/macosx.  Jim
> >>>>>>> has been doing a bunch of work trying to get the same
> to happen
> >>>>>>> on cygwin (10Minutes has the latest fruit of his work).
> >>>>>>>>
> >>>>>>>> As requested by Michel Stack I also attach my log (turned
to
> >>>>>>>> debug) when starting the server.
> >>>>>>>
> >>>>>>> Thanks for the log.  Comments interlaced.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Holger
> >>>>>>>>
> >>>>>>>> 07/11/02 13:40:09 DEBUG conf.Configuration:
> >>>>>>>> java.io.IOException: config()
> >>>>>>>>   at
> >>>>>>>>
> org.apache.hadoop.conf.Configuration.<init>(Configuration.java:
> >>>>>>>> 144)
> >>>>>>>>
> >>>>>>>>   at
> >>>>>>>>
> org.apache.hadoop.hbase.HBaseConfiguration.<init>(HBaseConfigur
> >>>>>>>> ation.java:30)
> >>>>>>>>
> >>>>>>>>   at
> org.apache.hadoop.hbase.HMaster.doMain(HMaster.java:3043)
> >>>>>>>>   at org.apache.hadoop.hbase.HMaster.main(HMaster.java:3097)
> >>>>>>
> >>>>>> ...
> >>>>>
> >>>
> >
>

Mime
View raw message