hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Stack <st...@duboce.net>
Subject Re: NoSuchElementException when creating a table
Date Wed, 07 Nov 2007 17:38:21 GMT
Great.  Thanks.  I added it to the wiki.
St.Ack

Holger Stenzhorn wrote:
> Ok, here is my first shot at it:
>
> --- SNIP ---
>
> 3.1. Set the hadoop tmp directory property, hadoop.tmp.dir (For 
> Windows/Cygwin users: If you do not include this variable in the site 
> file then the value /tmp/hadoop-${user.name} from hadoop-default.xml 
> is used. But since this does not work properly for Windows/Cygwin you 
> have to specify a Windows-formatted path for hadoop.tmp.dir such as 
> c:\tmp\hbase)
>
> --- SNIP ---
>
> Cheers,
> Holger
>
> Michael Stack wrote:
>> Holger Stenzhorn wrote:
>>> Perhaps you could add to the "10 minutes" page at point 3.1 an 
>>> example for Unix and Windows/Cygwin to make the different path 
>>> handling a bit clearer.
>>>
>> Sounds like a good idea to me Holger.  Would you mind taking a first 
>> cut at it since you've had first hand-experience of where the doc. is 
>> lacking?
>>
>> St.Ack
>>
>>> Cheers,
>>> Holger
>>>
>>> Michael Stack wrote:
>>>> Holger Stenzhorn wrote:
>>>>>>>> Hadoop RunJar looks to contain the only use of hadoop.tmp.dir

>>>>>>>> other than test and contrib code.  It does the following:
'new 
>>>>>>>> File(new Configuration().get("hadoop.tmp.dir"));' using 
>>>>>>>> java.io.File.  In hbase, its all Path and hadoop FileSystem.

>>>>>>>> So, yes, content of hadoop.tmp.dir is treated differently.
>>>>>>> Could that be changed? ...meaning, wouldn't it be sensible to

>>>>>>> change HBase (to use Path etc.) so it behaves like Hadoop?
>>>>>>
>>>>>> You mean, can hadoop be changed?  (Its hbase that's drunk the 
>>>>>> kool-aid; its the one that goes via hadoop FileSystem for all 
>>>>>> filesystem accesses whereas the hadoop use of hadoop.tmp.dir goes

>>>>>> via java.io.File).
>>>>> Nope. I mean the other way around: Can you change the HBase code 
>>>>> so it goes via the Hadoop filesystem? ...or would this imply a 
>>>>> performance drop for HBase?
>>>> It already does (See my comments above).
>>>>
>>>>
>>>>>> What happens if you clean up your local directory removing 
>>>>>> ${hadoop.tmp.dir} and retry?
>>>>> Again, if I use "c:\tmp\hadoop-holste" as value for 
>>>>> "${hadoop.tmp.dir}" it works: The log reflects just the situation 
>>>>> when this variable is not set in "hbase-site.xml" - then it does 
>>>>> not work.
>>>>> I also forgot to mention in my last my that it is not just 
>>>>> "${hadoop.tmp.dir}" set to "/tmp/hadoop-${user.name}" that causes 
>>>>> the problem but also if it is set to "/tmp/hadoop-holste".
>>>>
>>>> Did you try my suggestion of removing anything that might once have 
>>>> been a value for hadoop.tmp.dir?
>>>>
>>>> Testing here on cygwin using local filesystem and all defaults, it 
>>>> all just works fine.  I'm disturbed by the state you seem to be in 
>>>> where the .META. table is not being deployed AND there is no 
>>>> exception saying why.   The only think I can think is that previous 
>>>> runs have left a horked -ROOT- region on the filesystem.
>>>>
>>>> Good on you Holger,
>>>> St.Ack
>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> St.Ack
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>> Did you just change the value of hadoop.tmp.dir from 
>>>>>>>> /tmp/hadoop-${user.name} to /tmp/hadoop-holste on cygwin
and it 
>>>>>>>> then worked?
>>>>>>> No. I changed it to changed the value "c:\tmp\hadoop-holste".
>>>>>>>>
>>>>>>>>
>>>>>>>>> Also, I tried to do the very same (as described in my
intitial 
>>>>>>>>> mail) on Ubuntu also *without* specifying "hadoop.tmp.dir".
>>>>>>>>> There it works wihtout problems...
>>>>>>>>
>>>>>>>> Yeah.  I know things work out-of-the-box on linux/macosx.
 Jim 
>>>>>>>> has been doing a bunch of work trying to get the same to
happen 
>>>>>>>> on cygwin (10Minutes has the latest fruit of his work).
>>>>>>>>>
>>>>>>>>> As requested by Michel Stack I also attach my log (turned
to 
>>>>>>>>> debug) when starting the server.
>>>>>>>>
>>>>>>>> Thanks for the log.  Comments interlaced.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Holger
>>>>>>>>>
>>>>>>>>> 07/11/02 13:40:09 DEBUG conf.Configuration: 
>>>>>>>>> java.io.IOException: config()
>>>>>>>>>   at 
>>>>>>>>> org.apache.hadoop.conf.Configuration.<init>(Configuration.java:144)

>>>>>>>>>
>>>>>>>>>   at 
>>>>>>>>> org.apache.hadoop.hbase.HBaseConfiguration.<init>(HBaseConfiguration.java:30)

>>>>>>>>>
>>>>>>>>>   at org.apache.hadoop.hbase.HMaster.doMain(HMaster.java:3043)
>>>>>>>>>   at org.apache.hadoop.hbase.HMaster.main(HMaster.java:3097)
>>>>>>>
>>>>>>> ...
>>>>>>
>>>>
>>


Mime
View raw message