hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From surfer <sur...@crs4.it>
Subject Re: Problem using distributed cache
Date Tue, 11 Dec 2012 10:08:29 GMT
On 12/07/2012 03:49 PM, surfer wrote:
> Hello Peter
> In my, humble, experience I never get hadoop 1.0.3 to work with
> distributed cache and the new api (mapreduce). with the old api it works.
> giovanni
>
> P.S. I already tried the approaches suggested by both Dhaval and  Harsh J
I'm writing this hoping that maybe could help someone else....

the missing piece was that I was doing as a test for the DC a datajoin
job and the parameter that overrides the default hadoop key/value
separator (\t) for the KeyValueFromTextInputFormat is changed in the new
api.
old api syntax:   conf.set("key.value.separator.in.input.line", ",");
new api syntax:
conf.set("mapreduce.input.keyvaluelinerecordreader.key.value.separator",
",");

about the addCacheFile method location. It works both by placing it
right after the conf creation (Dhaval approach):

|DistributedCache.addCacheFile(new URI("/user/peter/cacheFile/testCache1"), conf);
|


or after the the job creation (Harsh approach):

|DistributedCache.addCacheFile(new URI("/user/peter/cacheFile/testCache1"),job.getConfiguration());|


regards
giovanni

Mime
View raw message