hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marko Dinic <marko.di...@nissatech.com>
Subject Re: Reading a sequence file from distributed cache
Date Tue, 12 May 2015 08:03:07 GMT
Hello Shahab,

I'm using 1.2.1 in pseudo-distributed mode and the same code on a 
cluster with 0.20.2, but I'm having same problem in both cases. I'm 
hopping that 1.2.1 code is back-compatible with 0.20.2 cluster?

Do you have any idea what could be the problem? And what do you mean by 
- Have you seen this?

Maybe I'm making some mistake by using context passed to Mapper to read 
the file?

Configuration conf = context.getConfiguration();

Best regards,
Marko

On Tue 12 May 2015 12:09:52 AM CEST, Shahab Yunus wrote:
> What version are you using?
>
> Have you seen this?
>
> Regards,
> Shahab
>
> On Mon, May 11, 2015 at 5:25 PM, <marko.dinic@nissatech.com
> <mailto:marko.dinic@nissatech.com>> wrote:
>
>     Hello,
>
>     I'm new to Hadoop and I'm having a problem reading from a sequence
>     file that I add to distributed cache.
>
>     I didn't have problems when I ran it in standalone mode, but now
>     in pseudo-distributed and distributed I do.
>
>     I'm adding file to distributed cache like this
>
>     |DistributedCache.addCacheFile(new URI(currentMedoids), conf);|
>
>     And reading from it in mapper's setup method
>
>     |         Configuration conf = context.getConfiguration();
>              FileSystem fs = FileSystem.get(conf);
>
>              Path[] paths = DistributedCache.getLocalCacheFiles(conf);
>
>              List<Element> sketch = new ArrayList<Element>();
>
>              SequenceFile.Reader medoidsReader = new SequenceFile.Reader(fs, paths[0],
conf);
>
>              Writable medoidKey = (Writable) medoidsReader.getKeyClass().newInstance();
>              Writable medoidValue = (Writable) medoidsReader.getValueClass().newInstance();
>
>              while(medoidsReader.next(medoidKey, medoidValue)){
>
>                  ElementWritable medoidWritable = (ElementWritable)medoidValue;
>                  sketch.add(medoidWritable.getElement());
>              }|
>
>     And I'm getting FileNotFoundException.
>
>     Can anyone please help me and explain to me what is the problem
>     and how to do this properly?
>
>     Thanks
>
>     Sent with inky <http://inky.com?kme=signature>
>
>

Mime
View raw message