hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr Kozikowski <pi...@rapleaf.com>
Subject Re: java.io.IOException: Could not get block locations. Aborting...
Date Sat, 09 Aug 2008 00:41:50 GMT
Thank you for the reply. Apparently whatever it was is now gone after a
hadoop restart, but I'll keep that in mind should it happen again.

Piotr

On Fri, 2008-08-08 at 17:31 -0700, Dhruba Borthakur wrote:
> It is possible that your namenode is overloaded and is not able to
> respond to RPC requests from clients. Please check the namenode logs
> to see if you see lines of the form "discarding calls...".
> 
> dhrua
> 
> On Fri, Aug 8, 2008 at 3:41 AM, Alexander Aristov
> <alexander.aristov@gmail.com> wrote:
> > I come across the same issue and also with hadoop 0.17.1
> >
> > would be interesting if someone say the cause of the issue.
> >
> > Alex
> >
> > 2008/8/8 Steve Loughran <stevel@apache.org>
> >
> >> Piotr Kozikowski wrote:
> >>
> >>> Hi there:
> >>>
> >>> We would like to know what are the most likely causes of this sort of
> >>> error:
> >>>
> >>> Exception closing
> >>> file
> >>> /data1/hdfs/tmp/person_url_pipe_59984_3405334/_temporary/_task_200807311534_0055_m_000022_0/part-00022
> >>> java.io.IOException: Could not get block locations. Aborting...
> >>>        at
> >>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2080)
> >>>        at
> >>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1300(DFSClient.java:1702)
> >>>        at
> >>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1818)
> >>>
> >>> Our map-reduce job does not fail completely but over 50% of the map tasks
> >>> fail with this same error.
> >>> We recently migrated our cluster from 0.16.4 to 0.17.1, previously we
> >>> didn't have this problem using the same input data in a similar map-reduce
> >>> job
> >>>
> >>> Thank you,
> >>>
> >>> Piotr
> >>>
> >>>
> >> When I see this, its because the filesystem isnt completely up: there are
> >> no locations for a specific file, meaning the client isn't getting back the
> >> names of any datanodes holding the data from the name nodes.
> >>
> >> I've got a patch in JIRA that prints out the name of the file in question,
> >> as that could be useful.
> >>
> >
> >
> >
> > --
> > Best Regards
> > Alexander Aristov
> >


Mime
View raw message