hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sanjay Subramanian <Sanjay.Subraman...@wizecommerce.com>
Subject EBADF: Bad file descriptor
Date Wed, 10 Jul 2013 15:19:22 GMT
2013-07-10 07:11:50,131 WARN [Readahead Thread #1] org.apache.hadoop.io.ReadaheadPool: Failed
readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO.posix_fadvise(Native Method)
at org.apache.hadoop.io.nativeio.NativeIO.posixFadviseIfPossible(NativeIO.java:145)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:205)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Hi

I have a Oozie workflow that runs a MR job and I have started getting this error past two
days in one of the MR jobs that is being processed.
However if I run it again , it succeeds :-(  but about 1 hr is wasted in the process.

Any clues ?

Or should I post this issue in the Oozie postings ?

Thanks

sanjay

Configuration
Name    Value
impression.log.record.cached.tag        cached=
impression.log.record.end.tag   [end
impressions.mapreduce.conf.file.full.path       /workflows/impressions/config/aggregations.conf<http://thv01:8888/filebrowser/view/workflows/impressions/config/aggregations.conf>
mapred.job.queue.name   default
mapred.mapper.new-api   true
mapred.reducer.new-api  true
mapreduce.input.fileinputformat.inputdir        /data/input/impressionlogs/outpdirlogs/9999-99-99<http://thv01:8888/filebrowser/view/data/input/impressionlogs/outpdirlogs/9999-99-99>
mapreduce.job.inputformat.class com.wizecommerce.utils.mapred.ZipMultipleLineRecordInputFormat
mapreduce.job.map.class com.wizecommerce.parser.mapred.OutpdirImpressionLogMapper
mapreduce.job.maps      500
mapreduce.job.name      OutpdirImpressions_0000475-130611151004460-oozie-oozi-W
mapreduce.job.output.value.class        org.apache.hadoop.io.Text
mapreduce.job.outputformat.class        com.wizecommerce.utils.mapred.NextagTextOutputFormat
mapreduce.job.reduce.class      com.wizecommerce.parser.mapred.OutpdirImpressionLogReducer
mapreduce.job.reduces   8
mapreduce.map.output.compress   true
mapreduce.map.output.compress.codec     org.apache.hadoop.io.compress.SnappyCodec
mapreduce.map.output.key.class  org.apache.hadoop.io.Textorg.apache.hadoop.io.Text
mapreduce.map.output.value.class        com.wizecommerce.parser.dao.OutpdirLogRecord
mapreduce.output.fileoutputformat.compress      true
mapreduce.output.fileoutputformat.compress.codec        com.hadoop.compression.lzo.LzopCodec
mapreduce.output.fileoutputformat.outputdir     /data/output/impressions/outpdir/9999-99-99/0000475-130611151004460-oozie-oozi-W/outpdir_impressions_ptitle<http://thv01:8888/filebrowser/view/data/output/impressions/outpdir/9999-99-99/0000475-130611151004460-oozie-oozi-W/outpdir_impressions_ptitle>
mapreduce.tasktracker.map.tasks.maximum 12
mapreduce.tasktracker.reduce.tasks.maximum      8
outpdir.log.exclude.processing.datatypes        header,sellerhidden




CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s)
and may contain confidential and privileged information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient, please contact the sender
by reply email and destroy all copies of the original message along with any attachments,
from your computer system. If you are the intended recipient, please be advised that the content
of this message is subject to access, review and disclosure by the sender's Email System Administrator.

Mime
View raw message