hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HDFS-2583) Hadoop mapreduce job to process S3 logs gets hung at INFO mapred.JobClient: map 0% reduce 0%.
Date Tue, 22 Nov 2011 04:44:42 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J resolved HDFS-2583.
---------------------------

    Resolution: Invalid

Hello Nitika,

Please direct your question to mapreduce-user@hadoop.apache.org mailing lists. The JIRA is
meant to be only for project discussion and development, not a user support channel.

I'm sure a lot of folks (including me) would be happy to answer your question on the mailing
list instead. Also helps if you post logs and error stacktraces you've received instead of
just code.

Resolving as Invalid.
                
> Hadoop mapreduce job to process S3 logs gets hung at INFO mapred.JobClient:  map 0% reduce
0%.
> ----------------------------------------------------------------------------------------------
>
>                 Key: HDFS-2583
>                 URL: https://issues.apache.org/jira/browse/HDFS-2583
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Nitika Gupta
>            Priority: Blocker
>
> I am trying to run a mapreduce job to process the Amazon S3 logs. However, the code hangs
at INFO mapred.JobClient:  map 0% reduce 0% and does not even attempt to launch the tasks.
The sample code for the job setup is given below:
> public int run(CommandLine cl) throws Exception 
> {
>        Configuration conf = getConf();
>        String inputPath = "";
>        String outputPath = "";
>        try
>        {
>            Job job = new Job(conf, "Dummy");
>            job.setNumReduceTasks(0);
>            job.setMapperClass(Mapper.class);
>            inputPath = cl.getOptionValue("input"); //input is an s3n path
>            outputPath = cl.getOptionValue("output");
>            FileInputFormat.setInputPaths(job, inputPath);
>            FileOutputFormat.setOutputPath(job, new Path(outputPath));
>            _log.info("Input path set as " + inputPath);
>            _log.info("Output path set as " + outputPath);
>            job.waitForCompletion(true);
>            return 0;
>        } catch (Exception ex)
>        {
>            _log.error(ex);
>            return 1;
>        }
> }
> The above code works on the staging machine. However, it fails on the production machine
which is same as the staging machine with more capacity.
> Does anyone know what could be the possible reason for the error? 
> Thanks in advance!
> Nitika

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message