Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B04757FAC for ; Mon, 12 Sep 2011 14:04:49 +0000 (UTC) Received: (qmail 86197 invoked by uid 500); 12 Sep 2011 14:01:14 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 85888 invoked by uid 500); 12 Sep 2011 14:00:50 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 85826 invoked by uid 99); 12 Sep 2011 14:00:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Sep 2011 14:00:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of tucu@cloudera.com designates 209.85.216.176 as permitted sender) Received: from [209.85.216.176] (HELO mail-qy0-f176.google.com) (209.85.216.176) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Sep 2011 14:00:38 +0000 Received: by qyk36 with SMTP id 36so144516qyk.14 for ; Mon, 12 Sep 2011 07:00:17 -0700 (PDT) Received: by 10.68.62.232 with SMTP id b8mr286249pbs.523.1315836017191; Mon, 12 Sep 2011 07:00:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.68.54.228 with HTTP; Mon, 12 Sep 2011 06:59:47 -0700 (PDT) In-Reply-To: References: From: Alejandro Abdelnur Date: Mon, 12 Sep 2011 06:59:47 -0700 Message-ID: Subject: Re: Stop chained mapreduce. To: mapreduce-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=bcaec539618063295204acbef2a4 --bcaec539618063295204acbef2a4 Content-Type: text/plain; charset=ISO-8859-1 Ilyal, The MR output files names follow the pattern part-#### and you'll have as many as reducers your job had. As you know the output directory, you could do a fs.listStatus() of the output directory and check all the part-* files. Hope this helps. Thanks. Alejandro On Sun, Sep 11, 2011 at 4:52 AM, ilyal levin wrote: > Hi > I created a chained mapreduce program where each job creates a SequenceFile > output. > My stopping condition is simply to check if the last output file (Type - > SequenceFile) is empty. > In order to do that i need to use the SequenceFile.Reader > and for him to read the data i need the path of the output file. The > problem is that i don't know the name of the file, > it usually depends on the number of the reducer. What can i do to solve > this? > > Thanks. > --bcaec539618063295204acbef2a4 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Ilyal,

The MR output files names follow the pattern part= -#### and you'll have as many as reducers your job had.

As you know the output directory, you could do a fs.listStatus() of the = output directory and check all the part-* files.

Hope this helps.

Thanks.

Alejandro

On S= un, Sep 11, 2011 at 4:52 AM, ilyal levin <nipponilyal@gmail.com> wrote:

Hi
I created a chained= mapreduce program where each job creates a SequenceFile output.=A0
My stopping condition is simply to check if the last=A0output file (Type - = SequenceFile) is empty.

In order to do that i need to use the=A0SequenceFile.Reader=A0
an= d for him to read the data i need the path of the output file. The problem = is that i don't know the name of the file,
it usually depends= on the number of the reducer. What can i do to solve this?

Thanks.=A0

--bcaec539618063295204acbef2a4--