Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of sigurd.spieckermann@gmail.com
 designates 209.85.160.48 as permitted sender)
MIME-Version: 1.0
Date: Tue, 25 Sep 2012 10:32:50 +0200
Message-ID: 
 <CABVuHiix67S=i8T=uMjcNWhPbjXXdUZkwVghu0GPYjaJgUQFWQ@mail.gmail.com>
Subject: Join-package combiner number of input and output records the same
From: Sigurd Spieckermann <sigurd.spieckermann@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=047d7b15a80531272404ca828deb

--047d7b15a80531272404ca828deb
Content-Type: text/plain; charset=ISO-8859-1

Hi guys,

I'm experiencing a strange behavior when I use the Hadoop join-package.
After running a job the result statistics show that my combiner has an
input of 100 records and an output of 100 records. From the task I'm
running and the way it's implemented, I know that each key appears multiple
times and the values should be combinable before getting passed to the
reducer. I'm running my tests in pseudo-distributed mode with one or two
map tasks. From using the debugger, I noticed that each key-value pair is
processed by a combiner individually so there's actually no list passed
into the combiner that it could aggregate. Can anyone think of a reason
that causes this undesired behavior?

Thanks
Sigurd

--047d7b15a80531272404ca828deb
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi guys,<br><br>I&#39;m experiencing a strange behavior when I use the Hado=
op join-package. After running a job the result statistics show that my com=
biner has an input of 100 records and an output of 100 records. From the ta=
sk I&#39;m running and the way it&#39;s implemented, I know that each key a=
ppears multiple times and the values should be combinable before getting pa=
ssed to the reducer. I&#39;m running my tests in pseudo-distributed mode wi=
th one or two map tasks. From using the debugger, I noticed that each key-v=
alue pair is processed by a combiner individually so there&#39;s actually n=
o list passed into the combiner that it could aggregate. Can anyone think o=
f a reason that causes this undesired behavior?<br>
<br>Thanks<br>Sigurd<br>

--047d7b15a80531272404ca828deb--