Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of harsh@cloudera.com designates
 209.85.223.178 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAC+-qb9goe9kFq+XEBCeQ9KtgQVu5h22T0=W4PK6EDn+bknkEQ@mail.gmail.com>
References: 
 <CAC+-qb9goe9kFq+XEBCeQ9KtgQVu5h22T0=W4PK6EDn+bknkEQ@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Wed, 9 Apr 2014 00:00:57 +0530
Message-ID: 
 <CAOcnVr0vJQ8CavBdiib1=urACUJihVeNUT-OSeS4iyqRs5obZg@mail.gmail.com>
Subject: Re: MapReduce for complex key/value pairs?
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Content-Type: text/plain; charset=ISO-8859-1

Yes, you can write custom writable classes that detail and serialise
your required data structure. If you have Hadoop: The Definitive
Guide, checkout its section "Serialization" under chapter "Hadoop
I/O".

On Tue, Apr 8, 2014 at 9:16 PM, Natalia Connolly
<natalia.v.connolly@gmail.com> wrote:
> Dear All,
>
>     I was wondering if the following is possible using MapReduce.
>
>     I would like to create a job that loops over a bunch of documents,
> tokenizes them into ngrams, and stores the ngrams and not only the counts of
> ngrams but also _which_ document(s) had this particular ngram.  In other
> words, the key would be the ngram but the value would be an integer (the
> count) _and_ an array of document id's.
>
>     Is this something that can be done?  Any pointers would be appreciated.
>
>     I am using Java, btw.
>
>    Thank you,
>
>    Natalia Connolly
>


-- 
Harsh J