Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@accumulo.apache.org
Received-SPF: pass (nike.apache.org: domain of josh.elser@gmail.com designates
 209.85.220.176 as permitted sender)
Message-ID: <527927CA.7030500@gmail.com>
Date: Tue, 05 Nov 2013 12:15:54 -0500
From: Josh Elser <josh.elser@gmail.com>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
 rv:24.0) Gecko/20100101 Thunderbird/24.1.0
MIME-Version: 1.0
To: dev@accumulo.apache.org
Subject: Re: MultipleInputs with AccumuloInputFormat
References: <52790DC1.1030708@gmail.com>
 <CAEA3U3SvMJ_cvzAfJLObfZ4bt=Q-xT85vg3KEbrHrArdJVhFZw@mail.gmail.com>
In-Reply-To: 
 <CAEA3U3SvMJ_cvzAfJLObfZ4bt=Q-xT85vg3KEbrHrArdJVhFZw@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Heh, ok.

I'm currently working through a bit of a prototype to see how it works.

I'm not a mapred/mapreduce expert, but I *think* I have an approach that 
will work. Keep an eye out for a Jira -- would love feedback.

On 11/5/13, 12:13 PM, Kevin Faro wrote:
> I recently looked into that and came to the same realization.
>
> I ended up writing a new input format that did the cartesian product of two
> tables.  But to do that I had to store values for the left configuration
> and right configuration and then copy over whichever config settings I
> wanted to use for the AIF depending on which split i needed in the
> RecordReader.
>
> It would have been awesome if I could have just used the MultipleInputs ...
>
> --Kevin
>
>
> On Tue, Nov 5, 2013 at 10:24 AM, Josh Elser <josh.elser@gmail.com> wrote:
>
>> In executing some MapReduce over Accumulo with the AccumuloInputFormat, I
>> came to the realization that AIF fundamentally doesn't work with concepts
>> like MultipleInputs in Hadoop (http://hadoop.apache.org/
>> docs/current/api/org/apache/hadoop/mapreduce/lib/input/MultipleInputs.html).
>> Given that you can only write one set of configuration for AIF into a
>> Configuration object, there's not a mechanism to support multiple. This
>> appears to be the case across all versions.
>>
>> Is this correct? Have I overlooked something?
>>
>