accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ACCUMULO-1854) AccumuloInputFormat can be less strict about contents of Configuration
Date Mon, 25 Nov 2013 18:44:37 GMT

     [ https://issues.apache.org/jira/browse/ACCUMULO-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Elser updated ACCUMULO-1854:
---------------------------------

    Description: 
The AccumuloInputFormat required that the same exact Configuration object which was provided
to its static configuration methods is also provided to its getSplits() and createRecordReader()
methods.

In practice, some tools, like MultipleInputs or Pig, don't guarantee that the same Configuration
object is provided to getSplits() and createRecordReader(). This tends to be a common method
of interaction to ensure that multiple InputFormats don't collide with one another in the
same Configuration object (e.g. FileInputFormat). By serialization the necessary information
from the Configuration regarding how to query Accumulo into the InputSplit AccumuloInputFormat
creates, we can alleviate this requirement and make client interaction a little more standard.

  was:
I noticed that I was unable to properly use MultipleInputs (or any code which uses a similar
approach) with the AccumuloInputFormat class because of the way it builds up information in
the Configuration object.

It would be useful to be able to have multiple instances of AIF (and AOF) configured within
one Job (Configuration).


> AccumuloInputFormat can be less strict about contents of Configuration
> ----------------------------------------------------------------------
>
>                 Key: ACCUMULO-1854
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1854
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.4.4, 1.5.0
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 1.4.5, 1.5.1, 1.6.0
>
>
> The AccumuloInputFormat required that the same exact Configuration object which was provided
to its static configuration methods is also provided to its getSplits() and createRecordReader()
methods.
> In practice, some tools, like MultipleInputs or Pig, don't guarantee that the same Configuration
object is provided to getSplits() and createRecordReader(). This tends to be a common method
of interaction to ensure that multiple InputFormats don't collide with one another in the
same Configuration object (e.g. FileInputFormat). By serialization the necessary information
from the Configuration regarding how to query Accumulo into the InputSplit AccumuloInputFormat
creates, we can alleviate this requirement and make client interaction a little more standard.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message