accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (ACCUMULO-387) Support map reduce directly over files
Date Fri, 17 Feb 2012 23:14:57 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210658#comment-13210658
] 

Keith Turner edited comment on ACCUMULO-387 at 2/17/12 11:13 PM:
-----------------------------------------------------------------

This input format could run against offline tables.   It does not care if you clone or not,
but it will only start if the table is offline.  This is easy to achieve, just clone the table
and take it offline.  This is simpler than trying to adjust settings to disable compactions
and writes, setting that may change over time.

One draw back with this approach  is that the current code to take a table offline is async.
 It starts a table going offline, but does not wait for it to happen.  The inputformat could
probably get around this pretty easily.  It could check that the table states is offline and
then wait for there to be no locations in the metadata table.  Once there are no locations
it could start computing input splits.
                
      was (Author: kturner):
    This input format could run against offline tables.   It does not care if you clone or
not, but it will only start if the table is offline.  This is easy to achieve, just clone
the table and take it offline.  This is simpler than trying to adjust settings to disable
compactions and reads, setting that may change over time.

One draw back with this approach  is that the current code to take a table offline is async.
 It starts a table going offline, but does not wait for it to happen.  The inputformat could
probably get around this pretty easily.  It could check that the table states is offline and
then wait for there to be no locations in the metadata table.  Once there are no locations
it could start computing input splits.
                  
> Support map reduce directly over files
> --------------------------------------
>
>                 Key: ACCUMULO-387
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-387
>             Project: Accumulo
>          Issue Type: New Feature
>            Reporter: Keith Turner
>            Assignee: Eric Newton
>             Fix For: 1.4.1
>
>
> Support map reduce jobs that directly read Accumulo files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message