accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1732) Resolve table name to table id once in Accumulo input format
Date Sat, 05 Oct 2013 02:29:47 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13786883#comment-13786883
] 

ASF subversion and git services commented on ACCUMULO-1732:
-----------------------------------------------------------

Commit b96701f220ecb3e891a71741179b867429fa1d39 in branch refs/heads/master from [~sonixbp]
[ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=b96701f ]

Squashed commit of the following:

commit 3227a822379718d6c1297f11d7af37a716f78a60
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Tue Oct 1 23:20:34 2013 -0400

    Adding the following:
    - Deprecation to InputConfigurator, mapred.InputFormatBase, mapreduce.InputFormatBase
    - Comments to TableQueryConfig
    - Multi-table support to mapred.InputFormatBase

    ACCUMULO-391

commit 6648e8a1c97939f740b24f9368ecda9f7072cbd2
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Tue Oct 1 21:45:37 2013 -0400

    Fixing some more formatting. Adding license headers. ACCUMULO-391

commit 53bcc85689510fc988c9e9f6aff0da0cb7091c6c
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Mon Sep 30 21:01:55 2013 -0400

    Cleaning up tests. Adding test for legacy input for base + new multi-table methods. ACCUMULO-391

commit e4e05c804ea7f486290181f0246cf6b2880f5d1a
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Sun Sep 29 21:05:55 2013 -0400

    Fixing some formatting. Adding some comments. ACCUMULO-391

commit 10b4eb8206ab4395ef2d4df375b52a7ffe77d655
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Sun Sep 29 20:37:07 2013 -0400

    ACCUMULO-1732 Using table id in RangeInputSplit so that it can be resolved back to "working"
table name in mappers. Scanner uses the "working" table name while everything else can still
safely use the original configured table name.

commit 7b8585f0333c09674f7612b4dc24887f684413fe
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Sat Sep 28 23:23:48 2013 -0400

    Removing deprecation for now until we have some discussions. Updating/adding comments.
ACCUMULO-391

commit 273ee49530de28c2c5dfe39c80ab0c90c3c3a95f
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Sat Sep 28 23:01:04 2013 -0400

    The legacy mapred InputFormatBase now verifies (and fixes the scanner for) a possible
change in table name that could happen between the configuration of the map/reduce job and
the actual processing of the scanner for a specific split. In that case, the most recent table
name associated with the id is always used for the scanner (though the table name that was
expected during job setup is still used in the RangeInputSplit). ACCUMULO-391

commit e6a7c962f707487d832ba4b16c1f9066d13ff8f1
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Sat Sep 28 22:53:42 2013 -0400

    The original single-table setters/getters now populate a "default" TableQueryConfig object
under the hood. This should make the switch over much easier. Deprecated single table methods
in light of the API changes for the new configuration object. ACCUMULO-391

commit fdf4cadb16c29fc03a610cf83399ee26d7f83bc9
Author: Corey J. Nolet <cjnolet@gmail.com>
Date:   Sat Sep 28 21:58:40 2013 -0400

    Adding new TableQueryConfig object for setting multiple table info in the InputFormatBase


> Resolve table name to table id once in Accumulo input format
> ------------------------------------------------------------
>
>                 Key: ACCUMULO-1732
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1732
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Keith Turner
>            Assignee: Corey J. Nolet
>            Priority: Minor
>
> AccumuloInputFormat (and I suspect AccumuloOutputFormat) sends the table name to each
mapper.  The mapper uses this table name to create a scanner.  In the case of the following
events a map reduce job could read from two different table ids.   
>  # start M/R job reading table A
>  # rename table A (tableId=1) to table C
>  # rename table B (tableId=2) to table A
> If the input format passed table id 1 to the mappers, then the renames would not cause
a problem.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message