accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-118) accumulo could work across HDFS instances, which would help it to scale past a single namenode
Date Fri, 31 Jan 2014 17:48:18 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13887940#comment-13887940
] 

Keith Turner commented on ACCUMULO-118:
---------------------------------------

bq. This suite of tickets is really starting to concern me.

I am also concerned, I am worried about administrators having a really bad experience (bricked
Accumulo instance).   In hindsight I think this feature was merged in before it was complete
and should not have been merged in.  What can we learn from this to make development go more
smoothly in the future?

Personally I think if a new feature is incomplete (in terms of test, usability, documentation,
etc), breaks other features of Accumulo, or introduces new problems it should not be merged
in.  I did not realize all of the problems absolute paths could cause until well after it
was merged in.   In hindsight I think the design of this feature should have started with
administrative use cases.     Maybe that would have brought to attention the problems w/ absolute
paths much earlier.   

bq.  I would like to start the conversation about marking this item and items around it as
@Experimental

I wish it were that simple.  Wether you use multiple namenodes or not, Accumulo will use absolute
paths in 1.6 and administrators do not have a reliable way to manage those paths.  I have
a fix for ACCUMULO-1832 in the works, I should be done w/ that very soon.  I think that fix
will address my major concern w/ this issue.

> accumulo could work across HDFS instances, which would help it to scale past a single
namenode
> ----------------------------------------------------------------------------------------------
>
>                 Key: ACCUMULO-118
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-118
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: master, tserver
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>            Priority: Blocker
>             Fix For: 1.6.0
>
>         Attachments: ACCUMULO-118-01.txt, ACCUMULO-118-02.txt
>
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> Consider using full path names to files, which would allow the servers to access the
files on any HDFS file system.
> Work may exist elsewhere to run HDFS using a number of NameNode instances to break up
the namespace.
> We may need a pluggable strategy to determine namespace for new files.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message