accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-722) Accumulo using Accumulo as its own NameNode
Date Thu, 04 Oct 2012 02:05:07 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469083#comment-13469083
] 

Eric Newton commented on ACCUMULO-722:
--------------------------------------

David, I browsed the CFS site.  The first question on that page asks about the tricky part
where there's only one writer to a file.  This is a key problem, since having a single writer
to a write-ahead-log is key for proper failure conditions.  In particular, we must ensure
that the writes by a failing tablet server to its write-ahead log are denied while we use
that file for recovery.  The response "HDFS does not implement posix semantics" is true, but
it understates the importance of this feature to the HBase and/or Accumulo WAL.  And, it appears
this is not an open-source solution, so I'm unable to test it without committing additional
resources.  Does anyone know if they support exclusive writer semantics?

                
> Accumulo using Accumulo as its own NameNode
> -------------------------------------------
>
>                 Key: ACCUMULO-722
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-722
>             Project: Accumulo
>          Issue Type: Bug
>            Reporter: Eric Newton
>            Assignee: Eric Newton
>
> On large clusters, the NameNode can become a performance bottleneck.  The NameNode is
also a single-point of failure.  Recent improvements to HDFS to support High Availability
and Federation [See ACCUMULO-118] help address these issues, but at greater administrative
costs and specialized hardware.
> We have seen demonstrations of using HBase to host a NameNode.  There's Aaron Cordova's
example of a Distributed Name Node:
> [Design for a Distributed Name Node |http://www.slideshare.net/acordova00/design-for-a-distributed-name-node
]
> And giraffa:
> [Dynamic Namespace Partitioning with Giraffa File System |http://www.slideshare.net/Hadoop_Summit/dynamic-namespace-partitioning-with-giraffa-file-system]
> We could incrementally implement a self-hosted Accumulo, which would run as its own NameNode.
 This would be useful for large Accumulo installations.  Over the long term, we could incorporate
all NameNode functions to provide a scalable, distributed NameNode for other large Hadoop
installations.
> Hopefully the approach used could be trivially ported to HBase as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message