accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ACCUMULO-2793) Clean up handling of moving HDFS under Accumulo from non-HA to HA
Date Sat, 10 May 2014 22:04:24 GMT

     [ https://issues.apache.org/jira/browse/ACCUMULO-2793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Busbey updated ACCUMULO-2793:
----------------------------------

    Summary: Clean up handling of moving HDFS under Accumulo from non-HA to HA  (was: 1.6+
Manual missing instructions on HDFS URL rewriting util)

Work in progress steps to change from non-HA to HA:


assumptions, based on starting with a clean 1.6.0 install on top of non-HA HDFS:

* original HDFS FS URL is hdfs://namenode.example.com:8020
* new HA nameservice URL is hdfs://nameservice1
* you didn't have instance.volumes configured
* you didn't change the legacy instance dir stuff from /accumulo

1) use ```$ACCUMULO_HOME/bin/accumulo admin volumes``` to confirm that only the expected volume
is present
{noformat}
Listing volumes referenced in zookeeper
        Volume : hdfs://namenode.example.com:8020/accumulo

Listing volumes referenced in accumulo.root tablets section
        Volume : hdfs://namenode.example..com:8020/accumulo
Listing volumes referenced in accumulo.root deletes section (volume replacement occurrs at
deletion time)

Listing volumes referenced in accumulo.metadata tablets section
        Volume : hdfs://namenode.example..com:8020/accumulo

Listing volumes referenced in accumulo.metadata deletes section (volume replacement occurrs
at deletion time)
{noformat}
2) shut down the Accumulo cluster
3) Transition to HDFS HA with given nameservice above
4) Edit $ACCUMULO_CONF_DIR/accumulo-site.xml
{code:xml}
    <property>
      <name>instance.volumes</name>
      <value>hdfs://namenode.example.com:8020/accumulo,hdfs://nameservice1/accumulo</value>
    </property>
    <property>
      <name>instance.volumes.replacements</name>
      <value>hdfs://namenode.example.com:8020/accumulo hdfs://nameservice1/accumulo</value>
    </property>
{code}
5) run $ACCUMULO_HOME/bin/accumulo init --add-volumes
6) start the Accumulo cluster
7) Verify that the new volume now shows up
{noformat}
Listing volumes referenced in zookeeper
        Volume : hdfs://namenode.example.com:8020/accumulo
        Volume : hdfs://nameservice1/accumulo

Listing volumes referenced in accumulo.root tablets section
        Volume : hdfs://namenode.example..com:8020/accumulo
        Volume : hdfs://nameservice1/accumulo
Listing volumes referenced in accumulo.root deletes section (volume replacement occurrs at
deletion time)

Listing volumes referenced in accumulo.metadata tablets section
        Volume : hdfs://namenode.example..com:8020/accumulo
        Volume : hdfs://nameservice1/accumulo
Listing volumes referenced in accumulo.metadata deletes section (volume replacement occurrs
at deletion time)
{noformat}
8) You may still see some erroneous GC failure messages as things transition.

I can successfully ingest data both before and after failover using these steps

Other things to clean up
* The manual instructions around instance.volumes incorrectly lists the command as ```accumulo
init --add-volume```
* The init --add-volumes doesn't say that it has done anything
* I get errors from the GC and when trying to start the shell after failover, looks like they're
trying to use the defunct Volume
* How do I remove the defunct Volume so that my configuration only refers to the HA nameservice?

> Clean up handling of moving HDFS under Accumulo from non-HA to HA
> -----------------------------------------------------------------
>
>                 Key: ACCUMULO-2793
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2793
>             Project: Accumulo
>          Issue Type: Bug
>          Components: docs
>    Affects Versions: 1.6.0
>            Reporter: Sean Busbey
>            Priority: Blocker
>             Fix For: 1.6.1, 1.7.0
>
>
> We added a util to handle cluster maintenance (such as going from non-HA HDFS to HA HDFS)
in ACCUMULO-1832.
> It should be added to section 13.5 Tools in the user manual, and a reference to that
section should be added in the section 12 explanation of the impact of supporting multi-volume
setups.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message