hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hbase/HowToMigrate" by izaakrubin
Date Fri, 18 Jul 2008 22:56:37 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The following page has been changed by izaakrubin:

  As you will notice, the [http://wiki.apache.org/hadoop/Hadoop%20Upgrade Hadoop Upgrade Instructions]
(specifically steps 2-4, 16-18) ask you to generate several logs to compare and ensure that
the upgrade ran correctly.  I did notice some inconsistency in my logs between ''dfs-v-old-report-1.log''
and ''dfs-v-new-report-1.log''; specifically the ''Total effective bytes'' and ''Effective
replication multiplier'' fields did not match (in the new log, the values reported were zero
and infinity, respectively).  Additionally, ''dfs-v-new-report-1.log'' claimed that the update
was not finalized.  Running {{{{$HADOOP_HOME}/bin/hadoop dfsadmin -finalizeUpgrade}}} resolves
the second issue, finalizing the upgrade as expected.  I could not find a way to resolve the
inconsistencies with the ''Total effective bytes'' and ''Effective replication multiplier''
fields.  Nonetheless, I found no problems with the migration and the data appeared to be completely
+ == Other Migration-Related Concerns ==
+ === Redo Logs ===
+ It is possible that, when running the HBase migration command, the migration will fail because
of "unrecovered redo logs."  Redo logs are generated every time HBase is started, and under
normal circumstances they are removed when HBase is stopped.  However, if you have ever stopped
HBase in some atypical way (for example, using {{{kill -9}}}), these redo logs will persist
in Hadoop DFS.  To see if you have any unrecovered redo logs, stop any currently-running instances
of HBase and enter: {{{{$HADOOP_HOME}/bin/hadoop dfs -ls /hbase}}}.  All existing redo logs
will be in this directory.  Redo log directories can be removed using the dfs {{{-rmr}}} option.
 WARNING: redo logs are the only way to recover any data entered before HBase was improperly
stopped.  Removing redo logs with file size greater than zero may result in irreversible data

View raw message