hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1048) HLog: Found 0 logs to remove out of total 1450; oldest outstanding seqnum is 162297053 fr om region -ROOT-,,0
Date Sun, 07 Dec 2008 07:17:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654178#action_12654178
] 

stack commented on HBASE-1048:
------------------------------

Testing I see logs like the following:

{code}
2008-12-07 06:45:05,736 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Too many logs: logs=41, maxlogs=40; forcing flush of region with oldest edits: -ROOT-,,0
2008-12-07 06:45:29,346 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Too many logs: logs=42, maxlogs=40; forcing flush of region with oldest edits: -ROOT-,,0
2008-12-07 06:45:46,294 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Too many logs: logs=43, maxlogs=40; forcing flush of region with oldest edits: .META.,,1
{code}

If I just extract the HLog edits, then I see:

{code}
2008-12-07 06:44:39,945 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Found 0 logs to remove  out of total 38; oldest outstanding seqnum is 2 from region -ROOT-,,0
2008-12-07 06:44:45,892 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Closed hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632271713,
entries=100008. New log writer: /hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632285857
2008-12-07 06:44:45,892 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Found 0 logs to remove  out of total 39; oldest outstanding seqnum is 2 from region -ROOT-,,0
2008-12-07 06:44:52,674 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Closed hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632279929,
entries=112307. New log writer: /hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632292660
2008-12-07 06:44:52,674 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Found 0 logs to remove  out of total 40; oldest outstanding seqnum is 2 from region -ROOT-,,0
2008-12-07 06:45:05,736 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Closed hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632285857,
entries=180201. New log writer: /hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632305721
2008-12-07 06:45:05,736 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Found 0 logs to remove  out of total 41; oldest outstanding seqnum is 2 from region -ROOT-,,0
2008-12-07 06:45:05,736 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Too many logs: logs=41, maxlogs=40; forcing flush of region with oldest edits: -ROOT-,,0
2008-12-07 06:45:29,346 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Closed hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632292660,
entries=249390. New log writer: /hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632329343
2008-12-07 06:45:29,346 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Found 0 logs to remove  out of total 42; oldest outstanding seqnum is 2 from region -ROOT-,,0
2008-12-07 06:45:29,346 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Too many logs: logs=42, maxlogs=40; forcing flush of region with oldest edits: -ROOT-,,0
2008-12-07 06:45:46,294 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Closed hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632305721,
entries=132451. New log writer: /hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632346266
2008-12-07 06:45:46,294 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Found 0 logs to remove  out of total 43; oldest outstanding seqnum is 4 from region .META.,,1
2008-12-07 06:45:46,294 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Too many logs: logs=43, maxlogs=40; forcing flush of region with oldest edits: .META.,,1
2008-12-07 06:45:55,785 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
Closed hdfs://aa0-000-12.u.powerset.com:9000/hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632329343,
entries=155862. New log writer: /hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.1228632355780
2008-12-07 06:45:55,785 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] DEBUG org.apache.hadoop.hbase.regionserver.HLog:
Found 42 logs to remove  out of total 44; oldest outstanding seqnum is 6335280 from region
TestTable,0006486018,1228632049567
2008-12-07 06:45:55,785 [regionserver/0:0:0:0:0:0:0:0:60020.logRoller] INFO org.apache.hadoop.hbase.regionserver.HLog:
removing old log file /hbasetrunk2/log_208.76.44.140_1228631862878_60020/hlog.dat.0 whose
highest sequence/edit id is 114842
{code}

Notice how the flushing of the root and meta allowed us clean up 42 of 44 logs.

Committing because it seems to work.  Will make default 100 though instead of 40 logs so we
flush meta and root less often.

> HLog: Found 0 logs to remove out of total 1450; oldest outstanding seqnum is 162297053
fr om region -ROOT-,,0
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1048
>                 URL: https://issues.apache.org/jira/browse/HBASE-1048
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>             Fix For: 0.19.0
>
>         Attachments: 1048.patch
>
>
> Above was in tim sell log just before it OOME'd.  1450 log files are going to mess us
up.  Need to clear them.  Catalog tables should probably keep up the old optional flush or,
if N commit logs, schedule a flush on the region with the oldest sequent number.
> Bringing into 0.19.0.    Gets in the way of our scaling.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message