hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anil gupta <anilgupt...@gmail.com>
Subject mapreduce.LoadIncrementalHFiles: Split occured while grouping HFiles
Date Fri, 26 May 2017 00:53:25 GMT
Cross posting since this seems to be an HBase issue.
I think completeBulkLoad step is failing. Please refer to the mail below.

---------- Forwarded message ----------
From: anil gupta <anilgupta84@gmail.com>
Date: Thu, May 25, 2017 at 4:38 PM
Subject: [IndexTool NOT working] mapreduce.LoadIncrementalHFiles: Split
occured while grouping HFiles
To: "user@phoenix.apache.org" <user@phoenix.apache.org>


Hi,

We are using HDP2.3.2(Phoenix 4.4 and HBase 1.1), we created a secondary
index on an already existing table. We paused all writes to Primary table.
Then we invoked IndexTool to populate secondary index table. We have tried
same steps many times but we keep on getting following error(we have also
tried drop the index and adding it again):

2017-05-24 18:00:10,281 WARN  [LoadIncrementalHFiles-2]
util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit is
deprecated by hbase.regionserver.global.memstore.size
2017-05-24 18:00:10,340 WARN  [LoadIncrementalHFiles-12]
util.HeapMemorySizeUtil: hbase.regionserver.global.memstore.upperLimit is
deprecated by hbase.regionserver.global.memstore.size
2017-05-24 18:00:10,342 INFO  [LoadIncrementalHFiles-11]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
c79ae6d27824424f99523dad586e86b1 first=JF2GPADC8GH331037\x00\
x80\x00\x1A0\x80\x00\x01Wj\x03r1defc4d301e4ec172b49be4a7ea33c2f7
last=JTHBK1GG4E2122477\x00\x80\x00$\xE4\x80\x00\x01[\xAD`{\
x17901d036d588292854ac5b1d4c29d8e1e
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-14]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
f0e97b218aed4abf9949cf49a57e559b first=5NPEB4AC3DH620091\x00\
x80\x00\xE0\x16\x80\x00\x01X\xE5g\xD6\x0B81d210ac753ed281e8627e5edb7eb59f
last=JF2GPADC8GH331037\x00\x80\x00\x1A0\x80\x00\x01W]&\
xE54f37d636104f6cd916b2b07bf3aa94d3f
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-2]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
27c0a905ee174c9898d324acf1554bf9 first=WMWZP3C58FT708786\x00\
x80\x00\xE0\x16\x80\x00\x01Y\xB8\x95U\xA0d21d32aed18af976dd53735705c728cd
last=`1GCRCPE05BZ430377\x00\x80\x00}\x05\x80\x00\x01[\xDEE\
x91L383768c6ac5f306fa99f68964b4f18aa
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-12]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
d0a6c4b727bb416f840ed254658f3982 first=1N4BZ0CP4GC308715\x00\
x80\x01T\xFC\x80\x00\x01U\xE3\x7FL\x9A37b77d47941e99e430fcb0e0657f5558
last=2GKALMEK7H6220949\x00\x80\x00!\x1A\x80\x00\x01Y\x18\xE6\xB3\
xB42e72036f7e7e03078f41fc82712c5de7
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-0]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
11627a2861e3446e9d6f684ab534563e first=3C7WRNAJ6GG313342\x00\
x80\x00NB\x80\x00\x01V}\xFD\xE4+65bbebdd06dedd8466a31ebd33841a51
last=3N1CE2CP2FL407481\x00\x80\x00\xE0\x16\x80\x00\x01W\x1B\x0A\x02\
xC1fc95d4114d5e91197a5e41bf37c9e8c7
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-1]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
23df78bafd304ff887385a2b6becf06d first=1C4RJFLT6HC742023\x00\
x80\x00x\xFF\x80\x00\x01[J8\x0Ac8b65a80fe1662fb25d80798a66cc83dc
last=1FMCU9J90EUB68140\x00\x80\x01X\xA4\x80\x00\x01[\x1B\xDD\xB2\
x1C577502512ec987844b0108738a9ec6ba
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-3]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
39dc73882bec49a0bdd5d787b06ac032 first=1G1JD5SB5H4136951\x00\
x80\x00!\x8A\x80\x00\x01Z%\xF6\x7Ffef0b8faeeeb4a10103e1a67ea5ebdbec
last=1GNKVHKD7HJ275239\x00\x80\x00$\x87\x80\x00\x01Z%\xF6s\
xDC0961566a370af3b7da440e9705bc4c8c
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-8]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
a37a2a56ff5c48399cf1abd92f99662f first=###########239824\x00\
x80\x01(\xFE\x80\x00\x01Z\xAE\xD6\xE0Xe5a45a2beab337228bdba90c06f34a12
last=1C4RJFLT6HC742023\x00\x80\x00x\xFF\x80\x00\x01[H\xF9w\
x8D60edb518c27ef80f8a751701926d9174
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-10]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
b5843a62a6bd47fbbfc29303bee158e3 first=5FNRL5H4XFB033259\x00\
x80\x00\x1EZ\x80\x00\x01\x5C"\x87s\xF5ce24ec7e2a3698836386bccabc1265af
last=5NPEB4AC3DH620091\x00\x80\x00\xE0\x16\x80\x00\x01X\xE4\x1B\
x9Dq95568f371c1ebd06c497df7129f248a2
2017-05-24 18:00:10,343 INFO  [LoadIncrementalHFiles-5]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
595e89b2fae8494d8a878bc6ba306e2f first=JTHBK1GG4E2122477\x00\
x80\x00$\xE4\x80\x00\x01[\xAF\xF0]%d306ddb81ea3bc093c40efe9f198f03a
last=KNDPMCAC1H7201793\x00\x80\x00\x1E\xC7\x80\x00\x01X\xD5\xF5\xEA\
xD183ff28e7e1aeeff315b167d3d869b797
2017-05-24 18:00:10,348 INFO  [LoadIncrementalHFiles-9]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
ac4edbfdf7394e88aaf84f5489511b7e first=1GNKVHKD7HJ275239\x00\
x80\x00$\x87\x80\x00\x01Z+\x1C\x87\xB65ab1c776dde09cf3257821e420288075
last=1N4BZ0CP4GC308715\x00\x80\x01T\xFC\x80\x00\x01U\xE2\xA1 \
xB743fe1c3b744fae3c2f9e6a896003869f
2017-05-24 18:00:10,348 INFO  [LoadIncrementalHFiles-4]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
5580694921e846949f49c1426162da6e first=2GKALMEK7H6220949\x00\
x80\x00!\x1A\x80\x00\x01Y!\xE5\xCF\xC110e1f23860ff6a21fe848bb85e2e0969
last=3C7WRNAJ6GG313342\x00\x80\x00NB\x80\x00\x01V}\x1D\xF5\
x10620faf6227fb4181a7ee99fa07a52f08
2017-05-24 18:00:10,349 INFO  [LoadIncrementalHFiles-7]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
7bc03c5b15304ec083e18c2376789562 first=1FMCU9J90EUB68140\x00\
x80\x01X\xA4\x80\x00\x01[\x1D&\x9D\x12f80b9cac0af584642ecbd6e1b1bbd617
last=1G1JD5SB5H4136951\x00\x80\x00!\x8A\x80\x00\x01Z$\xB0\xAC\
x1625dc1ae357cb3e9332d823210102de85
2017-05-24 18:00:10,349 INFO  [LoadIncrementalHFiles-13]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
d28c4d2668cc4169b201b349b2522f9a first=3N1CE2CP2FL407531\x00\
x80\x00\x0A\xC9\x80\x00\x01X\xD1\xBC\xD195d73fa7cd13082d2e6f590ca1578b453
last=5FNRL5H4XFB033259\x00\x80\x00\x1EZ\x80\x00\x01\x5C!8\xA7.
dd776deeea36c2c5b132fafed7013b16
2017-05-24 18:00:10,350 INFO  [LoadIncrementalHFiles-6]
mapreduce.LoadIncrementalHFiles:
Trying to load hfile=hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
5d90de564af645f084319a03f64b0a3d first=KNDPMCAC1H7201793\x00\
x80\x00\x1E\xC7\x80\x00\x01X\xD7F#\xEBcd2872ac4f258a091a79a55d4dbfef9e
last=WMWZP3C58FT708786\x00\x80\x00\xE0\x16\x80\x00\x01Y\xB7?\xCA\
xADb28da871b5581e409561c813ed9c0805
2017-05-24 18:00:11,780 WARN  [LoadIncrementalHFiles-15]
mapreduce.LoadIncrementalHFiles:
Attempt to bulk load region containing  into table BI.VIN_IDX with files
[family:I path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
d28c4d2668cc4169b201b349b2522f9a, family:I path:hdfs://HDFS1/user/hbase/
idx_test_7/BI.VIN_IDX/I/23df78bafd304ff887385a2b6becf06d, family:I
path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
39dc73882bec49a0bdd5d787b06ac032, family:I path:hdfs://HDFS1/user/hbase/
idx_test_7/BI.VIN_IDX/I/a37a2a56ff5c48399cf1abd92f99662f, family:I
path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
11627a2861e3446e9d6f684ab534563e, family:I path:hdfs://HDFS1/user/hbase/
idx_test_7/BI.VIN_IDX/I/ac4edbfdf7394e88aaf84f5489511b7e, family:I
path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
f0e97b218aed4abf9949cf49a57e559b, family:I path:hdfs://HDFS1/user/hbase/
idx_test_7/BI.VIN_IDX/I/7bc03c5b15304ec083e18c2376789562, family:I
path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
5d90de564af645f084319a03f64b0a3d, family:I path:hdfs://HDFS1/user/hbase/
idx_test_7/BI.VIN_IDX/I/5580694921e846949f49c1426162da6e, family:I
path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
595e89b2fae8494d8a878bc6ba306e2f, family:I path:hdfs://HDFS1/user/hbase/
idx_test_7/BI.VIN_IDX/I/27c0a905ee174c9898d324acf1554bf9, family:I
path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
c79ae6d27824424f99523dad586e86b1, family:I path:hdfs://HDFS1/user/hbase/
idx_test_7/BI.VIN_IDX/I/b5843a62a6bd47fbbfc29303bee158e3, family:I
path:hdfs://HDFS1/user/hbase/idx_test_7/BI.VIN_IDX/I/
d0a6c4b727bb416f840ed254658f3982] failed.  This is recoverable and they
will be retried.
2017-05-24 18:00:11,793 INFO  [main] mapreduce.LoadIncrementalHFiles: Split
occured while grouping HFiles, retry attempt 1 with 15 files remaining to
group or split


-- 
Thanks & Regards,
Anil Gupta



-- 
Thanks & Regards,
Anil Gupta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message