hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Bertozzi <theo.berto...@gmail.com>
Subject Re: Bulk-loading HFiles after table split (on ACL enabled cluster)
Date Tue, 16 Sep 2014 21:28:56 GMT
are you using the SecureBulkLoadEndpoint? that should take care of
permissions
http://hbase.apache.org/book/hbase.secure.bulkload.html

Matteo


On Tue, Sep 16, 2014 at 2:26 PM, Daisy Zhou <daisy@wibidata.com> wrote:

> Hi,
>
> I can't find mention of this issue on the Jira.  Is it known?  I think that
> if a split of the HFiles is required, LoadIncrementalHFiles should create
> the new HFiles with the correct permissions to be bulk-loaded. Currently it
> just hangs because the permissions are wrong.
>
> Here is how I reproduce my issue:
>
> On a cluster with ACL enabled, I generate HFiles for a bulk-load, then
> *force a table split*, and then attempt to bulk-load the HFiles.  The
> bulk-load hangs (similar to when the hfiles' directory is not chown'ed
> properly):
>
> 14/09/15 15:44:41 INFO
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles: Trying to
> load
> hfile=hdfs://bento:8020/user/daisy/kiji-mr-tutorial/hfile-output/part-r-00000.hfile/B/00000
> first=\x00fs\xC0song-32\x00 last=\xFEI\x99~song-44\x0014/09/15
> 15:44:41 INFO org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles:
> HFile at
> hdfs://bento:8020/user/daisy/kiji-mr-tutorial/hfile-output/part-r-00000.hfile/B/00000
> no longer fits inside a single region. Splitting...14/09/15 15:44:42
> INFO org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles:
> Successfully split into new HFiles
>
> hdfs://bento:8020/user/daisy/kiji-mr-tutorial/hfile-output/part-r-00000.hfile/B/_tmp/kiji.kiji_music.table.songs,1.bottom
> and
> hdfs://bento:8020/user/daisy/kiji-mr-tutorial/hfile-output/part-r-00000.hfile/B/_tmp/kiji.kiji_music.table.songs,1.top14/09/15
> 15:44:42 INFO org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles:
> Split occured while grouping HFiles, retry attempt 1 with 2 files
> remaining to group or split
> 14/09/15 15:44:42 INFO
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles: Trying to
> load
> hfile=hdfs://bento:8020/user/daisy/kiji-mr-tutorial/hfile-output/part-r-00000.hfile/B/_tmp/kiji.kiji_music.table.songs,1.top
> first=c\xA8\x0D\x81song-9\x00 last=\xFEI\x99~song-44\x0014/09/15
> 15:44:42 INFO org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles:
> Trying to load
> hfile=hdfs://bento:8020/user/daisy/kiji-mr-tutorial/hfile-output/part-r-00000.hfile/B/_tmp/kiji.kiji_music.table.songs,1.bottom
> first=\x00fs\xC0song-32\x00 last=^49\xDEsong-13\x00
>
>
> If I chmod -R 777 the directory and try again, the bulk load completes
> successfully.
>
>
> Daisy
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message