ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shtykh <rsht...@yahoo.com>
Subject Re: Is it necessary to configure setting HBase which use Secondary File System?
Date Thu, 08 Jun 2017 14:45:18 GMT
Isn't it the point of "hbase.wal.dir" to have WAL in a different directory or even a different
file system? [1]
Also, both HBase WAL and IgniteHadoopFileSystem use 'org.apache.hadoop.fs.FileSystem'. I am
just curious how it breaks HBase.

[1] https://issues.apache.org/jira/browse/HBASE-17437[2] https://github.com/apache/hbase/blob/c38bf12444aca77c7cb12637147c07dc711acbe9/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
-- Roman

    On Thursday, June 8, 2017 5:00 PM, Vladimir Ozerov <vozerov@gridgain.com> wrote:

 Hi Takashi,
"igfs://" prefix should be used in your application code, in those places where data is accessed.
It is illegal to change "hbase.wal.dir" property, as it breaks HBase internals.
On Thu, Jun 8, 2017 at 6:30 AM, Takashi Sasaki <tsasaki609@gmail.com> wrote:


I used igfs:// instead of hdfs:// for hbase.wal.dir property, then
HBase Master Server throwed Exception.

2017-06-08 02:51:56,745 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class
org.apache.hadoop.hbase. master.HMaster.
        at org.apache.hadoop.hbase. master.HMaster. constructMaster(HMaster.java: 2577)
        at org.apache.hadoop.hbase. master.HMasterCommandLine. startMaster( HMasterCommandLine.java:231)
        at org.apache.hadoop.hbase. master.HMasterCommandLine.run( HMasterCommandLine.java:137)
        at org.apache.hadoop.util. ToolRunner.run(ToolRunner. java:70)
        at org.apache.hadoop.hbase.util. ServerCommandLine.doMain( ServerCommandLine.java:126)
        at org.apache.hadoop.hbase. master.HMaster.main(HMaster. java:2587)
Caused by: java.io.IOException: File system is already initialized:
org.apache.ignite.internal. processors.hadoop.impl.igfs. HadoopIgfsWrapper@1dbd580
        at org.apache.ignite.hadoop.fs. v1.IgniteHadoopFileSystem. initialize( IgniteHadoopFileSystem.java:
        at org.apache.hadoop.hbase.fs. HFileSystem.<init>( HFileSystem.java:87)
        at org.apache.hadoop.hbase. regionserver.HRegionServer. initializeFileSystem(
        at org.apache.hadoop.hbase. regionserver.HRegionServer.< init>(HRegionServer.java:576)
        at org.apache.hadoop.hbase. master.HMaster.<init>(HMaster. java:397)
        at sun.reflect. NativeConstructorAccessorImpl. newInstance0(Native Method)

I checked Ignite source code.
It seems to be called initialize method more than once, so the server
throw the exception.

I added properties to core-site.xml:
  <value>org.apache.ignite. hadoop.fs.v1. IgniteHadoopFileSystem</value>
  <name>fs.AbstractFileSystem. igfs.impl</name>
  <value>org.apache.ignite. hadoop.fs.v2. IgniteHadoopFileSystem</value>

I changed property hbase-site.xml:
  <value>igfs://igfs@/user/ hbase/WAL</value>

Hadoop version: 2.7.3
HBase version: 1.3.0
Ignite version: 2.0.0

How can I solve this problem?
Give me advice if you like.


2017-06-07 21:38 GMT+09:00 Takashi Sasaki <tsasaki609@gmail.com>:
> Hello,
> I'm newbie of Ignite, so have some question.
> When I use Secondary File System to write HBase WAL, should I use
> igfs:// instead of hdfs:// ?
> hbase-site.xml(default) is hdfs://.
> ----------
> <property>
>   <name>hbase.wal.dir</name>
>   <value>hdfs://[dnsname]:[port] /user/hbase/WAL</value>
> </property>
> ----------
> Does the secondary file system require some configuration changes to Hbase?
> Please give me advice.
> Thanks,
> Takashi

View raw message