hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debraj Manna <subharaj.ma...@gmail.com>
Subject Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Date Wed, 22 Jul 2020 10:14:00 GMT
Yes I am using hadoop 3.0.0.

On Wed, Jul 22, 2020 at 12:31 PM zheng wang <18031031@qq.com> wrote:

> Are you sure you are using hadoop3.0.0?
>
>
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
>                                                   "user"
>                                                                 <
> subharaj.manna@gmail.com&gt;;
> 发送时间:&nbsp;2020年7月22日(星期三) 凌晨3:17
> 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
>
> 主题:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
>
>
>
> Thanks for the pointer.
>
> One more query in I am getting this error from CommonFSUtils.
> invokeSetStoragePolicy()
> <
> https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt
> ;.
> I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
> extends from) I do see the method&nbsp; setStoragePolicy
> <
> https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt
> ;
> .
>
> Can someone explain how is this NoSuchMethodException is being thrown or I
> am looking at the wrong code path for LocalFileSystem?
>
> On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <busbey@apache.org&gt; wrote:
>
> &gt; that is the detail message of why it determined that the FileSystem
> &gt; doesn't support setStoragePolicy. As opposed to e.g. a security
> &gt; manager denying access to introspect the methods available.
> &gt;
> &gt; On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <
> subharaj.manna@gmail.com&gt;
> &gt; wrote:
> &gt; &gt;
> &gt; &gt; I understood the "util.CommonFSUtils: FileSystem doesn't support
> &gt; &gt; setStoragePolicy;" part.
> &gt; &gt;
> &gt; &gt; Can you let me know why it is saying
> "java.lang.NoSuchMethodException:
> &gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> &gt; &gt; 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]" ?
> &gt; &gt;
> &gt; &gt;
> &gt; &gt; On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18031031@qq.com&gt;
> wrote:
> &gt; &gt;
> &gt; &gt; &gt; This log info just as a warning that cant make it disappear
> for now,
> &gt; but
> &gt; &gt; &gt; will not impact anything, so you can just ignore it in
> local mode.
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------
> &gt; &gt; &gt; 发件人:
> &gt; &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> "user"
> &gt; &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> <
> &gt; &gt; &gt; subharaj.manna@gmail.com&amp;gt;;
> &gt; &gt; &gt; 发送时间:&amp;nbsp;2020年7月21日(星期二) 晚上9:19
> &gt; &gt; &gt; 收件人:&amp;nbsp;"Hbase-User"<user@hbase.apache.org&amp;gt;;
> &gt; &gt; &gt;
> &gt; &gt; &gt; 主题:&amp;nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> &gt; &gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; Thanks for replying.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Yes it is a single node hbase cluster. I am not specifying
> any storage
> &gt; &gt; &gt; policy. Looking at the HStore
> &gt; &gt; &gt; <
> &gt; &gt; &gt;
> &gt;
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt
> &gt
> <https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt&gt>;
> &gt; &gt; ;
> &gt; &gt; &gt; code it appears even if no storage policy is specified it
> will take
> &gt; HOT.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Can you explain this a bit more how can I get around this
> error or in a
> &gt; &gt; &gt; single node hbase cluster I should be ignoring this?
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com&amp;gt;
> wrote:
> &gt; &gt; &gt;
> &gt; &gt; &gt; &amp;gt; LocalFileSystem? The&amp;amp;nbsp;setStoragePolicy
> could only be used
> &gt; in
> &gt; &gt; &gt; &amp;gt; distributed hdfs.
> &gt; &gt; &gt; &amp;gt; &amp;amp;nbsp;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> ------------------&amp;amp;nbsp;原始邮件&amp;amp;nbsp;------------------
> &gt; &gt; &gt; &amp;gt; 发件人:
> &gt; &gt; &gt;
> &gt;
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; "user"
> &gt; &gt; &gt;
> &gt;
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; <
> &gt; &gt; &gt; &amp;gt; subharaj.manna@gmail.com&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt; 发送时间:&amp;amp;nbsp;2020年7月21日(星期二)
下午5:58
> &gt; &gt; &gt; &amp;gt; 收件人:&amp;amp;nbsp;"Hbase-User"<
> user@hbase.apache.org&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; 主题:&amp;amp;nbsp;HBase 2.1.0 -
> NoSuchMethodException
> &gt; &gt; &gt; &amp;gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; Hi
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; I am using HBase 2.1.0 with Hadoop 3.0.0. In
hbase
> master logs I
> &gt; am
> &gt; &gt; &gt; &amp;gt; seeing a warning like below
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; 2020-07-20 06:02:24,859 WARN&amp;amp;nbsp;
> [StoreOpener-1588230740-1]
> &gt; &gt; &gt; &amp;gt; util.CommonFSUtils: FileSystem doesn't support
> setStoragePolicy;
> &gt; &gt; &gt; &amp;gt; HDFS-6584, HDFS-9345 not available. This is normal
> and expected on
> &gt; &gt; &gt; &amp;gt; earlier Hadoop versions.
> &gt; &gt; &gt; &amp;gt; java.lang.NoSuchMethodException:
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> &gt; &gt; &gt; &amp;gt; java.lang.String)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt; java.lang.Class.getDeclaredMethod(Class.java:2130)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HStore.<init&amp;amp;gt;(HStore.java:264)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt; java.lang.Thread.run(Thread.java:748)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; As per my understanding, this error should not
be
> coming with
> &gt; Hadoop
> &gt; &gt; &gt; &amp;gt; 3.0.0. Can someone let me know if my understanding
> is correct or
> &gt; what
> &gt; &gt; &gt; &amp;gt; could be going wrong here?
> &gt;
> &gt;
> &gt;
> &gt; --
> &gt; Sean
> &gt;

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message