phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <saint....@gmail.com>
Subject Re: [VOTE] The first hbase-2.0.0-beta-1 Release Candidate is available
Date Sun, 31 Dec 2017 14:58:05 GMT
There is nothing further up in the master log from regionservers or on
regionservers side on open?

Thanks,
S

On Dec 31, 2017 8:37 AM, "stack" <saint.ack@gmail.com> wrote:

> Good questions.  If you disable snappy does it work?  If you start over
> fresh does it work?  It should be picking up native libs.  Make an issue
> please jms.  Thanks for giving it a go.
>
> S
>
> On Dec 30, 2017 11:49 PM, "Jean-Marc Spaggiari" <jean-marc@spaggiari.org>
> wrote:
>
>> Hi Stack,
>>
>> I just tried to give it a try... Wipe out all HDFS content and code, all
>> HBase content and code, and all ZK. Re-build a brand new cluster with 7
>> physical worker nodes. I'm able to get HBase start, how-ever I'm not able
>> to get my regions online.
>>
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node8.16020,1514693333206, table=pageMini,
>> region=a778eb67898dfd378e426f2e7700faea
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node6.16020,1514693336563, table=work_proposed,
>> region=4a1d86197ace3f4c8b1c8de28dbe1d34
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node1.16020,1514693336898, table=page_crc,
>> region=86b3912a09a5676b6851636ed22c2abc
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node7.16020,1514693337406, table=pageAvro,
>> region=391784c43c87bdea6df05f96accad0ff
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node8.16020,1514693333206, table=page,
>> region=5850d782a3beea18872769bf8fd70fc7
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node5.16020,1514693330961, table=work_proposed,
>> region=1d892c9b54b66f802b82c2f9fe847f1f
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node5.16020,1514693330961, table=pageAvro,
>> region=e9de2c68cc01883e959d7953a4251687
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node3.16020,1514693337210, table=page,
>> region=e2e5fc1c262273893f10e92f24817d1b
>> 2017-12-31 00:42:03,187 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node3.16020,1514693337210, table=page,
>> region=89c443c09f10bd1584b1bb86a637e1a8
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node5.16020,1514693330961, table=page,
>> region=8ca93e9285233ca7b31992f194056bc1
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node4.16020,1514693339685, table=work_proposed,
>> region=9afcf06c4d0d21d7e04b0223edcfc40a
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node6.16020,1514693336563, table=page,
>> region=3457b3237c576eecd550eccee3f584cd
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node1.16020,1514693336898, table=page,
>> region=dd5fb1dbd41945a9ccbc110b8d4a51b5
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node7.16020,1514693337406, table=work_proposed,
>> region=480bb37af54d9fa57c727da9e8a33578
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node8.16020,1514693333206, table=page_crc,
>> region=56b18d470a569c5474ea084f0d995726
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node6.16020,1514693336563, table=page_duplicate,
>> region=e744a9af161de965c70c7d1a08b07660
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node1.16020,1514693336898, table=page_proposed,
>> region=1c75e53308acac6313db4be63c2b48fe
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node8.16020,1514693333206, table=work_proposed,
>> region=45a25ba85f6341a177db7b15554259f9
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node3.16020,1514693337210, table=work_proposed,
>> region=d0a58b76ad9376b12b3e763660049d3d
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node3.16020,1514693337210, table=page,
>> region=599a4b7b21b1d93fa232ebbbef37a31b
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node1.16020,1514693336898, table=page_proposed,
>> region=55c07269cc907b8e8875c2a1c4ec27d5
>> 2017-12-31 00:42:03,188 WARN  [ProcExecTimeout]
>> assignment.AssignmentManager: TODO Handle stuck in transition:
>> rit=OPENING,
>> location=node5.,16020,1514693330961, table=page_crc,
>> region=fa3a3d7ebc64ce2a5494cae01477d8d8
>>
>> I'm 99% confident this is because of SNAPPY. I'm fighting to get it
>> working
>> but it's such a pain! My concern here is I don't see any exception
>> anywhere
>> on any logs. Nothing on the RS side, nothing on the master side (Except
>> extract above).
>>
>> I suspect it's snappy because of this:
>>
>> hbase@node2:~/hbase-2.0.0-beta-1$ bin/hbase
>> org.apache.hadoop.hbase.util.CompressionTest hdfs://node2/tmp/snappy
>> snappy
>> 2017-12-31 00:45:31,006 WARN  [main] util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes
>> where
>> applicable
>> 2017-12-31 00:45:33,283 INFO  [main] metrics.MetricRegistries: Loaded
>> MetricRegistries class
>> org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
>> 2017-12-31 00:45:33,366 INFO  [main] hfile.CacheConfig: Created
>> cacheConfig: CacheConfig:disabled
>> Exception in thread "main" java.lang.RuntimeException: native snappy
>> library not available: this version of libhadoop was built without snappy
>> support.
>>         at
>> org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoa
>> ded(SnappyCodec.java:65)
>>         at
>> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(
>> SnappyCodec.java:134)
>>         at
>> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150)
>>         at
>> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168)
>>         at
>> org.apache.hadoop.hbase.io.compress.Compression$Algorithm.
>> getCompressor(Compression.java:355)
>>         at
>> org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultEncodin
>> gContext.<init>(HFileBlockDefaultEncodingContext.java:90)
>>         at
>> org.apache.hadoop.hbase.io.hfile.NoOpDataBlockEncoder.newDat
>> aBlockEncodingContext(NoOpDataBlockEncoder.java:85)
>>         at
>> org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(
>> HFileBlock.java:923)
>>         at
>> org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.finishInit(
>> HFileWriterImpl.java:296)
>>         at
>> org.apache.hadoop.hbase.io.hfile.HFileWriterImpl.<init>(HFil
>> eWriterImpl.java:186)
>>         at
>> org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(
>> HFile.java:339)
>>         at
>> org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(Com
>> pressionTest.java:129)
>>         at
>> org.apache.hadoop.hbase.util.CompressionTest.main(Compressio
>> nTest.java:167)
>>
>> But I think my installation is fine:
>> hbase@node2:~/hbase-2.0.0-beta-1$ ll native-build/
>> total 308
>> lrwxrwxrwx 1 hbase hbase     24 déc 31 00:29 libhadoopsnappy.so ->
>> libhadoopsnappy.so.0.0.1
>> lrwxrwxrwx 1 hbase hbase     24 déc 31 00:29 libhadoopsnappy.so.0 ->
>> libhadoopsnappy.so.0.0.1
>> -rwxr-xr-x 1 hbase hbase 120144 déc 31 00:29 libhadoopsnappy.so.0.0.1
>> lrwxrwxrwx 1 hbase hbase     18 déc  1  2012 libsnappy.so ->
>> libsnappy.so.1.1.3
>> lrwxrwxrwx 1 hbase hbase     18 déc  1  2012 libsnappy.so.1 ->
>> libsnappy.so.1.1.3
>> -rwxr-xr-x 1 hbase hbase 178210 déc  1  2012 libsnappy.so.1.1.3
>> drwxr-xr-x 3 hbase hbase   4096 déc 30 15:44 python2.6
>> drwxr-xr-x 4 hbase hbase   4096 déc 30 23:35 python2.7
>> drwxr-xr-x 3 hbase hbase   4096 déc 30 23:29 python3.5
>>
>> an in hbase-env.sh:
>> export JAVA_HOME=/usr/local/jdk1.8.0_151
>> export HBASE_LIBRARY_PATH=/home/hbase/hbase-2.0.0-beta-1/native-build
>>
>>
>> So there is 2 things here.
>> 1) Why are the region servers not reporting any error when they are not
>> able to open a region because of the compression codec not being loaded?
>> 2) Why is HBase not picking up the Snappy codec.
>>
>> Thanks,
>>
>> JMS
>>
>>
>> 2017-12-29 13:15 GMT-05:00 Stack <stack@duboce.net>:
>>
>> > The first release candidate for HBase 2.0.0-beta-1 is up at:
>> >
>> >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-beta-1-RC0/
>> >
>> > Maven artifacts are available from a staging directory here:
>> >
>> >  https://repository.apache.org/content/repositories/orgapachehbase-1188
>> >
>> > All was signed with my key at 8ACC93D2 [1]
>> >
>> > I tagged the RC as 2.0.0-beta-1-RC0
>> > (0907563eb72697b394b8b960fe54887d6ff304fd)
>> >
>> > hbase-2.0.0-beta-1 is our first beta release. It includes all that was
>> in
>> > previous alphas (new assignment manager, offheap read/write path,
>> in-memory
>> > compactions, etc.). The APIs and feature-set are sealed.
>> >
>> > hbase-2.0.0-beta-1 is a not-for-production preview of hbase-2.0.0. It is
>> > meant for devs and downstreamers to test drive and flag us if we messed
>> up
>> > on anything ahead of our rolling GAs. We are particular interested in
>> > hearing from Coprocessor developers.
>> >
>> > The list of features addressed in 2.0.0 so far can be found here [3].
>> There
>> > are thousands. The list of ~2k+ fixes in 2.0.0 exclusively can be found
>> > here [4] (My JIRA JQL foo is a bit dodgy -- forgive me if mistakes).
>> >
>> > I've updated our overview doc. on the state of 2.0.0 [6]. We'll do one
>> more
>> > beta before we put up our first 2.0.0 Release Candidate by the end of
>> > January, 2.0.0-beta-2. Its focus will be making it so users can do a
>> > rolling upgrade on to hbase-2.x from hbase-1.x (and any bug fixes found
>> > running beta-1). Here is the list of what we have targeted so far for
>> > beta-2 [5]. Check it out.
>> >
>> > One knownissue is that the User API has not been properly filtered so it
>> > shows more than just InterfaceAudience Public content (HBASE-19663, to
>> be
>> > fixed by beta-2).
>> >
>> > Please take this beta for a spin. Please vote on whether it ok to put
>> out
>> > this RC as our first beta (Note CHANGES has not yet been updated). Let
>> the
>> > VOTE be open for 72 hours (Monday)
>> >
>> > Thanks,
>> > Your 2.0.0 Release Manager
>> >
>> > 1. http://pgp.mit.edu/pks/lookup?op=get&search=0x9816C7FC8ACC93D2
>> > 3. https://goo.gl/scYjJr
>> > 4. https://goo.gl/dFFT8b
>> > 5. https://issues.apache.org/jira/projects/HBASE/versions/12340862
>> > 6. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
>> > ktczrlKHK8N4SZzs/
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message