hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: What is our current understanding regards state of integration tests?
Date Mon, 01 Dec 2014 03:57:21 GMT
If you want to run IntegrationTestIngestWithEncryption, follow the
instructions in the security section of the manual when setting up the test
cluster first. It's not a bug that some configuration is necessary.

hbase-it doesn't have much deployment tooling. We ssh around here and there
via ClusterManager. We could do more. Maybe with Puppet?

On Sun, Nov 30, 2014 at 10:54 PM, Andrew Purtell <apurtell@apache.org>
wrote:

> Yes, you have to set up cluster configuration for encryption or the IT
> won't work. Encryption requires creating a keystore, etc.  If you run the
> test as all-localhost, using mvn verify, it should pass, because all
> configuration can be programatically set up and deployed since the
> environment is a mini cluster. You can see how the test sets up the
> necessary configuration using HTU for that case.
>
>
> On Sun, Nov 30, 2014 at 6:52 PM, Stack <stack@duboce.net> wrote:
>
>> I just tried running all tests and notice that at least
>> IntegrationTestIngestWithEcnryption fails reliably with the below. Is this
>> expected? Does this test require a particular context setup first to run,
>> one that is not present normally? Once it has run, I cannot get past it.
>> If I restart servers, they fail on log replay with same exception. Is this
>> expected?
>>
>> On IntegrationTests in general, I was thinking they should generally pass
>> and that it is a bug if they do not.  Is that what others think?
>>
>> Has anyone been running IT tests regularly? If so, what has been your
>> experience? Or if you have been running it tests, do you run individual
>> tests?
>>
>> Thanks. Just trying to figure what current understanding of their state is
>> before digging in.
>> Yours,
>> St.Ack
>>
>> 2014-11-29 21:19:06,447 DEBUG
>> [B.defaultRpcServer.handler=14,queue=2,port=16020] regionserver.HRegion:
>> Flush requested on
>>
>> IntegrationTestIngestWithEncryption,cccccccc,1417324570806.325074071473eb28b662f7d694e8c609.
>> 2014-11-29 21:19:06,640 FATAL [MemStoreFlusher.1]
>> regionserver.HRegionServer: ABORTING region server
>> c2021.halxg.cloudera.com,16020,1417323325816:
>> Replay of HLog required. Forcing server shutdown
>> org.apache.hadoop.hbase.DroppedSnapshotException: region:
>>
>> IntegrationTestIngestWithEncryption,,1417324570806.55e317f49ddf8ed35323d9b675611548.
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1964)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1730)
>>         at
>> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1662)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:434)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:407)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$800(MemStoreFlusher.java:69)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:225)
>>         at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
>> KeyProvider scheme should specify KeyStore type
>>         at
>>
>> org.apache.hadoop.hbase.io.crypto.Encryption.getKeyProvider(Encryption.java:535)
>>         at
>>
>> org.apache.hadoop.hbase.io.crypto.Encryption.getSecretKeyForSubject(Encryption.java:425)
>>         at
>>
>> org.apache.hadoop.hbase.io.crypto.Encryption.encryptWithSubjectKey(Encryption.java:449)
>>         at
>>
>> org.apache.hadoop.hbase.security.EncryptionUtil.wrapKey(EncryptionUtil.java:92)
>>         at
>>
>> org.apache.hadoop.hbase.io.hfile.HFileWriterV3.finishClose(HFileWriterV3.java:127)
>>         at
>>
>> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.close(HFileWriterV2.java:366)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.StoreFile$Writer.close(StoreFile.java:986)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.StoreFlusher.finalizeWriter(StoreFlusher.java:67)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:80)
>>         at
>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:871)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2080)
>>         at
>>
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1918)
>>         ... 7 more
>> Caused by: java.lang.RuntimeException: KeyProvider scheme should specify
>> KeyStore type
>>         at
>>
>> org.apache.hadoop.hbase.io.crypto.KeyStoreKeyProvider.init(KeyStoreKeyProvider.java:142)
>>         at
>>
>> org.apache.hadoop.hbase.io.crypto.Encryption.getKeyProvider(Encryption.java:528)
>>         ... 18 more
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message