accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Bad GC credentials after moving namenode
Date Fri, 06 Mar 2015 18:06:59 GMT
The !SYSTEM user's credentials include a hashing of all properties in 
accumulo-site.xml that begin with 'instance.'. This includes 
instance.secret and has two goals: 1) prevent rogue servers from 
maliciously entering the Accumulo instance. 2) prevent servers with bad 
configurations from accidentally entering the Accumulo instance.

BAD_CREDENTIALS on !SYSTEM likely means that the new host has (or the 
processing on that host are using) a different version of 
accumulo-site.xml than the other servers.

Trying `diff`'ing accumulo-site.xml on the host running your GC and one 
of the hosts running a TabletServer. If they appear to be accurate, you 
can check the print-out of site configuration in the debug.log for the 
GC and see if it somehow picked up a different version of 
accumulo-site.xml than you expected.

Russ Weeks wrote:
> Hi, folks,
>
> I just finished moving my namenode, following the instructions at [1]. I
> think I got everything just about right but now I see these errors in
> the monitor. They seem to be associated with the GC process.
>
> Unable to scan metadata table
> 	java.lang.RuntimeException: org.apache.accumulo.core.client.AccumuloSecurityException:
Error BAD_CREDENTIALS for user !SYSTEM on table accumulo.root(ID:+r) - Username or Password
is Invalid
> 	at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:187)
> 	at org.apache.accumulo.server.util.MetadataTableUtil.getLogEntries(MetadataTableUtil.java:554)
> 	at org.apache.accumulo.server.util.MetadataTableUtil$LogEntryIterator.<init>(MetadataTableUtil.java:622)
> 	at org.apache.accumulo.server.util.MetadataTableUtil.getLogEntries(MetadataTableUtil.java:659)
> 	at org.apache.accumulo.gc.GarbageCollectWriteAheadLogs.removeMetadataEntries(GarbageCollectWriteAheadLogs.java:321)
> 	at org.apache.accumulo.gc.GarbageCollectWriteAheadLogs.collect(GarbageCollectWriteAheadLogs.java:151)
> 	at org.apache.accumulo.gc.SimpleGarbageCollector.run(SimpleGarbageCollector.java:608)
> 	at org.apache.accumulo.gc.SimpleGarbageCollector.main(SimpleGarbageCollector.java:159)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.accumulo.start.Main$1.run(Main.java:141)
> 	at java.lang.Thread.run(Thread.java:745)
> 	Caused by: org.apache.accumulo.core.client.AccumuloSecurityException: Error BAD_CREDENTIALS
for user !SYSTEM on table accumulo.root(ID:+r) - Username or Password is Invalid
> 	at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:458)
> 	at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:279)
> 	at org.apache.accumulo.core.client.impl.ScannerIterator$Reader.run(ScannerIterator.java:84)
> 	at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(ScannerIterator.java:177)
> 		... 13 more
> 	Caused by: ThriftSecurityException(user:!SYSTEM, code:BAD_CREDENTIALS)
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$startScan_resultStandardScheme.read(TabletClientService.java:6684)
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$startScan_resultStandardScheme.read(TabletClientService.java:6661)
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result.read(TabletClientService.java:6584)
> 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startScan(TabletClientService.java:232)
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startScan(TabletClientService.java:208)
> 	at org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:403)
> 		... 16 more
>
>
> Any ideas? Everything seems to be working, eg. I can connect using the
> shell and browse the contents of my tables.
>
> 1:
> http://accumulo.apache.org/1.6/accumulo_user_manual.html#_instance_volumes_replacements

Mime
View raw message