accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <>
Subject Re: Bad GC credentials after moving namenode
Date Fri, 06 Mar 2015 18:06:59 GMT
The !SYSTEM user's credentials include a hashing of all properties in 
accumulo-site.xml that begin with 'instance.'. This includes 
instance.secret and has two goals: 1) prevent rogue servers from 
maliciously entering the Accumulo instance. 2) prevent servers with bad 
configurations from accidentally entering the Accumulo instance.

BAD_CREDENTIALS on !SYSTEM likely means that the new host has (or the 
processing on that host are using) a different version of 
accumulo-site.xml than the other servers.

Trying `diff`'ing accumulo-site.xml on the host running your GC and one 
of the hosts running a TabletServer. If they appear to be accurate, you 
can check the print-out of site configuration in the debug.log for the 
GC and see if it somehow picked up a different version of 
accumulo-site.xml than you expected.

Russ Weeks wrote:
> Hi, folks,
> I just finished moving my namenode, following the instructions at [1]. I
> think I got everything just about right but now I see these errors in
> the monitor. They seem to be associated with the GC process.
> Unable to scan metadata table
> 	java.lang.RuntimeException: org.apache.accumulo.core.client.AccumuloSecurityException:
Error BAD_CREDENTIALS for user !SYSTEM on table accumulo.root(ID:+r) - Username or Password
is Invalid
> 	at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(
> 	at org.apache.accumulo.server.util.MetadataTableUtil.getLogEntries(
> 	at org.apache.accumulo.server.util.MetadataTableUtil$LogEntryIterator.<init>(
> 	at org.apache.accumulo.server.util.MetadataTableUtil.getLogEntries(
> 	at org.apache.accumulo.gc.GarbageCollectWriteAheadLogs.removeMetadataEntries(
> 	at org.apache.accumulo.gc.GarbageCollectWriteAheadLogs.collect(
> 	at
> 	at org.apache.accumulo.gc.SimpleGarbageCollector.main(
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> 	at java.lang.reflect.Method.invoke(
> 	at org.apache.accumulo.start.Main$
> 	at
> 	Caused by: org.apache.accumulo.core.client.AccumuloSecurityException: Error BAD_CREDENTIALS
for user !SYSTEM on table accumulo.root(ID:+r) - Username or Password is Invalid
> 	at org.apache.accumulo.core.client.impl.ThriftScanner.scan(
> 	at org.apache.accumulo.core.client.impl.ThriftScanner.scan(
> 	at org.apache.accumulo.core.client.impl.ScannerIterator$
> 	at org.apache.accumulo.core.client.impl.ScannerIterator.hasNext(
> 		... 13 more
> 	Caused by: ThriftSecurityException(user:!SYSTEM, code:BAD_CREDENTIALS)
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$startScan_result$
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$
> 	at org.apache.thrift.TServiceClient.receiveBase(
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startScan(
> 	at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startScan(
> 	at org.apache.accumulo.core.client.impl.ThriftScanner.scan(
> 		... 16 more
> Any ideas? Everything seems to be working, eg. I can connect using the
> shell and browse the contents of my tables.
> 1:

View raw message