hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Malte Otten <malte.maltesm...@gmail.com>
Subject Re: NoSuchElementException while executing MapReduce-job
Date Mon, 25 Aug 2014 18:12:23 GMT
Hi Ted,

thanks for your quick answer.
I don't think I'm running in a secure setup, because I didn't do any of 
the things specified in the reference guide chapter from your link and 
as far as I can see, the secure setup is not the default. My 
hbase-site.xml looks like the following:

<configuration>
<property>
   <name>hbase.cluster.distributed</name>
   <value>true</value>
</property>
<property>
<name>hbase.master.wait.on.regionservers.mintostart</name>
   <value>1</value>
</property>
   <property>
     <name>hbase.rootdir</name>
<value>hdfs://my.hbase.server.de:8020/hbase</value>
   </property>
   <property>
     <name>hbase.zookeeper.property.dataDir</name>
     <value>/my/path/hbase-data/zookeeper</value>
   </property>
<property>
   <name>hbase.zookeeper.quorum</name>
   <value>my.hbase.server.de</value>
</property>
</configuration>

Do you have any other suggestions?

Regards
Malte


Am 25.08.2014 um 18:01 schrieb Ted Yu:
> Looks like you were running in a secure setup.
>
> If that's the case, you should read:
> http://hbase.apache.org/book.html#hbase.secure.configuration
>
> Cheers
>
>
> On Mon, Aug 25, 2014 at 5:48 AM, Malte Maltesmann <
> malte.maltesmann@gmail.com> wrote:
>
>> Hi all,
>>
>> when I'm trying to execute a MapReduce-Job over the HBase-Java-API I get an
>> exception with the follwoing Stacktrace:
>>
>> java.util.NoSuchElementException
>>      at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
>>      at
>>
>> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:565)
>>      at
>>
>> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:534)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.checkPermissionOfOther(ClientDistributedCacheManager.java:276)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.isPublic(ClientDistributedCacheManager.java:240)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineCacheVisibilities(ClientDistributedCacheManager.java:162)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:58)
>>      at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
>>      at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
>>      at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
>>      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
>>      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
>>      at java.security.AccessController.doPrivileged(Native Method)
>>      at javax.security.auth.Subject.doAs(Subject.java:415)
>>      at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>      at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
>>      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
>> ...
>>
>> In the RawLocalFileSystem.java I can see, that its trying to get some
>> Permissions here, but the StringTokenizer gets an empty String, so the
>> first t.nextToken() fails with the Exception above:
>>
>> /// loads permissions, owner, and group from `ls -ld`
>>      private void loadPermissionInfo() {
>>        IOException e = null;
>>        try {
>>          String output = FileUtil.execCommand(new File(getPath().toUri()),
>>              Shell.getGetPermissionCommand());
>>          StringTokenizer t =
>>              new StringTokenizer(output, Shell.TOKEN_SEPARATOR_REGEX);
>>          //expected format
>>          //-rw-------    1 username groupname ...
>>          String permission = t.nextToken();
>>          if (permission.length() > FsPermission.MAX_PERMISSION_LENGTH) {
>>            //files with ACLs might have a '+'
>>            permission = permission.substring(0,
>>              FsPermission.MAX_PERMISSION_LENGTH);
>>          }
>>          setPermission(FsPermission.valueOf(permission));
>>          t.nextToken();
>>
>>          String owner = t.nextToken();
>>          // If on windows domain, token format is DOMAIN\\user and we want
>> to
>>          // extract only the user name
>>          if (Shell.WINDOWS) {
>>            int i = owner.indexOf('\\');
>>            if (i != -1)
>>              owner = owner.substring(i + 1);
>>          }
>>          setOwner(owner);
>>
>>          setGroup(t.nextToken());
>>        } catch (Shell.ExitCodeException ioe) {
>>          if (ioe.getExitCode() != 1) {
>>            e = ioe;
>>          } else {
>>            setPermission(null);
>>            setOwner(null);
>>            setGroup(null);
>>          }
>>        } catch (IOException ioe) {
>>          e = ioe;
>>        } finally {
>>          if (e != null) {
>>            throw new RuntimeException("Error while running command to get "
>> +
>>                                       "file permissions : " +
>>                                       StringUtils.stringifyException(e));
>>          }
>>        }
>>      }
>>
>> So my question is, what could have caused this Problem. My setup is the
>> following:
>>
>> Two nodes running on FreeBSD 9.0
>> Hadoop 2.4.1
>> HBase 0.98.4
>>
>> The shell-command "ls -ld" delivers the same output on FreeBSD as on any
>> other Linux-distribution, so this shouldn't be the problem.
>> Also I can use simple operations from the HBase-API like get or put without
>> any problems.
>>
>> I also posted this on stackoverflow
>> <
>> http://stackoverflow.com/questions/25364802/hbase-mapreduce-nosuchelementexception
>> a week ago, but didn't get any answers.
>>
>> Thanks in advance,
>> Malte
>>


Mime
View raw message