hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artem Ervits <are9...@nyp.org>
Subject Re: NoSuchElementException while executing MapReduce-job
Date Wed, 27 Aug 2014 14:44:25 GMT
Check your pom.xml, some artifacts changed from earlier releases.


Artem Ervits
Data Analyst
New York Presbyterian Hospital

----- Original Message -----
From: Malte Otten [mailto:malte.maltesmann@gmail.com]
Sent: Monday, August 25, 2014 02:12 PM
To: user@hbase.apache.org <user@hbase.apache.org>
Subject: Re: NoSuchElementException while executing MapReduce-job

Hi Ted,

thanks for your quick answer.
I don't think I'm running in a secure setup, because I didn't do any of
the things specified in the reference guide chapter from your link and
as far as I can see, the secure setup is not the default. My
hbase-site.xml looks like the following:

<configuration>
<property>
   <name>hbase.cluster.distributed</name>
   <value>true</value>
</property>
<property>
<name>hbase.master.wait.on.regionservers.mintostart</name>
   <value>1</value>
</property>
   <property>
     <name>hbase.rootdir</name>
<value>hdfs://my.hbase.server.de:8020/hbase</value>
   </property>
   <property>
     <name>hbase.zookeeper.property.dataDir</name>
     <value>/my/path/hbase-data/zookeeper</value>
   </property>
<property>
   <name>hbase.zookeeper.quorum</name>
   <value>my.hbase.server.de</value>
</property>
</configuration>

Do you have any other suggestions?

Regards
Malte


Am 25.08.2014 um 18:01 schrieb Ted Yu:
> Looks like you were running in a secure setup.
>
> If that's the case, you should read:
> http://hbase.apache.org/book.html#hbase.secure.configuration
>
> Cheers
>
>
> On Mon, Aug 25, 2014 at 5:48 AM, Malte Maltesmann <
> malte.maltesmann@gmail.com> wrote:
>
>> Hi all,
>>
>> when I'm trying to execute a MapReduce-Job over the HBase-Java-API I get an
>> exception with the follwoing Stacktrace:
>>
>> java.util.NoSuchElementException
>>      at java.util.StringTokenizer.nextToken(StringTokenizer.java:349)
>>      at
>>
>> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:565)
>>      at
>>
>> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:534)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.checkPermissionOfOther(ClientDistributedCacheManager.java:276)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.isPublic(ClientDistributedCacheManager.java:240)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineCacheVisibilities(ClientDistributedCacheManager.java:162)
>>      at
>>
>> org.apache.hadoop.mapreduce.filecache.ClientDistributedCacheManager.determineTimestampsAndCacheVisibilities(ClientDistributedCacheManager.java:58)
>>      at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:265)
>>      at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
>>      at
>>
>> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
>>      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
>>      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
>>      at java.security.AccessController.doPrivileged(Native Method)
>>      at javax.security.auth.Subject.doAs(Subject.java:415)
>>      at
>>
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
>>      at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
>>      at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1303)
>> ...
>>
>> In the RawLocalFileSystem.java I can see, that its trying to get some
>> Permissions here, but the StringTokenizer gets an empty String, so the
>> first t.nextToken() fails with the Exception above:
>>
>> /// loads permissions, owner, and group from `ls -ld`
>>      private void loadPermissionInfo() {
>>        IOException e = null;
>>        try {
>>          String output = FileUtil.execCommand(new File(getPath().toUri()),
>>              Shell.getGetPermissionCommand());
>>          StringTokenizer t =
>>              new StringTokenizer(output, Shell.TOKEN_SEPARATOR_REGEX);
>>          //expected format
>>          //-rw-------    1 username groupname ...
>>          String permission = t.nextToken();
>>          if (permission.length() > FsPermission.MAX_PERMISSION_LENGTH) {
>>            //files with ACLs might have a '+'
>>            permission = permission.substring(0,
>>              FsPermission.MAX_PERMISSION_LENGTH);
>>          }
>>          setPermission(FsPermission.valueOf(permission));
>>          t.nextToken();
>>
>>          String owner = t.nextToken();
>>          // If on windows domain, token format is DOMAIN\\user and we want
>> to
>>          // extract only the user name
>>          if (Shell.WINDOWS) {
>>            int i = owner.indexOf('\\');
>>            if (i != -1)
>>              owner = owner.substring(i + 1);
>>          }
>>          setOwner(owner);
>>
>>          setGroup(t.nextToken());
>>        } catch (Shell.ExitCodeException ioe) {
>>          if (ioe.getExitCode() != 1) {
>>            e = ioe;
>>          } else {
>>            setPermission(null);
>>            setOwner(null);
>>            setGroup(null);
>>          }
>>        } catch (IOException ioe) {
>>          e = ioe;
>>        } finally {
>>          if (e != null) {
>>            throw new RuntimeException("Error while running command to get "
>> +
>>                                       "file permissions : " +
>>                                       StringUtils.stringifyException(e));
>>          }
>>        }
>>      }
>>
>> So my question is, what could have caused this Problem. My setup is the
>> following:
>>
>> Two nodes running on FreeBSD 9.0
>> Hadoop 2.4.1
>> HBase 0.98.4
>>
>> The shell-command "ls -ld" delivers the same output on FreeBSD as on any
>> other Linux-distribution, so this shouldn't be the problem.
>> Also I can use simple operations from the HBase-API like get or put without
>> any problems.
>>
>> I also posted this on stackoverflow
>> <
>> http://stackoverflow.com/questions/25364802/hbase-mapreduce-nosuchelementexception
>> a week ago, but didn't get any answers.
>>
>> Thanks in advance,
>> Malte
>>


________________________________

This electronic message is intended to be for the use only of the named recipient, and may
contain information that is confidential or privileged. If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited. If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message. Thank you.

This electronic message is intended to be for the use only of the named recipient, and may
contain information that is confidential or privileged.  If you are not the intended recipient,
you are hereby notified that any disclosure, copying, distribution or use of the contents
of this message is strictly prohibited.  If you have received this message in error or are
not the named recipient, please notify us immediately by contacting the sender at the electronic
mail address noted above, and delete and destroy all copies of this message.  Thank you.

Mime
View raw message