hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Håvard Wahl Kongsgård <haavard.kongsga...@gmail.com>
Subject Re: Error importing hbase table on new system
Date Sun, 27 Sep 2015 11:47:01 GMT
>Is the single node system secure ?

No have not activated, just defaults

the mapred conf.

<?xml version="1.0"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>


<configuration>

  <property>

    <name>mapred.job.tracker</name>

    <value>rack3:8021</value>

  </property>


  <!-- Enable Hue plugins -->

  <property>

    <name>mapred.jobtracker.plugins</name>

    <value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>

    <description>Comma-separated list of jobtracker plug-ins to be activated.

    </description>

  </property>

  <property>

    <name>jobtracker.thrift.address</name>

    <value>0.0.0.0:9290</value>

  </property>

</configuration>


>>Have you checked hdfs healthiness ?


sudo -u hdfs hdfs dfsadmin -report

Configured Capacity: 2876708585472 (2.62 TB)

Present Capacity: 1991514849280 (1.81 TB)

DFS Remaining: 1648230617088 (1.50 TB)

DFS Used: 343284232192 (319.71 GB)

DFS Used%: 17.24%

Under replicated blocks: 52

Blocks with corrupt replicas: 0

Missing blocks: 0


-------------------------------------------------

Datanodes available: 1 (1 total, 0 dead)


Live datanodes:

Name: 127.0.0.1:50010 (localhost)

Hostname: rack3

Decommission Status : Normal

Configured Capacity: 2876708585472 (2.62 TB)

DFS Used: 343284232192 (319.71 GB)

Non DFS Used: 885193736192 (824.40 GB)

DFS Remaining: 1648230617088 (1.50 TB)

DFS Used%: 11.93%

DFS Remaining%: 57.30%

Last contact: Sun Sep 27 13:44:45 CEST 2015


>>To which release of hbase were you importing ?

Hbase 0.94 (CHD 4)

the new one is CHD 5.4

On Sun, Sep 27, 2015 at 1:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> Is the single node system secure ?
> Have you checked hdfs healthiness ?
> To which release of hbase were you importing ?
>
> Thanks
>
>> On Sep 27, 2015, at 3:06 AM, Håvard Wahl Kongsgård <haavard.kongsgaard@gmail.com>
wrote:
>>
>> Hi, Iam trying to import a old backup to a new smaller system (just
>> single node, to get the data out)
>>
>> when I use
>>
>> sudo -u hbase hbase -Dhbase.import.version=0.94
>> org.apache.hadoop.hbase.mapreduce.Import crawler
>> /crawler_hbase/crawler
>>
>> I get this error in the tasks . Is this a permission problem?
>>
>>
>> 2015-09-26 23:56:32,995 ERROR
>> org.apache.hadoop.security.UserGroupInformation:
>> PriviledgedActionException as:mapred (auth:SIMPLE)
>> cause:java.io.IOException: keyvalues=NONE read 4096 bytes, should read
>> 14279
>> 2015-09-26 23:56:32,996 WARN org.apache.hadoop.mapred.Child: Error running child
>> java.io.IOException: keyvalues=NONE read 4096 bytes, should read 14279
>> at org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2221)
>> at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:74)
>> at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:483)
>> at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:76)
>> at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:85)
>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
>> at org.apache.hadoop.mapred.Child.main(Child.java:262)
>> 2015-09-26 23:56:33,002 INFO org.apache.hadoop.mapred.Task: Runnning
>> cleanup for the task
>>
>>
>>
>> --
>> Håvard Wahl Kongsgård
>> Data Scientist



-- 
Håvard Wahl Kongsgård
Data Scientist

Mime
View raw message