hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Error importing hbase table on new system
Date Sun, 27 Sep 2015 14:06:20 GMT
Have you verified that the files to be imported are in HFilev2 format ?

http://hbase.apache.org/book.html#_hfile_tool

Cheers

On Sun, Sep 27, 2015 at 4:47 AM, Håvard Wahl Kongsgård <
haavard.kongsgaard@gmail.com> wrote:

> >Is the single node system secure ?
>
> No have not activated, just defaults
>
> the mapred conf.
>
> <?xml version="1.0"?>
>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
>
> <configuration>
>
>   <property>
>
>     <name>mapred.job.tracker</name>
>
>     <value>rack3:8021</value>
>
>   </property>
>
>
>   <!-- Enable Hue plugins -->
>
>   <property>
>
>     <name>mapred.jobtracker.plugins</name>
>
>     <value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
>
>     <description>Comma-separated list of jobtracker plug-ins to be
> activated.
>
>     </description>
>
>   </property>
>
>   <property>
>
>     <name>jobtracker.thrift.address</name>
>
>     <value>0.0.0.0:9290</value>
>
>   </property>
>
> </configuration>
>
>
> >>Have you checked hdfs healthiness ?
>
>
> sudo -u hdfs hdfs dfsadmin -report
>
> Configured Capacity: 2876708585472 (2.62 TB)
>
> Present Capacity: 1991514849280 (1.81 TB)
>
> DFS Remaining: 1648230617088 (1.50 TB)
>
> DFS Used: 343284232192 (319.71 GB)
>
> DFS Used%: 17.24%
>
> Under replicated blocks: 52
>
> Blocks with corrupt replicas: 0
>
> Missing blocks: 0
>
>
> -------------------------------------------------
>
> Datanodes available: 1 (1 total, 0 dead)
>
>
> Live datanodes:
>
> Name: 127.0.0.1:50010 (localhost)
>
> Hostname: rack3
>
> Decommission Status : Normal
>
> Configured Capacity: 2876708585472 (2.62 TB)
>
> DFS Used: 343284232192 (319.71 GB)
>
> Non DFS Used: 885193736192 (824.40 GB)
>
> DFS Remaining: 1648230617088 (1.50 TB)
>
> DFS Used%: 11.93%
>
> DFS Remaining%: 57.30%
>
> Last contact: Sun Sep 27 13:44:45 CEST 2015
>
>
> >>To which release of hbase were you importing ?
>
> Hbase 0.94 (CHD 4)
>
> the new one is CHD 5.4
>
> On Sun, Sep 27, 2015 at 1:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > Is the single node system secure ?
> > Have you checked hdfs healthiness ?
> > To which release of hbase were you importing ?
> >
> > Thanks
> >
> >> On Sep 27, 2015, at 3:06 AM, Håvard Wahl Kongsgård <
> haavard.kongsgaard@gmail.com> wrote:
> >>
> >> Hi, Iam trying to import a old backup to a new smaller system (just
> >> single node, to get the data out)
> >>
> >> when I use
> >>
> >> sudo -u hbase hbase -Dhbase.import.version=0.94
> >> org.apache.hadoop.hbase.mapreduce.Import crawler
> >> /crawler_hbase/crawler
> >>
> >> I get this error in the tasks . Is this a permission problem?
> >>
> >>
> >> 2015-09-26 23:56:32,995 ERROR
> >> org.apache.hadoop.security.UserGroupInformation:
> >> PriviledgedActionException as:mapred (auth:SIMPLE)
> >> cause:java.io.IOException: keyvalues=NONE read 4096 bytes, should read
> >> 14279
> >> 2015-09-26 23:56:32,996 WARN org.apache.hadoop.mapred.Child: Error
> running child
> >> java.io.IOException: keyvalues=NONE read 4096 bytes, should read 14279
> >> at
> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2221)
> >> at
> org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:74)
> >> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:483)
> >> at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:76)
> >> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:85)
> >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
> >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
> >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
> >> at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> >> at java.security.AccessController.doPrivileged(Native Method)
> >> at javax.security.auth.Subject.doAs(Subject.java:415)
> >> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> >> at org.apache.hadoop.mapred.Child.main(Child.java:262)
> >> 2015-09-26 23:56:33,002 INFO org.apache.hadoop.mapred.Task: Runnning
> >> cleanup for the task
> >>
> >>
> >>
> >> --
> >> Håvard Wahl Kongsgård
> >> Data Scientist
>
>
>
> --
> Håvard Wahl Kongsgård
> Data Scientist
>

Mime
View raw message