hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From YouPeng Yang <yypvsxf19870...@gmail.com>
Subject Re: Exception comes out when counting the rows
Date Tue, 23 Apr 2013 10:12:38 GMT
Hi All

    I notice that there are lots of fimilar issues ,however there is no
perfect responses to these issues.
    I post this mail  to close the case because I happen to get the reason .
    Hope to be helpfull.

    The exception is due to the configuration variable mapred.job.tracker =
local  by default.
    I set mapred.job.tracker = MyJobtrackerIP:post,and it works.

    In addition,I have an declaration:
    Anyone should close the issue if he or she gets the  answers that
resolve the issues posted by himself  in this mail list.
    We should  help each other if we get the answer since we are in the
community






Regards



2013/4/23 Ramki Palle <ramki.palle@gmail.com>

> Normally, you do not use load data when you use external table. You either
> specify location in the create external table query or use alter table
> <tablename> set location syntax to set the location of the files that the
> external table stores the data.
>
> In this case, try describe extended NMS_CMTS_CPU_CDX_TEST and see what is
> the value for location.
>
> It is better to use the location as I mentioned above for external tables
> instead of using load data. This way there is no confusion between managed
> tables and external tables locations.
>
> Regards,
> Ramki.
>
>
> On Mon, Apr 22, 2013 at 2:01 AM, YouPeng Yang <yypvsxf19870706@gmail.com>wrote:
>
>>
>> Hi hive users
>>
>>   Sorry for missing the title on the previous mail.
>>
>>   This is my first time to post a question here.
>>
>> I have gotten an exception when I count the rows of  my hive table after
>> I have loaded the data:
>>
>> hive>create EXTERNAL TABLE  NMS_CMTS_CPU_CDX_TEST (CMTSID INT,MSEQ
>> INT,GOTTIME BIGINT,CMTSINDEX INT,CPUTOTAL INT,DESCR STRING) ROW FORMAT
>>  DELIMITED FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n'  STORED AS
>> TEXTFILE;
>>
>> hive>load data inpath '/user/sqoop/NMS_CMTS_CPU_CDX3/NMS_CMTS_CPU_CDX3'
>>  into table NMS_CMTS_CPU_CDX_TEST;
>>
>> hive> select count(1) from NMS_CMTS_CPU_CDX_TEST;
>> I get an exception on step 3,logs are as follows.
>>
>> Any helps will be gratefull.
>>
>> Regards
>>
>> -----------------------------------------------------------------------------------
>> ....
>> WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please
>> use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties
>> files.
>> Execution log at:
>> /tmp/hive/hive_20130422162020_791a7b61-6ba0-466d-99ba-5c2556bafaa4.log
>> Job running in-process (local Hadoop)
>> Hadoop job information for null: number of mappers: 1; number of
>> reducers: 1
>> 2013-04-22 16:24:21,604 null map = 0%,  reduce = 0%
>> 2013-04-22 16:25:21,965 null map = 0%,  reduce = 0%
>> 2013-04-22 16:26:22,902 null map = 0%,  reduce = 0%
>> 2013-04-22 16:26:27,312 null map = 100%,  reduce = 0%
>> Ended Job = job_1364348895095_0055 with errors
>> Error during job, obtaining debugging information...
>> Examining task ID: task_1364348895095_0055_m_000000 (and more) from job
>> job_1364348895095_0055
>> Unable to retrieve URL for Hadoop Task logs. Does not contain a valid
>> host:port authority: local
>>
>> Task with the most failures(4):
>> -----
>> Task ID:
>>   task_1364348895095_0055_m_000000
>>
>> URL:
>>   Unavailable
>> -----
>> Diagnostic Messages for this Task:
>> Error: java.lang.RuntimeException: java.io.FileNotFoundException:
>> /tmp/hive/hive_2013-04-22_16-20-45_720_3839682514463028560/-mr-10001/89dd576e-fb9d-409a-8b46-2e46b7d21160
>> (No such file or directory)
>> at
>> org.apache.hadoop.hive.ql.exec.Utilities.getMapRedWork(Utilities.java:224)
>>  at
>> org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
>> at
>> org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:381)
>>  at
>> org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:374)
>> at
>> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:536)
>>  at
>> org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:160)
>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:381)
>>  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
>> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:152)
>>  at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>>  at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1332)
>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:147)
>> Caused by: java.io.FileNotFoundException:
>> /tmp/hive/hive_2013-04-22_16-20-45_720_3839682514463028560/-mr-10001/89dd576e-fb9d-409a-8b46-2e46b7d21160
>> (No such file or directory)
>> at java.io.FileInputStream.open(Native Method)
>>  at java.io.FileInputStream.<init>(FileInputStream.java:120)
>> at java.io.FileInputStream.<init>(FileInputStream.java:79)
>>  at
>> org.apache.hadoop.hive.ql.exec.Utilities.getMapRedWork(Utilities.java:215)
>> ... 12 more
>>
>>
>>  Execution failed with exit status: 2
>> 13/04/22 16:26:28 ERROR exec.Task: Execution failed with exit status: 2
>> Obtaining error information
>> 13/04/22 16:26:28 ERROR exec.Task: Obtaining error information
>>
>> Task failed!
>> Task ID:
>>   Stage-1
>>
>> Logs:
>>
>> 13/04/22 16:26:28 ERROR exec.Task:
>> Task failed!
>> Task ID:
>>   Stage-1
>>
>> Logs:
>>
>> 13/04/22 16:26:28 ERROR exec.ExecDriver: Execution failed with exit
>> status: 2
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> 13/04/22 16:26:28 ERROR ql.Driver: FAILED: Execution Error, return code 2
>> from org.apache.hadoop.hive.ql.exec.MapRedTask
>> 13/04/22 16:26:28 INFO ql.Driver: </PERFLOG method=Driver.execute
>> start=1366618840573 end=1366619188646 duration=348073>
>> 13/04/22 16:26:28 INFO ql.Driver: <PERFLOG method=releaseLocks>
>> 13/04/22 16:26:28 INFO ql.Driver: </PERFLOG method=releaseLocks
>> start=1366619188646 end=1366619188646 duration=0>
>>
>>
>>
>

Mime
View raw message