hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexandre Linte (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-15415) Random "java.util.ConcurrentModificationException"
Date Mon, 12 Dec 2016 10:47:59 GMT
Alexandre Linte created HIVE-15415:
--------------------------------------

             Summary: Random "java.util.ConcurrentModificationException"
                 Key: HIVE-15415
                 URL: https://issues.apache.org/jira/browse/HIVE-15415
             Project: Hive
          Issue Type: Bug
          Components: Beeline
    Affects Versions: 2.1.0
         Environment: Hadoop 2.7.3, Hive 2.1.0
            Reporter: Alexandre Linte


I'm regularly facing Hive job failures through Oozie or through the beeline CLI. The jobs
exit with an error "FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask.
java.util.ConcurrentModificationException (state=08S01,code=1)" but not 100% of the time.

it's also important to underline that only one user is working on the table when the jobs
are running.

- stderr
{noformat}
Connecting to jdbc:hive2://hiveserver2.bigdata.fr:10000/default
Connected to: Apache Hive (version 2.1.0)
Driver: Hive JDBC (version 2.1.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
No rows affected (1.475 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (58.977 seconds)
No rows affected (5.524 seconds)
No rows affected (5.235 seconds)
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask.
java.util.ConcurrentModificationException (state=08S01,code=1)
Closing: 0: jdbc:hive2://hiveserver2.bigdata.fr:10000/default
Intercepting System.exit(2)
{noformat}

- stdout
{noformat}
Beeline command arguments :
             -u
             jdbc:hive2://hiveserver2.bigdata.fr:10000/default
             -n
             my_user
             -p
             DUMMY
             -d
             org.apache.hive.jdbc.HiveDriver
             -f
             full_job
             -a
             delegationToken
             --hiveconf
             mapreduce.job.tags=oozie-75b060aacd7ec48c4ed637855e413280

Fetching child yarn jobs
tag id : oozie-75b060aacd7ec48c4ed637855e413280
Child yarn jobs are found -
=================================================================

>>> Invoking Beeline command line now >>>

0: jdbc:hive2://hiveserver2.bigdata.fr> use my_db;
0: jdbc:hive2://hiveserver2.bigdata.fr> set hive.execution.engine=tez;
0: jdbc:hive2://hiveserver2.bigdata.fr> set tez.queue.name=tez_queue;
0: jdbc:hive2://hiveserver2.bigdata.fr>
0: jdbc:hive2://hiveserver2.bigdata.fr> insert overwrite table main_table ^M_fd_livcfm
. . . . . . . . . . . . . . . . . . . . . . .> select
. . . . . . . . . . . . . . . . . . . . . . .> col.co_cd as co_cd,
. . . . . . . . . . . . . . . . . . . . . . .> col.line_co_cd as line_co_cd,
. . . . . . . . . . . . . . . . . . . . . . .> unix_timestamp(min(tt.statut_dt)) ^M as
statut_dt
. . . . . . . . . . . . . . . . . . . . . . .> from dlk_scf_rn_customer_order_li ^Mne col
. . . . . . . . . . . . . . . . . . . . . . .> join dlk_scf_rn_shipment_handling ^M_utility
shu
. . . . . . . . . . . . . . . . . . . . . . .> on shu.co_cd =col.co_cd
. . . . . . . . . . . . . . . . . . . . . . .> and shu.line_co_cd = col.line_co_ ^Mcd
. . . . . . . . . . . . . . . . . . . . . . .> join ( select scaler_internal_ref ^M, statut_dt,recep_number,state,reason
. . . . . . . . . . . . . . . . . . . . . . .> from dlk_scf_rn_transport_trackin ^Mg where
state='LIV' and reason='CFM' ) tt
. . . . . . . . . . . . . . . . . . . . . . .> on concat('CAL',shu.c_waybill_no) ^M =tt.scaler_internal_ref
group by col.co_cd,col.line_co_cd;
Heart beat
Heart beat
0: jdbc:hive2://hiveserver2.bigdata.fr>
0: jdbc:hive2://hiveserver2.bigdata.fr> insert overwrite table main_table ^M_fd_cae
. . . . . . . . . . . . . . . . . . . . . . .> select
. . . . . . . . . . . . . . . . . . . . . . .> po_cd as cae, line_po_cd as lcae, ^M origin_co_cd,
origin_line_co_cd
. . . . . . . . . . . . . . . . . . . . . . .> from dlk_scf_rn_purchase_order_li ^Mne
. . . . . . . . . . . . . . . . . . . . . . .> where instr(po_cd,"7")=1;
0: jdbc:hive2://hiveserver2.bigdata.fr>
0: jdbc:hive2://hiveserver2.bigdata.fr> insert overwrite table main_table ^M_fd_cai
. . . . . . . . . . . . . . . . . . . . . . .> select
. . . . . . . . . . . . . . . . . . . . . . .> po_cd as cai, line_po_cd as lcai, ^M origin_co_cd,
origin_line_co_cd
. . . . . . . . . . . . . . . . . . . . . . .> from dlk_scf_rn_purchase_order_li ^Mne
. . . . . . . . . . . . . . . . . . . . . . .> where instr(po_cd,"L")=1;
0: jdbc:hive2://hiveserver2.bigdata.fr>
0: jdbc:hive2://hiveserver2.bigdata.fr> insert overwrite table main_table ^M_fd_cce
. . . . . . . . . . . . . . . . . . . . . . .> select co_cd as cce,line_co_cd as ^M lcce,demand_order_ref1,
. . . . . . . . . . . . . . . . . . . . . . .> case when demand_order_ref2=''
. . . . . . . . . . . . . . . . . . . . . . .> then 1
. . . . . . . . . . . . . . . . . . . . . . .> else demand_order_ref2
. . . . . . . . . . . . . . . . . . . . . . .> end as demand_order_ref2
. . . . . . . . . . . . . . . . . . . . . . .> from dlk_scf_rn_customer_order_li ^Mne
. . . . . . . . . . . . . . . . . . . . . . .> where instr(co_cd,"I")=1;
Heart beat


<<< Invocation of Beeline command completed <<<

 Hadoop Job IDs executed by Beeline:

Intercepting System.exit(2)

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.Hive2Main], exit code [2]

Oozie Launcher failed, finishing Hadoop job gracefully
{noformat}

This error was present with Hadoop 2.7.2 and is still visible with Hadoop 2.7.3.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message