atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nixon Rodrigues (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ATLAS-1567) Regression: AtlasHook for falcon seems to be broken, it is not invoked when a falcon cluster entity is submitted
Date Wed, 22 Feb 2017 15:06:44 GMT

     [ https://issues.apache.org/jira/browse/ATLAS-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nixon Rodrigues updated ATLAS-1567:
-----------------------------------
    Attachment: ATLAS-1567.patch

> Regression: AtlasHook for falcon seems to be broken, it is not invoked when a falcon
cluster entity is submitted
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-1567
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1567
>             Project: Atlas
>          Issue Type: Bug
>          Components: atlas-intg
>    Affects Versions: trunk
>            Reporter: Ayub Khan
>            Assignee: Nixon Rodrigues
>            Priority: Blocker
>             Fix For: trunk, 0.8-incubating
>
>         Attachments: ATLAS-1567.patch
>
>
> Falcon logs showing the cluster submitted successfully, but still the hook is not invoked
> {noformat}
> 2017-02-17 19:23:40,020 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:]
~ HttpServletRequest RemoteUser is hrt_qa (Servlets:47)
> 2017-02-17 19:23:40,020 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Logging in hrt_qa (CurrentUser:65)
> 2017-02-17 19:23:40,021 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Request from authenticated user: hrt_qa, URL=/api/entities/submit/cluster, doAs user: null
(FalconAuthenticationFilter:185)
> 2017-02-17 19:23:40,021 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Authorizing user=hrt_qa against request=RequestParts{resource='entities', action='submit',
entityType='cluster'} (FalconAuthorizationFilter:78)
> 2017-02-17 19:23:40,021 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Authorization for action=submit will be done in the API (DefaultAuthorizationProvider:315)
> 2017-02-17 19:23:40,021 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Authorization succeeded for user=hrt_qa, proxy=hrt_qa (FalconAuthorizationFilter:88)
> 2017-02-17 19:23:40,034 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Parsed Entity: cl1 (EntityParser:102)
> 2017-02-17 19:23:40,040 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Parsed Entity: cl1 (EntityParser:102)
> 2017-02-17 19:23:40,040 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Lock acquired for submit on (cluster) cl1 by 726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782
(MemoryLocks:54)
> 2017-02-17 19:23:40,040 ERROR - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Entity node (CLUSTER) cl1 not found in entity graph. (EntityGraph:71)
> 2017-02-17 19:23:40,041 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ Successfully released lock on (cluster) cl1 by 726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782
(MemoryLocks:70)
> 2017-02-17 19:23:40,041 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ All locks released on cl1 (AbstractEntityManager:489)
> 2017-02-17 19:23:40,050 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:hrt_qa:POST//entities/submit/cluster]
~ {Action:submit, Dimensions:{colo=NULL, entityType=cluster}, Status: SUCCEEDED, Time-taken:26995845
ns} (METRIC:38)
> 2017-02-17 19:23:40,051 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:]
~ Audit: hrt_qa@172.27.21.202 performed http://ctr-e129-1487033772569-4286-01-000004.hwx.site:15000/api/entities/submit/cluster
(172.27.17.129) at 2017-02-17T19:23Z (AUDIT:48)
> 2017-02-17 19:23:40,051 INFO  - [726486469@qtp-1828868503-10 - 39424028-a0e3-490f-ba04-1d025619e782:]
~ Audit: hrt_qa@172.27.21.202 performed http://ctr-e129-1487033772569-4286-01-000004.hwx.site:15000/api/entities/submit/cluster
(172.27.17.129) at 2017-02-17T19:23Z (AUDIT:48)
> 2017-02-17 19:23:51,101 WARN  - [HiveClientCache-cleaner-0:] ~ Evicted client has non-zero
user count: 1 (metastore:443)
> {noformat}
> Falcon running command - showing hook jars classpath
> {noformat}
> [root@ctr-e129-1487033772569-4286-01-000004 conf]# ps axu | grep -i falcon
> falcon     66767  0.2  0.2 9242152 751692 ?      Sl   Feb16   4:00 /usr/lib/jvm/java-openjdk/bin/java
-Xmx1024m -noverify -Dfalcon.embeddedmq=True -Dfalcon.emeddedmq.port=61616 -Dfalcon.log.dir=/grid/0/log/falcon
-Dfalcon.embeddedmq.data=/hadoop/falcon/embeddedmq/data -Dfalcon.home=/usr/hdp/current/falcon-server
-Dconfig.location=/usr/hdp/current/falcon-server/conf -Dfalcon.app.type=falcon -Dfalcon.catalog.service.enabled=
-cp /usr/hdp/current/falcon-server/conf:/usr/hdp/2.6.0.0-512/hadoop/conf:/usr/hdp/2.6.0.0-512/hadoop/lib/*:/usr/hdp/2.6.0.0-512/hadoop/.//*:/usr/hdp/2.6.0.0-512/hadoop-hdfs/./:/usr/hdp/2.6.0.0-512/hadoop-hdfs/lib/*:/usr/hdp/2.6.0.0-512/hadoop-hdfs/.//*:/usr/hdp/2.6.0.0-512/hadoop-yarn/lib/*:/usr/hdp/2.6.0.0-512/hadoop-yarn/.//*:/usr/hdp/2.6.0.0-512/hadoop-mapreduce/lib/*:/usr/hdp/2.6.0.0-512/hadoop-mapreduce/.//*::mysql-connector-java.jar:mysql-jdbc-qe.jar:/usr/hdp/2.6.0.0-512/tez/*:/usr/hdp/2.6.0.0-512/tez/lib/*:/usr/hdp/2.6.0.0-512/tez/conf:/usr/hdp/current/falcon-server/server/webapp/falcon/WEB-INF/classes:/usr/hdp/current/falcon-server/server/webapp/falcon/WEB-INF/lib/*:/usr/hdp/current/falcon-server/libext/*:/usr/hdp/current/atlas-client/hook/falcon/*:
org.apache.falcon.FalconServer -app /usr/hdp/current/falcon-server/server/webapp/falcon -port
15000
> root     1091491  0.0  0.0  10632   968 pts/0    S+   19:27   0:00 grep --color=auto
-i falcon
> {noformat}
> Below output shows that - the falcon-bridge jars are loaded.
> {noformat}
> [root@ctr-e129-1487033772569-4286-01-000004 conf]# lsof -p 66767 | grep jar | grep falcon-bridge
> java    66767 falcon  mem       REG              253,8    35338  134808586 /usr/hdp/2.6.0.0-512/atlas/hook/falcon/atlas-falcon-plugin-impl/falcon-bridge-0.8.0.2.6.0.0-512.jar
> java    66767 falcon  mem       REG              253,8    10294   67140617 /usr/hdp/2.6.0.0-512/atlas/hook/falcon/falcon-bridge-shim-0.8.0.2.6.0.0-512.jar
> java    66767 falcon  613r      REG              253,8    10294   67140617 /usr/hdp/2.6.0.0-512/atlas/hook/falcon/falcon-bridge-shim-0.8.0.2.6.0.0-512.jar
> java    66767 falcon  664r      REG              253,8    35338  134808586 /usr/hdp/2.6.0.0-512/atlas/hook/falcon/atlas-falcon-plugin-impl/falcon-bridge-0.8.0.2.6.0.0-512.jar
> [root@ctr-e129-1487033772569-4286-01-000004 conf]#
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message