hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaxon Hu <hujiaxu...@gmail.com>
Subject how to get info about which data in hdfs or file system that a MapReduce job visits?
Date Thu, 27 Jul 2017 06:27:32 GMT

I was trying to implement a Hadoop/Spark audit tool, but l met a problem
that I can’t get  the input file location and file name. I can get
username, IP address, time, user command, all of these info  from
hdfs-audit.log. But When I submit a MapReduce job, I can’t see input file
location  neither in Hadoop logs or Hadoop ResourceManager. Does hadoop
have API or log that contains these info through some configuration ?If it
have ,What should I configure?


View raw message