spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Bollin <>
Subject Where are my log4j or exception output on EMR?
Date Thu, 18 Jun 2015 00:24:47 GMT
I'm running Spark on Amazon EMR using their install bootstrap.

My Scala code had println("message here") statements - where can I find the output of these

I changed my code to use log4j - my and log.error output is nowhere to be found.

I've checked /mnt/var/log/hadoop/steps and I can see Spark output .. actually, it looks like
YARN output from the spark job - but I don't see my error or log or println statements anywhere.

I've also tried the bootstrap script argument --log-dir and pointed it to an S3 location -
it creates the directory but doesn't put any logfiles there.

Help! :)

Sent from my iPhone
View raw message