hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Sturlese <marc.sturl...@gmail.com>
Subject Good practices using a jar with hadoop jobs
Date Mon, 20 Jun 2011 11:25:07 GMT
Let's say I have a /home/me/foo.jar which contains a main that runs a hadoop
job and once it get completed launches another job (a pipeline of a couple
of jobs). The jar contains all the hadoop libs and other stuff needed too. I
launch it with hadoop jar /home/me/foo.jar.
If while the first job is running I do some changes to the code (that just
affect the second job, or neither of them) and upload the new compiled jar
to /home/me/foo.jar, once the job that was running finishes and the second
tries to start, everything gets broken. If I launch everything form the
beginning again, it works with no problems.
The thing is that I have this execution croned, so , every time I change
something, I have to stop the cron, wait for the execution to end, upload
the new jar and activate the cron again. This way I can avid the crash.
Are there any good practices about doing this kind of uploads?
Thanks in advance.

View this message in context: http://lucene.472066.n3.nabble.com/Good-practices-using-a-jar-with-hadoop-jobs-tp3085755p3085755.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

View raw message