hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kartashov, Andy" <Andy.Kartas...@mpac.ca>
Subject Oozie apparent concurrency deadlocking
Date Thu, 15 Nov 2012 14:44:43 GMT

Have struggled for the last four days with this and still cannot find an answer even after
hours of searching the web.

I tried oozie workflow to execute my consecutive sqoop jobs in parallel.  I use forking that
executes 9 sqoop-action-nodes.

I had no problem executing the job on a pseudo-distributed cluster but with an added DN/TT
node I ran into (what seems like) deadlocking.  Oozie web interface displays those jobs as
"Running" indefinitely until I eventually kill the workflow.

What I did noticed wasd that if I was to reduce the number of sqoop-action-nodes to 3, all
works fine.

I found somewhere about oozie.service.CallableQueueService.callable.concurrency property to
be set by default to 3 and it hinted me that this must be it them. I tried to over-ride this
property by increasing this number to 5 in oozie-site.xml and restart oozie server and then
run 4 sqoop-action-nodes in fork but the result is the same. 2 out of 4 nodes execute successfully
(not in the same order every time) but the other 2 get hung in indefinite "Running...".

There were some suggestion about changing queue name from default but nothing was clear as
to what it change it to and where.

In case someone found a solution to this please do share. I will greatly appreciate it.

NOTICE: This e-mail message and any attachments are confidential, subject to copyright and
may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not
the intended recipient, please delete and contact the sender immediately. Please consider
the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe
qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts
par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite.
Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement
l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel

View raw message