giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <>
Subject [jira] [Created] (GIRAPH-571) Giraph on YARN could launch a job-local ZK instance from the AppMaster
Date Sat, 16 Mar 2013 19:24:13 GMT
Eli Reisman created GIRAPH-571:

             Summary: Giraph on YARN could launch a job-local ZK instance from the AppMaster
                 Key: GIRAPH-571
             Project: Giraph
          Issue Type: Improvement
          Components: zookeeper
    Affects Versions: 0.2.0
            Reporter: Eli Reisman

Once GIRAPH-13 is in, we can think differently about a lot of things if we choose too.

for one thing, we have had problems launching job-local ZK instances. We could (for YARN)
move that functionality to the App Master, having it launch a container just for ZK and populating
the Configuration's giraph.zkList setting so when the MRv1 ZK manager code sees the Conf,
it will think we already have a non-job-local ZK at zkList's host and port, and will just
connect instead of starting another local instance, making the whole affair transparent to
existing Giraph code.

Not important, but the YARN patch is currently defaulting to only execute jobs with a non-local
ZK instance already running, and giraph.zkList populated with its host:port.

Its quite possible when we get our MRv1 job local ZK working again, we can remove this and
it will work right out of the box, there's no reason it won't. But managing extraneous services
(especially those that hold up the job setup like launching a ZK) is what the YARN AppMaster
is all about anyway. i haven't been able to get our local ZK instance to launch outside of
test cases for a while now.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message