Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:from:to:in-reply-to:content-type:
	content-transfer-encoding:mime-version:subject:date:references:x-mailer;
	b=Txm3fJOXz5syUMFbcqIke8PIUZ3OY+77eUXgTWvqBCMySCyR0wOp1lcFP56HT9/0
Message-Id: <C59EF42F-16BD-41A5-9EFF-FB58491799A5@yahoo-inc.com>
From: Alan Gates <gates@yahoo-inc.com>
To: core-user@hadoop.apache.org
In-Reply-To: <4A295B03.2090205@cs.washington.edu>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: Hadoop scheduling question
Date: Fri, 5 Jun 2009 11:44:05 -0700
References: <4A295B03.2090205@cs.washington.edu>

To add a little context, Pig uses Hadoop's JobControl to schedule it's  
jobs.  Pig defines the dependencies between jobs in JobControl, and  
then submits the entire graph of jobs.  So, using JobControl, does  
Hadoop schedule jobs serially or in parallel (assuming no dependencies)?

Alan.

On Jun 5, 2009, at 10:50 AM, Kristi Morton wrote:

> Hi Pankil,
>
> Sorry about having to send my question email twice to the list...  
> the first time I sent it I had forgotten to subscribe to the list.   
> I resent it after subscribing, and your response to the first email  
> I sent did not make it into my inbox.  I saw your response on the  
> archives list.
>
> So, to recap, you said:
>
> "We are not able to carry out all joins in a single job..we also  
> tried our hadoop code using
> Pig scripts and found that for each join in PIG script new job is  
> used.So
> basically what i think its a sequential process to handle typesof  
> join where
> output of one job is required s an input to other one."
>
>
> I, too, have seen this sequential behavior with joins.  However, it  
> seems like it could be possible for there to be two jobs executing  
> in parallel whose output is the input to the subsequent job.  Is  
> this possible or are all jobs scheduled sequentially?
>
> Thanks,
> Kristi
>