Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 15A291175F for ; Fri, 18 Jul 2014 17:11:36 +0000 (UTC) Received: (qmail 57802 invoked by uid 500); 18 Jul 2014 17:11:31 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 57669 invoked by uid 500); 18 Jul 2014 17:11:31 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 57659 invoked by uid 99); 18 Jul 2014 17:11:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Jul 2014 17:11:30 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rdhaase@gmail.com designates 209.85.213.182 as permitted sender) Received: from [209.85.213.182] (HELO mail-ig0-f182.google.com) (209.85.213.182) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Jul 2014 17:11:23 +0000 Received: by mail-ig0-f182.google.com with SMTP id c1so833959igq.3 for ; Fri, 18 Jul 2014 10:11:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=AuLHvYXSN8LDJI9c/66sD5qcMq+LRxZQiRKN4/Vqk1U=; b=Bp1FLhCQUbUFexLBRulRIfbLyG7hBOb3xF0PjqHso+HzHSyRm0aHLPdLHp+RgjWKgN N7JJm+KHTtcN1g/rkFSSk7B69/0D2NwKCp6qX1UP3TGqdVg5qY6G2Dgz9z4Xi1LbIAl/ Npeufeq5xQmGgvqAQQrO9CNHyVB/gazKvwBK9qxv7Z+NMuynW5JH2acKCw52o6/6yb3L DpUq4gmMx3nMs4JoZqSCjb8bXnruuSHndMjwXfEYfoiMw6j/DI1vymu9x9Yjg8u5oEeB 1W1JaW8r8QSQyF4zXPhP9fqgmgwW97aRPE/ujc2mWWsp12XN1PaVBc72kF1hPXAM7ZtW 918w== MIME-Version: 1.0 X-Received: by 10.50.124.194 with SMTP id mk2mr41159497igb.42.1405703463252; Fri, 18 Jul 2014 10:11:03 -0700 (PDT) Received: by 10.64.80.228 with HTTP; Fri, 18 Jul 2014 10:11:03 -0700 (PDT) In-Reply-To: References: Date: Fri, 18 Jul 2014 11:11:03 -0600 Message-ID: Subject: Re: MR JOB From: Rich Haase To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001a1134b660963e0604fe7ad70d X-Virus-Checked: Checked by ClamAV on apache.org --001a1134b660963e0604fe7ad70d Content-Type: text/plain; charset=UTF-8 HDFS handles the splitting of files into multiple blocks. It's a file system operation that is transparent to the user. On Fri, Jul 18, 2014 at 11:07 AM, Ashish Dobhal wrote: > Rich Haase Thanks, > But if the copy ops do not occur as a MR job then how does the splitting > of a file into several blocks takes place. > > > On Fri, Jul 18, 2014 at 10:24 PM, Rich Haase wrote: > >> File copy operations do not run as map reduce jobs. All hadoop fs >> commands are run as operations against HDFS and do not use the MapReduce. >> >> >> On Fri, Jul 18, 2014 at 10:49 AM, Ashish Dobhal < >> dobhalashish772@gmail.com> wrote: >> >>> Does the normal operations of hadoop such as uploading and downloading a >>> file into the HDFS run as a MR job. >>> If so why cant I see the job being run on my task tracker and job >>> tracker. >>> Thank you. >>> >> >> >> >> -- >> *Kernighan's Law* >> "Debugging is twice as hard as writing the code in the first place. >> Therefore, if you write the code as cleverly as possible, you are, by >> definition, not smart enough to debug it." >> > > -- *Kernighan's Law* "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." --001a1134b660963e0604fe7ad70d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
HDFS handles the splitting of files into multiple blocks. = =C2=A0It's a file system operation that is transparent to the user.=C2= =A0


On F= ri, Jul 18, 2014 at 11:07 AM, Ashish Dobhal <dobhalashish772@gma= il.com> wrote:
Rich Haase Thanks,
But = if the copy ops do not occur as a MR job then how does the splitting of a f= ile into several blocks takes place.

On Fri, Jul 18, 2014 at 10:24 PM, Rich Haase <rdhaase@gmail.com> wrote:
File copy operations do not run as map reduce jobs. =C2=A0= All hadoop fs commands are run as operations against HDFS and do not use th= e MapReduce.


On Fri, Jul 18, 2014 at 10:49 AM, Ashish Dobhal <dobhalashish772@g= mail.com> wrote:
Does the normal operations = of hadoop such as uploading and downloading a file into the HDFS run as a M= R job.
If so why cant I see the job being run on my task tracker and job tracker.<= /div>
Thank you.=C2=A0



<= font color=3D"#888888">--
Kernighan's Law
"Debugging= is twice as hard as writing the code in the first place.=C2=A0 Therefore, = if you write the code as cleverly as possible, you are, by definition, not = smart enough to debug it."




--
= Kernighan's Law
"Debugging is twice as hard as writing t= he code in the first place.=C2=A0 Therefore, if you write the code as cleve= rly as possible, you are, by definition, not smart enough to debug it."= ;
--001a1134b660963e0604fe7ad70d--