Return-Path: Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: (qmail 34106 invoked from network); 2 Oct 2010 21:16:37 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 2 Oct 2010 21:16:37 -0000 Received: (qmail 13839 invoked by uid 500); 2 Oct 2010 21:16:36 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 13679 invoked by uid 500); 2 Oct 2010 21:16:36 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 13671 invoked by uid 99); 2 Oct 2010 21:16:36 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Oct 2010 21:16:36 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of marcoscba@gmail.com designates 74.125.82.176 as permitted sender) Received: from [74.125.82.176] (HELO mail-wy0-f176.google.com) (74.125.82.176) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 02 Oct 2010 21:16:29 +0000 Received: by wye20 with SMTP id 20so1053225wye.35 for ; Sat, 02 Oct 2010 14:16:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=NYATcC8ORh8kyWOzVXWk+KoRGk/Qi2TTHNYNM7tAyS0=; b=EKo17oVsUfTWmf+5bNwBz7ona63hvOdU4ap56Vz3QAvF4OoE6KHh/ivoTGn5yfel5p P2Rp6/76CMR51E8s333qthaAWuvrhrih9tHuYP7V/6eX8aH/tvM8sWoHh6ZBuR4/188m OXqOM/cDO4mlgwakVlSsUEhbolnM1UKH5N7Nw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=mrAFsDCV6+FDT4UTGvrbEvHvZnaiiCv7NeK78B+jYEhbaEN4Z9NkAUku/SWZfujtub 2eCrBf290iZfs64Q8xV8a/bwO6Wg2lcBudwqC3N/8O+cr6/oE4uCJv3+KErgymXSKV4e Z00TbZLADUBgcaE+sYz1TkpmbWknjrmiuz1pc= MIME-Version: 1.0 Received: by 10.216.26.211 with SMTP id c61mr1139103wea.26.1286054168551; Sat, 02 Oct 2010 14:16:08 -0700 (PDT) Received: by 10.216.51.145 with HTTP; Sat, 2 Oct 2010 14:16:08 -0700 (PDT) In-Reply-To: References: Date: Sat, 2 Oct 2010 17:16:08 -0400 Message-ID: Subject: Re: Is it possible write directly to datanode's? From: Marcos Pinto To: general@hadoop.apache.org Content-Type: multipart/alternative; boundary=001636d34693e0e26e0491a8d163 --001636d34693e0e26e0491a8d163 Content-Type: text/plain; charset=ISO-8859-1 Thanks Guys, I misunderstood the picture in the hadoop site. Now I know that the namenode says in which datanode should be written and which part of file. So I dont need to worry about it. Thanks. On Sat, Oct 2, 2010 at 2:57 PM, Ryan Rawson wrote: > I suggest you read this: > http://hadoop.apache.org/common/docs/r0.20.0/hdfs_design.html > > -ryan > > On Fri, Oct 1, 2010 at 8:06 AM, Marcos Pinto wrote: > > Hi guys, I gotta a question and I really would appreciate some help, > > suggestion, or advices. > > Is it possible write directly to the datanode's and eliminate the > bottleneck > > to write into HDFS? > > for example: I gotta 40 writers and I writes th data direct to namenode, > i > > was wondering if it would become a bottleneck so I would like to know > > if it is possible to write the data direct to the datanode's. > > I've read that Hadoop doesnt support multiple writes so how can we deal > with > > big cluster? I mean I am really going to have this bottleneck??? > > > > Thanks in advanced. > > > --001636d34693e0e26e0491a8d163--