Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7D74A69B7 for ; Tue, 2 Aug 2011 10:36:19 +0000 (UTC) Received: (qmail 64439 invoked by uid 500); 2 Aug 2011 10:36:14 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 64387 invoked by uid 500); 2 Aug 2011 10:35:58 -0000 Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-user@hadoop.apache.org Delivered-To: mailing list common-user@hadoop.apache.org Received: (qmail 64369 invoked by uid 99); 2 Aug 2011 10:35:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Aug 2011 10:35:54 +0000 X-ASF-Spam-Status: No, hits=-1.6 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [192.6.10.2] (HELO colossus.hpl.hp.com) (192.6.10.2) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Aug 2011 10:35:43 +0000 Received: from localhost (localhost [127.0.0.1]) by colossus.hpl.hp.com (Postfix) with ESMTP id D92F41BA728 for ; Tue, 2 Aug 2011 11:35:22 +0100 (BST) X-Virus-Scanned: Debian amavisd-new at hpl.hp.com Received: from colossus.hpl.hp.com ([127.0.0.1]) by localhost (colossus.hpl.hp.com [127.0.0.1]) (amavisd-new, port 10024) with LMTP id sHbk9domAZ48 for ; Tue, 2 Aug 2011 11:35:22 +0100 (BST) Received: from 0-imap-br1.hpl.hp.com (0-imap-br1.hpl.hp.com [16.25.144.60]) by colossus.hpl.hp.com (Postfix) with ESMTP id 6DEAF1BA6AD for ; Tue, 2 Aug 2011 11:35:22 +0100 (BST) MailScanner-NULL-Check: 1312886104.81349@SekQabdUxeUoyrXnOR44hQ Received: from [16.25.175.86] (wildhaus.hpl.hp.com [16.25.175.86]) by 0-imap-br1.hpl.hp.com (8.14.1/8.13.4) with ESMTP id p72AZ4QG019633 for ; Tue, 2 Aug 2011 11:35:04 +0100 (BST) Message-ID: <4E37D2D8.4080303@apache.org> Date: Tue, 02 Aug 2011 11:35:04 +0100 From: Steve Loughran User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.18) Gecko/20110617 Thunderbird/3.1.11 MIME-Version: 1.0 To: common-user@hadoop.apache.org Subject: Re: The best architecture for EC2/Hadoop interface? References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-HPL-MailScanner-Information: Please contact the ISP for more information X-MailScanner-ID: p72AZ4QG019633 X-HPL-MailScanner: Found to be clean X-HPL-MailScanner-From: stevel@apache.org X-Virus-Checked: Checked by ClamAV on apache.org On 02/08/11 05:09, Mark Kerzner wrote: > Hi, > > I want to give my users a GUI that would allow them to start Hadoop clusters > and run applications that I will provide on the AMIs. What would be a good > approach to make it simple for the user? Should I write a Java Swing app > that will wrap around the EC2 commands? Should I use some more direct EC2 > API? Or should I use a web browser interface? > > My idea was to give the user a Java Swing GUI, so that he gives his Amazon > credentials to it, and it would be secure because the application is not > exposed to the outside. Does this approach make sense? 1. I'm not sure that Java Swing GUI makes sense for anything anymore -if it ever did. 2. Have a look at what other people have done first before writing your own. Amazon provide something for their derivative of Hadoop, Elastic MR, I suspect KarmaSphere and others may provide UIs in front of it too. the other thing is most big jobs are more than one operation, so you are a workflow world. Things like cascading pig and oozie help here, and if you can bring them up in-cluster, you can get a web UI.