Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 20203 invoked from network); 2 Oct 2008 21:49:44 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 2 Oct 2008 21:49:44 -0000 Received: (qmail 80394 invoked by uid 500); 2 Oct 2008 21:49:38 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 80181 invoked by uid 500); 2 Oct 2008 21:49:38 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 80170 invoked by uid 99); 2 Oct 2008 21:49:38 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Oct 2008 14:49:38 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [128.8.132.52] (HELO mrouter2.umiacs.umd.edu) (128.8.132.52) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Oct 2008 21:48:36 +0000 Received: from oliverqueen.umiacs.umd.edu (oliverqueen.umiacs.umd.edu [128.8.120.156]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mrouter2.umiacs.umd.edu (Postfix) with ESMTP id 5DA25345124; Thu, 2 Oct 2008 17:48:39 -0400 (EDT) Received: from webmail.umiacs.umd.edu (IDENT:U2FsdGVkX1/ACdInf5XzOjUJLTxcuzS6BZdYA0YxheA@localhost [127.0.0.1]) by oliverqueen.umiacs.umd.edu (8.13.1/8.13.1) with ESMTP id m92Lmd2a019714; Thu, 2 Oct 2008 17:48:39 -0400 Received: from 156.40.120.234 (SquirrelMail authenticated user jimmylin) by webmail.umiacs.umd.edu with HTTP; Thu, 2 Oct 2008 17:48:39 -0400 (EDT) Message-ID: <1917.156.40.120.234.1222984119.squirrel@webmail.umiacs.umd.edu> Date: Thu, 2 Oct 2008 17:48:39 -0400 (EDT) Subject: Lazily deserializing Writables From: "Jimmy Lin" To: core-user@hadoop.apache.org Cc: jimmylin@umd.edu Reply-To: jimmylin@umd.edu User-Agent: SquirrelMail/1.4.5 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Virus-Checked: Checked by ClamAV on apache.org Hi everyone, I'm wondering if it's possible to lazily deserialize a Writable. That is, when my custom Writable is handed a DataInput from readFields, can I simply hang on to the reference and read from it later? This would be useful if the Writable is a complex data structure that may be expensive to deserialize, so I'd only want to do it on-demand. Or does the runtime mutate the underlying stream, leaving the Writable with a reference to something completely different later? I'm wondering about both present behavior, and the implicit contract provided by the Hadoop API. Thanks! -Jimmy