Mailing-List: contact hama-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hama-user@incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <37BC8C61-3545-4112-B980-0833AC6E3EBE@cs.ucsc.edu>
References: <37BC8C61-3545-4112-B980-0833AC6E3EBE@cs.ucsc.edu>
Date: Wed, 28 Mar 2012 07:51:35 +0900
Message-ID: 
 <CAGQgZQSZ2Qky1_ptE82-J+wf=E+4i2kgppsGY30Y9R8oONTVYQ@mail.gmail.com>
Subject: Re: multi-dimensional array storage
From: "Edward J. Yoon" <edwardyoon@apache.org>
To: hama-user@incubator.apache.org
Content-Type: text/plain; charset=UTF-8

Hi,

I believe that HBase is the best way to store multi-dimensional
arrays. HBase provides storage efficiencies as number of dimensions
grow, ordering capability, and also allows you to record and access
data corrections and updates directly via HBase client library.

Another option is use of SequenceFile and MapFile. Once data loaded to
the program initially, your math operations can run directly in memory
and and synchronized using a standard BSP APIs.

Thanks.

On Wed, Mar 28, 2012 at 12:46 AM, Noah Watkins <jayhawk@cs.ucsc.edu> wrote:
> Hi Hama list,
>
> I'm interested in using Hama to process large multi-dimensional arrays (sparse and dense). What is the best way to store and represent this type of data for processing in Hama?
>
> Thanks,
> Noah


-- 
Best Regards, Edward J. Yoon
@eddieyoon