Return-Path: X-Original-To: apmail-metamodel-dev-archive@minotaur.apache.org Delivered-To: apmail-metamodel-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0CCA018F42 for ; Wed, 10 Jun 2015 11:12:01 +0000 (UTC) Received: (qmail 27708 invoked by uid 500); 10 Jun 2015 11:12:00 -0000 Delivered-To: apmail-metamodel-dev-archive@metamodel.apache.org Received: (qmail 27678 invoked by uid 500); 10 Jun 2015 11:12:00 -0000 Mailing-List: contact dev-help@metamodel.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@metamodel.apache.org Delivered-To: mailing list dev@metamodel.apache.org Received: (qmail 27666 invoked by uid 99); 10 Jun 2015 11:12:00 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Jun 2015 11:12:00 +0000 Date: Wed, 10 Jun 2015 11:12:00 +0000 (UTC) From: =?utf-8?Q?Kasper_S=C3=B8rensen_=28JIRA=29?= To: dev@metamodel.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (METAMODEL-148) Add a HdfsResource implementation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/METAMODEL-148?page=3Dcom.atlas= sian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kasper S=C3=B8rensen resolved METAMODEL-148. --------------------------------------- Resolution: Fixed Fixed as of commit https://git-wip-us.apache.org/repos/asf?p=3Dmetamodel.gi= t;a=3Dcommit;h=3D6cf39add29cebf0787e81399ec6c3b9b150ff262 > Add a HdfsResource implementation > --------------------------------- > > Key: METAMODEL-148 > URL: https://issues.apache.org/jira/browse/METAMODEL-148 > Project: Apache MetaModel > Issue Type: New Feature > Reporter: Kasper S=C3=B8rensen > Assignee: Kasper S=C3=B8rensen > > I suggest to implement a Resource class that will allow reading and writi= ng files in Hadoop's HDFS file system. > Background: > Many of the file-based DataContext implementations we have accept a Resou= rce - an interface which abstracts the file system. We currently have imple= mentations like FileResource, UrlResource, ClasspathResource. > A request I get often is to also support Hadoop. Now obviously the ideal = Hadoop integration would not even imply using MetaModel's query-based appro= ach to data access, but for many simple use cases it is actually not that i= mportant whether the job runs natively in Hadoop (for example in Map-Reduce= ) or if the process simply fetches the file over the wire. I have seen many= cases of small-ish CSV files on Hadoop for example, where it would actuall= y be quicker to run through the file on a client than submitting a job to H= adoop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)