Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Mon, 14 Mar 2016 18:03:33 +0000 (UTC)
From: "Keith Turner (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12949987.1457978590000.10651.1457978613683@Atlassian.JIRA>
In-Reply-To: <JIRA.12949987.1457978590000@Atlassian.JIRA>
References: <JIRA.12949987.1457978590000@Atlassian.JIRA>
 <JIRA.12949987.1457978590975@arcas>
Subject: [jira] [Created] (ACCUMULO-4165) Create a user level API for RFile
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Keith Turner created ACCUMULO-4165:
--------------------------------------

             Summary: Create a user level API for RFile
                 Key: ACCUMULO-4165
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4165
             Project: Accumulo
          Issue Type: Improvement
            Reporter: Keith Turner
             Fix For: 1.8.0


Users can bulk import RFiles.  Currently the only way users can create RFiles using Accumulo's public API is via AccumuloFileOutputFormat.  There is no way to read RFiles in the public API.   Also, the internal APIs for reading and writing RFiles are cumbersome to use.

I am experimenting with a simple RFile API like the following.  Below is an example of writing data.

{code:java}
    LocalFileSystem localFs = FileSystem.getLocal(new Configuration());
    RFileWriter writer = RFileFactory.newWriter()
                                       .withFileName("/tmp/test100M.rf")
                                       .withFileSystem(localFs).build();

    writer.startDefaultLocalityGroup();
    for (int r = 0; r < 10000000; r++) {
      for (int cq = 0; cq < 10; cq++) {
        writer.append(genKey(r, cq), genVal(r, cq));
      }
    }

    writer.close();
{code}

Below is an example of reading data.

{code:java}
    LocalFileSystem localFs = FileSystem.getLocal(new Configuration());
    Scanner scanner = RFileFactory.newScanner()
                                          .withFileName("/tmp/test100M.rf")
                                          .withFileSystem(localFs)
                                          .withDataCache(250000000)
                                          .withIndexCache(1000000).build();
{code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)