hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3706) Generalize native HBase writer for additional tables
Date Fri, 29 May 2015 06:05:17 GMT

     [ https://issues.apache.org/jira/browse/YARN-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Joep Rottinghuis updated YARN-3706:
    Attachment: YARN-3726-YARN-2928.002.patch

Attaching updated patch.
Less hackery in this patch. this should actually cleanly apply.

Ripped out stubbed out methods.
Left to do: 
- Fix remainder of unit test that is currently commented out
- moving EntityColum.readResult implementation into ColumnImpl and have EntityColumn delegate.
- Add readResult to interface
- Create similar readResult for the column prefix.
- Creating ColumnFactory to move EntityColumn 
- Clean up unneeded methods from TimelineWriterUtils
- Add additional unit tests for methods newly added to TimelineWriterUtils
- Create a reader method (some class) that reads an entire entity back
- Add unit test to read entire entity back and then compare all values to written entity.
That's pretty much the ultimate test and will ensure we have clean reader APIs.

This patch contains some additional handling of potential occurrences of separators (!) in
fields used for column names and fields (through separate value separator ?).

- At the end another perf test over a larger set to ensure that new code didn't regress in
Once this is all in place it should be relatively easy to insert column compression by adding
an additional prefix in the column name (I think).

> Generalize native HBase writer for additional tables
> ----------------------------------------------------
>                 Key: YARN-3706
>                 URL: https://issues.apache.org/jira/browse/YARN-3706
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Joep Rottinghuis
>            Assignee: Joep Rottinghuis
>            Priority: Minor
>         Attachments: YARN-3706-YARN-2928.001.patch, YARN-3726-YARN-2928.002.patch
> When reviewing YARN-3411 we noticed that we could change the class hierarchy a little
in order to accommodate additional tables easily.
> In order to get ready for benchmark testing we left the original layout in place, as
performance would not be impacted by the code hierarchy.
> Here is a separate jira to address the hierarchy.

This message was sent by Atlassian JIRA

View raw message