hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-3595) Performance optimization for the connection cache of Phoenix timeline writer
Date Thu, 07 May 2015 21:14:59 GMT
Li Lu created YARN-3595:

             Summary: Performance optimization for the connection cache of Phoenix timeline
                 Key: YARN-3595
                 URL: https://issues.apache.org/jira/browse/YARN-3595
             Project: Hadoop YARN
          Issue Type: Sub-task
            Reporter: Li Lu
            Assignee: Li Lu

The story about the connection cache in Phoenix timeline storage is a little bit long. In
YARN-3033 we planned to have shared writer layer for all collectors in the same collector
manager. In this way we can better reuse the same heavy-weight storage layer connection, therefore
it's more friendly to conventional storage layer connections which are typically heavy-weight.

Phoenix, on the other hand, implements its own connection interface layer to be light-weight,
thread-unsafe. To make these connections work with our "multiple collector, single writer"
model, we're adding a thread indexed connection cache. However, many performance critical
factors are yet to be tested. 

In this JIRA we're tracing performance optimization efforts for this connection cache. At
the very beginning stage we may need to understand:

# If the current, thread-based indexing is an appropriate approach, or we can use some better
ways to index the connections. 
# the best size of the cache, presumably as the proposed default value of a configuration.

# how long we need to preserve a connection in the cache. 

Please feel free to add this list. 

This message was sent by Atlassian JIRA

View raw message