phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tulasi P (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1711) Improve performance of CSV loader
Date Tue, 10 Mar 2015 21:01:38 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355704#comment-14355704
] 

Tulasi P commented on PHOENIX-1711:
-----------------------------------

checked-out version: +4.0.0-incubating-release+
{code}
$ git apply PHOENIX-1711_4.0.patch 
PHOENIX-1711_4.0.patch:19: trailing whitespace.
PHOENIX-1711_4.0.patch:123: trailing whitespace.
                    if (constantExpression.getDataType() != null &&  
PHOENIX-1711_4.0.patch:200: trailing whitespace.    
PHOENIX-1711_4.0.patch:204: trailing whitespace.    
PHOENIX-1711_4.0.patch:218: trailing whitespace.    
error: patch failed: phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java:83
error: phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java: patch does
not apply
error: patch failed: phoenix-core/src/main/java/org/apache/phoenix/expression/LiteralExpression.java:54
error: phoenix-core/src/main/java/org/apache/phoenix/expression/LiteralExpression.java: patch
does not apply
error: patch failed: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java:272
error: phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java: patch does
not apply
error: patch failed: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvToKeyValueMapper.java:28
error: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/CsvToKeyValueMapper.java: patch
does not apply
error: patch failed: phoenix-core/src/main/java/org/apache/phoenix/query/HTableFactory.java:48
error: phoenix-core/src/main/java/org/apache/phoenix/query/HTableFactory.java: patch does
not apply
error: patch failed: phoenix-core/src/main/java/org/apache/phoenix/util/csv/CsvUpsertExecutor.java:29
error: phoenix-core/src/main/java/org/apache/phoenix/util/csv/CsvUpsertExecutor.java: patch
does not apply
error: patch failed: phoenix-core/src/test/java/org/apache/phoenix/util/csv/CsvUpsertExecutorTest.java:17
error: phoenix-core/src/test/java/org/apache/phoenix/util/csv/CsvUpsertExecutorTest.java:
patch does not apply
{code}

> Improve performance of CSV loader
> ---------------------------------
>
>                 Key: PHOENIX-1711
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1711
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: James Taylor
>         Attachments: PHOENIX-1711.patch, PHOENIX-1711_4.0.patch
>
>
> Here is a break-up of percentage execution time for some of the steps inthe mapper:
> csvParser: 18%
> csvUpsertExecutor.execute(ImmutableList.of(csvRecord)): 39%
> PhoenixRuntime.getUncommittedDataIterator(conn, true): 9%
> ´╗┐while (uncommittedDataIterator.hasNext()): 15%
> Read IO & custom processing: 19%
> See details here: http://s.apache.org/6rl



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message