phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajeshbabu Chintaguntla (JIRA)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-1973) Improve CsvBulkLoadTool performance by moving keyvalue construction from map phase to reduce phase
Date Fri, 15 May 2015 05:11:00 GMT
Rajeshbabu Chintaguntla created PHOENIX-1973:
------------------------------------------------

             Summary: Improve CsvBulkLoadTool performance by moving keyvalue construction
from map phase to reduce phase
                 Key: PHOENIX-1973
                 URL: https://issues.apache.org/jira/browse/PHOENIX-1973
             Project: Phoenix
          Issue Type: Improvement
            Reporter: Rajeshbabu Chintaguntla
            Assignee: Rajeshbabu Chintaguntla
             Fix For: 5.0.0, 4.4.1


It's similar to HBASE-8768. Only thing is we need to write custom mapper and reducer in Phoenix.
In Map phase we just need to get row key from primary key columns and write the full text
of a line as usual(to ensure sorting). In reducer we need to get actual key values by running
upsert query.
It's basically reduces lot of map output to write to disk and data need to be transferred
through network.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message