orc-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuanbo Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ORC-430) Support combining orc files in Hadoop streaming
Date Thu, 01 Nov 2018 07:19:00 GMT
Yuanbo Liu created ORC-430:
------------------------------

             Summary: Support combining orc files in Hadoop streaming
                 Key: ORC-430
                 URL: https://issues.apache.org/jira/browse/ORC-430
             Project: ORC
          Issue Type: Improvement
            Reporter: Yuanbo Liu


In the case of huge number of orc files, there seems no CombineOrcFile class existing  to decrease
mappers. When we use hadoop streaming in such case, hadoop cluster will apply a lot of mappers.
It would be great that we can combine a batch of  orc files into one mapper.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message