hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1978) Hive SymlinkTextInputFormat does not estimate input size correctly
Date Wed, 09 Feb 2011 06:36:57 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12992338#comment-12992338
] 

Namit Jain commented on HIVE-1978:
----------------------------------

Also, it might be simpler to add the new function 'getContentSummary' in all existing
input formats.

You can create a dummy class which all other hive input formats (other than symlinktextinputformat)
extend.
In the abstract dummy class, the existing defn. can be there.

	            FileSystem fs = p.getFileSystem(ctx.getConf());	          
                    cs = fs.getContentSummary(p);


That waym you dont need any special checking in Utilities.java - it calls getContentSummary(),
which is implemented by all input formats that hive supports.




> Hive SymlinkTextInputFormat does not estimate input size correctly
> ------------------------------------------------------------------
>
>                 Key: HIVE-1978
>                 URL: https://issues.apache.org/jira/browse/HIVE-1978
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: He Yongqiang
>            Assignee: He Yongqiang
>         Attachments: HIVE-1978.1.patch
>
>


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message