spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gatorsmile <...@git.apache.org>
Subject [GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...
Date Thu, 08 Dec 2016 23:08:02 GMT
Github user gatorsmile commented on the issue:

    https://github.com/apache/spark/pull/14638
  
    In the original Hive JIRA of this feature: https://issues.apache.org/jira/browse/HIVE-5795

    
    See the latest reply from `Sergey Shelukhin`. 
    > This forces the entire input into a single split, which defeats the purpose of using
Hive in the first place - might as well run the analysis on a local machine. I would not recommend
anyone to use this feature except for experimentation. The headers/footers should be cleared
as part of an ETL process.
    
    I do not know how many Hive users are using them in the production system. This might
not be recommended in the Hive side. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message