spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Ability to auto-detect input data for datasources (by file extension).
Date Fri, 19 Feb 2016 02:25:55 GMT
Hi all,

I am planning to submit a PR for
https://issues.apache.org/jira/browse/SPARK-8000.

Currently, file format is not detected by the file extension unlike
compression codecs are being detected.

I am thinking of introducing another interface (a function) at
DataSourceRegister just like shortName() at in order to specify possible
file exceptions so that we can detect datasources by file extensions just
like Hadoop does for compression codecs.

Since adding an interface should be carefully done, I want to first ask if
this approach looks appropriate.

Could you please give me some feedback for this?


Thanks!

Mime
View raw message