hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ray Navarette <>
Subject Resources/Distributed Cache on Spark
Date Thu, 08 Feb 2018 14:28:17 GMT

I'm hoping to find some information about using "ADD FILES <PATH>" when using the spark
execution engine.  I've seen some jira tickets reference this functionality, but little else.
 We have written some custom UDFs which require some external resources.  When using the MR
execution engine, we can reference the file paths using a relative path and they are properly
distributed and resolved.  When I try to do the same under spark engine, I receive an error
saying the file is unavailable.

Does "ADD FILES <PATH>" work on spark, and if so, how should I properly reference those
files in order to read them in the executors?

Thanks much for your help,

View raw message