drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From paul-rogers <...@git.apache.org>
Subject [GitHub] drill issue #1014: DRILL-5771: Fix serDe errors for format plugins
Date Fri, 10 Nov 2017 01:40:10 GMT
Github user paul-rogers commented on the issue:

    https://github.com/apache/drill/pull/1014
  
    Not sure the description here is entirely correct. Let's separate two concepts: the plugin
(code) and the plugin definition (the stuff in JSON.)
    
    Plugin definitions are stored in ZK and retrieved by the Foreman. There may be some form
of race condition in the Foreman, but that's not my focus here.
    
    The plugin *definition* is read by the Forman and serialized into the physical plan. Each
worker reads the definition from the physical plan. For this reason, the worker's definition
can never be out of date: it is the definition used when serializing the plan.
    
    Further, Drill allows table functions which provide query-time name/value pair settings
for format plugin properties. The only way these can work is to be serialized along with the
query. So, the actual serialized format plugin definition, included with the query, includes
both the ZK information and the table function information.


---

Mime
View raw message