From issues-return-65519-archive-asf-public=cust-asf.ponee.io@drill.apache.org Fri May 3 16:05:02 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1F28F18064D for ; Fri, 3 May 2019 18:05:02 +0200 (CEST) Received: (qmail 98883 invoked by uid 500); 3 May 2019 16:05:01 -0000 Mailing-List: contact issues-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list issues@drill.apache.org Received: (qmail 98874 invoked by uid 99); 3 May 2019 16:05:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 May 2019 16:05:01 +0000 Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 69F94E2B80 for ; Fri, 3 May 2019 16:05:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 16E25256C1 for ; Fri, 3 May 2019 16:05:00 +0000 (UTC) Date: Fri, 3 May 2019 16:05:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (DRILL-7030) Make format plugins fully pluggable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/DRILL-7030?page=3Dcom.atlassian= .jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1683= 2623#comment-16832623 ]=20 ASF GitHub Bot commented on DRILL-7030: --------------------------------------- arina-ielchiieva commented on pull request #1780: DRILL-7030: Make format p= lugins fully pluggable URL: https://github.com/apache/drill/pull/1780#discussion_r280835496 =20 =20 ########## File path: exec/java-exec/src/main/java/org/apache/drill/exec/store/Storag= ePluginRegistryImpl.java ########## @@ -334,6 +331,48 @@ private StoragePlugins loadBootstrapPlugins(LogicalPl= anPersistence lpPersistence } } =20 + private void loadStoragePlugins(URL url, StoragePlugins bootstrapPlugins= , Map pluginURLMap, LogicalPlanPersistence lpPersistence) thro= ws IOException { + StoragePlugins plugins =3D getPluginsFromResource(url, lpPersistence); + plugins.forEach(plugin -> { + StoragePluginConfig oldPluginConfig =3D bootstrapPlugins.putIfAbsent= (plugin.getKey(), plugin.getValue()); + if (oldPluginConfig !=3D null) { + logger.warn("Duplicate plugin instance '{}' defined in [{}, {}], i= gnoring the later one.", + plugin.getKey(), pluginURLMap.get(plugin.getKey()), url); + } else { + pluginURLMap.put(plugin.getKey(), url); + } + }); + } + + private void loadFormatPlugins(URL url, StoragePlugins bootstrapPlugins,= Map pluginURLMap, LogicalPlanPersistence lpPersistence) throw= s IOException { =20 Review comment: Please add Javadoc. =20 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. =20 For queries about this service, please contact Infrastructure at: users@infra.apache.org > Make format plugins fully pluggable > ----------------------------------- > > Key: DRILL-7030 > URL: https://issues.apache.org/jira/browse/DRILL-7030 > Project: Apache Drill > Issue Type: Improvement > Affects Versions: 1.15.0 > Reporter: Arina Ielchiieva > Assignee: Anton Gozhiy > Priority: Major > Labels: doc-impacting > Fix For: 1.17.0 > > > Discussion on the mailing list -=C2=A0[https://lists.apache.org/thread.ht= ml/0c7de9c23ee9a8e18f8548ae0a323284cf1311b9570bd77ba544f63d@%3Cdev.drill.ap= ache.org%3E] > {noformat} > Before we were adding new formats / plugins into the exec module. Eventua= lly we came up to the point that exec package size is growing and adding pl= ugin and format contributions is better to separate out in the different mo= dule. > Now we have contrib module where we add such contributions. Plugins are p= luggable, there are added automatically by means of having drill-module.con= f file which points to the scanning packages. > Format plugins are using the same approach, the only problem is that they= are not added into bootstrap-storage-plugins.json. So when adding new form= at plugin, in order for it to automatically appear in Drill Web UI, develop= er has to update bootstrap file which is in the exec module. > My suggestion we implement some functionality that would merge format con= fig with the bootstrap one. For example, each plugin would have to have boo= tstrap-format.json file with the information to which plugin format should = be added (structure the same as in bootstrap-storage-plugins.json): > Example: > { > "storage":{ > dfs: { > formats: { > "psv" : { > type: "msgpack", > extensions: [ "mp" ] > } > } > } > } > } > Then during Drill start up such bootstrap-format.json files will be merge= d with bootstrap-storage-plugins.json.{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)