drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Laurent Goujon (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-4292) Avoid multiple new Configuration/FileSystem instantiation
Date Thu, 21 Jan 2016 01:07:39 GMT
Laurent Goujon created DRILL-4292:

             Summary: Avoid multiple new Configuration/FileSystem instantiation 
                 Key: DRILL-4292
                 URL: https://issues.apache.org/jira/browse/DRILL-4292
             Project: Apache Drill
          Issue Type: Task
            Reporter: Laurent Goujon

There are lots of places where Drill code has the following pattern:
conf = new Configuration();

    Configuration conf = new Configuration();
    fs = FileSystem.get(conf);

Creating Configuration instances is a pretty expensive operation as it triggers a classpath
scan to find resources (this lazily happens when accessing a key). Also, these extra instances
use more memory than expected (because the default resources are not shared between Configuration

FileSystem instances should not be expensive to create, because by default, instances are
cached (by scheme/authority/ugi), but it also means that the Configuration instance has little
value when getting the FileSystem instnce (except when getting the default filesystem (can
be replaced by FileSystem#get(URI, Configuration)) or creating the FileSystem for the given
scheme, IF not already in the cache).

If possible, all these examples should be refactored to inject the right FileSystem instance,
instead of letting each class trying to create its own.

This message was sent by Atlassian JIRA

View raw message