hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Borchard (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-18597) ExportSnapshot tool closes FileSystem objects obtained from the cache
Date Mon, 14 Aug 2017 18:16:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-18597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ben Borchard updated HBASE-18597:
    Summary: ExportSnapshot tool closes FileSystem objects obtained from the cache  (was:
ExportSnapshot tool closes FileSystems objects obtained from the cache)

> ExportSnapshot tool closes FileSystem objects obtained from the cache
> ---------------------------------------------------------------------
>                 Key: HBASE-18597
>                 URL: https://issues.apache.org/jira/browse/HBASE-18597
>             Project: HBase
>          Issue Type: Bug
>          Components: snapshots
>            Reporter: Ben Borchard
>              Labels: snapshot, snapshots
>   Original Estimate: 0.25h
>  Remaining Estimate: 0.25h
> In the ExportSnapshot tool the input and output FileSystem objects are being obtained
using the method _FileSystem.get_ which will either create a new FileSystem object or simply
take an existing object out of the cache if a suitable one exists.
> After it is done with the these FileSystem objects it will close them.
> The issue here is that the ExportSnapshot tool will potentially remove preexisting FileSystem
objects from the cache by closing said objects.  This will break any apis running in the same
process that are relying on these cached FileSystem objects.
> A simple solution could, I believe, fix this problem without affecting the functionality
of the ExportSnapshot tool:
> Change ExportSnapshot.java lines 943:
> {{FileSystem inputFs = FileSystem.get(inputRoot.toUri(), srcConf);}}
> and 947:
> {{FileSystem outputFs = FileSystem.get(outputRoot.toUri(), destConf);}}
> to use _FileSystem.newInstance_ instead of _FileSystem.get_:
> 943
> {{FileSystem inputFs = FileSystem.newInstance(inputRoot.toUri(), srcConf);}}
> 947
> {{FileSystem outputFs = FileSystem.newInstance(outputRoot.toUri(), destConf);}}
> This will create a unique entry in the cache and in this way prevent the closure of these
FileSystem objects from wiping out any preexisting FileSystem objects.  It will also ensure
that no unused FileSystem objects created by the ExportSnapshot tool will be left taking up
heap space and potentially causing memory issues.
> I am happy to submit a fix for this, but figured I would open an issue first so the issue
can be properly discussed and tracked.  
> Also note this is the first issue I have opened so I apologize in advance for any standard
procedures and/or best practices I haven't followed.

This message was sent by Atlassian JIRA

View raw message