ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Veselovsky (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (IGNITE-3343) IGFS: Do not query secondary file system properties during create/append/mkdirs.
Date Thu, 28 Jul 2016 11:24:20 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397415#comment-15397415

Ivan Veselovsky commented on IGNITE-3343:

	at java.util.Collections$UnmodifiableList$1.set(Collections.java:1244)
	at java.util.Collections.sort(Collections.java:221)
	at org.apache.ignite.internal.processors.igfs.IgfsUtils.preparePathModes(IgfsUtils.java:835)
	at org.apache.ignite.internal.processors.igfs.IgfsModeResolver.<init>(IgfsModeResolver.java:65)
	at org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem.initialize(IgniteHadoopFileSystem.java:313)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2397)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2431)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2413)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)

> IGFS: Do not query secondary file system properties during create/append/mkdirs.
> --------------------------------------------------------------------------------
>                 Key: IGNITE-3343
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3343
>             Project: Ignite
>          Issue Type: Sub-task
>          Components: IGFS
>    Affects Versions: 1.6
>            Reporter: Vladimir Ozerov
>            Assignee: Ivan Veselovsky
>            Priority: Critical
>              Labels: performance
>             Fix For: 1.7
> Currently when we create something in a secondary file system, we perform additional
calls to the secondary file system to get file/directory info. This significantly slows down
structural operations, while usually it is not really needed in most cases.
> We should do the following:
> 1) Do not write modification time, access time and properties for DUAL entries. Instead,
we should propagate "info" and "listFiles" calls to secondary file system right away.
> 2) For {{create()}} we do not need length, as the file is either created from scratch,
or truncated.
> 3) For {{append()}} we need to know current length, so the second file system call appears
to be inevitable.

This message was sent by Atlassian JIRA

View raw message