Return-Path: X-Original-To: apmail-cordova-dev-archive@www.apache.org Delivered-To: apmail-cordova-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C2E1D10BCB for ; Tue, 10 Dec 2013 21:37:50 +0000 (UTC) Received: (qmail 89858 invoked by uid 500); 10 Dec 2013 21:37:50 -0000 Delivered-To: apmail-cordova-dev-archive@cordova.apache.org Received: (qmail 89836 invoked by uid 500); 10 Dec 2013 21:37:50 -0000 Mailing-List: contact dev-help@cordova.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cordova.apache.org Delivered-To: mailing list dev@cordova.apache.org Received: (qmail 89828 invoked by uid 99); 10 Dec 2013 21:37:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Dec 2013 21:37:50 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of mike@silverorange.com designates 24.222.7.12 as permitted sender) Received: from [24.222.7.12] (HELO mail.silverorange.com) (24.222.7.12) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Dec 2013 21:37:46 +0000 Received: from localhost (clementine.grove.silverorange.com [127.0.0.1]) by mail.silverorange.com (Postfix) with ESMTP id 3A3F97C03BC for ; Tue, 10 Dec 2013 17:37:24 -0400 (AST) X-Virus-Scanned: amavisd-new at silverorange.com X-Spam-Score: -3.061 X-Spam-Level: Received: from mail.silverorange.com ([192.168.0.12]) by localhost (clementine.grove.silverorange.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZXb1iWLZxoUa for ; Tue, 10 Dec 2013 17:37:23 -0400 (AST) Received: from [10.10.6.156] (tangerine.silverorange.com [24.222.7.25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.silverorange.com (Postfix) with ESMTPSA id DA2077C013E for ; Tue, 10 Dec 2013 17:37:22 -0400 (AST) Message-ID: <52A78992.6040305@silverorange.com> Date: Tue, 10 Dec 2013 17:37:22 -0400 From: Michael Gauthier User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: dev@cordova.apache.org Subject: Re: Transitioning to a better File API implementation References: <52866a86.e4b0320a.3292.ffffbd5f@mx.google.com> <52A74752.4000408@silverorange.com> <52A76DF9.7050705@silverorange.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Checked: Checked by ClamAV on apache.org X-Old-Spam-Flag: NO X-Old-Spam-Status: No, score=-3.061 tagged_above=-999999 required=3.5 tests=[ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.161] autolearn=ham On 10/12/13 05:02 PM, Ian Clelland wrote: > Yep. > > I think that Library is the more natural place for the HTML persistent > filesystem, since it's used by an app for whatever persistent storage it > needs, without exposing all of it's implementation details to the user. > (There's lots of room for debate on this, I'm sure) > > The trouble is that we've historically used /Documents for persistent > storage, and changing that will break apps. > I'm fine with a BC break, but I don't have to maintain any legacy applications. /Library does make more sense as the default for PERSISTENT. The big problem with BC is for installed apps with existing data on the filesystem, right? > One idea is to allow something like requestFilesystem(DOCUMENT), in > addition to PERSISTENT and TEMPORARY. Another suggestion has been to add > extra arguments -- hints -- such as "{sync: true}", or maybe in this case > "{purpose: documents}" to specify the attributes of the filesystem that is > returned. > {sync: true} is a bit tricky because /Library/Cache is not synced but /Library/Application Data is synced. Having a DOCUMENT type would match /tmp and /Library as the top-level dirs mapping to file-system constants. All my feedback is only from the iOS perspective though. Not sure if these ideas make other platforms more or less complicated. Cheers, Mike > > > > On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier wrote: > >> Hmm.. The two directories have different defined roles on iOS and both are >> normal targets for applications to read/write files. Library is for cache >> data or app-specific resources. Documents is for saving/loading actual >> things created by users within apps. >> >> See https://developer.apple.com/library/ios/documentation/ >> FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/ >> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14 >> >> Supporting both should be a goal. >> >> Cheers, >> Mike >> >> >> >> On 2013-12-10 14:39, Ian Clelland wrote: >> >>> If you could do that before, it was probably a bug :) (You shouldn't be >>> able to use getParent on the filesystem root, and the native code was >>> supposed to be checking for the correct path prefix before allowing access >>> to any files on the device) >>> >>> At the moment, it's not possible to do that -- the filesystems should be >>> properly isolated, and only allow access to correctly formed urls, like >>> filesystem://localhost/persistent/some-file-here.txt. >>> >>> What we *can* do easily, though, is allow a new URL scheme for library >>> assets; something like filesystem://localhost/library/some-file-here.txt, >>> and we can have a filesystem handler for those URLs. That'll work if your >>> use case is just "I need access to files in /Library", rather than "I need >>> to get to them via string manipulation". >>> >>> I've also had some discussions about making /Library the default place for >>> the persistent filesystem, and leaving /Documents either just for legacy >>> apps, or making *that* the target of the special URL scheme. That's a >>> proposal for a different day, though. There are some pretty big >>> backwards-compatibility issues there. >>> >>> >>> >>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier >>> wrote: >>> >>> Ian, >>>> >>>> With the new URLs will it be possible to write to both /Documents and >>>> /Library for iOS apps? In the old version, the filesystem root resolved >>>> as >>>> /Documents but it was possible to get to /Library by navigating the the >>>> parent dir. >>>> >>>> Cheers, >>>> Mike >>>> >>>> >>>> >>>> On 2013-11-15 15:19, Ian Clelland wrote: >>>> >>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage >>>>> >>>>> wrote: >>>>> >>>>> Considering the magnitude of the changes I would have expected that >>>>> this >>>>> >>>>>> was just a new file plugin. The previous version was based on a spec, >>>>>> and >>>>>> if we are deviating from it we should probably maintain both, and >>>>>> possibly >>>>>> even make a recommendation to the w3c. >>>>>> I hope we at least do a major version update for this if APIs have >>>>>> changed. >>>>>> >>>>>> >>>>>> Yes, definitely a major version bump, at least so that devs aren't >>>>> caught >>>>> by the URL-format-change. >>>>> >>>>> There aren't any changes to external APIs; I've been very careful to >>>>> maintain conformance with the existing tests, except where those tests >>>>> have >>>>> been for undocumented implementation details. The only app-visible >>>>> change >>>>> is that URLs returned by the .toURL() method will look like >>>>> >>>>> filesystem://localhost/persistent/my/file.jpg >>>>> >>>>> rather than >>>>> >>>>> file:///var/mobile/Applications//Documents/my/file.jpg >>>>> >>>>> We are still following the spec -- in fact, this brings us closer in >>>>> line >>>>> with the W3C File API spec, on these points: >>>>> >>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- The >>>>> fullPath >>>>> of an entry should be the path from the root (of the HTML filesystem, >>>>> not >>>>> the underlying device file system). >>>>> >>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString -- >>>>> This >>>>> is >>>>> a note, rather than a requirement, but a filesystem url scheme is >>>>> mentioned >>>>> there. >>>>> >>>>> We're considering extending the spec in one way, which should be >>>>> compatible >>>>> with the spirit of the spec, and backwards-compatible with the previous >>>>> API. That is the ability to declare new filesystem types, beyond >>>>> "temporary" and "persistent", so that we can access things like device >>>>> media and app bundle assets using the same APIs that we use for >>>>> accessing >>>>> user-defined files. If we're happy with the way that works, then we >>>>> should >>>>> definitely propose it for inclusion in the standard (not the specific >>>>> filesystems, perhaps, but the mechanism for requesting and interacting >>>>> with >>>>> them) >>>>> >>>>> Ian >>>>> >>>>> >>>>> >>>>> Sent from my iPhone >>>>>> >>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland >>>>>> >>>>>>> >>>>>>> wrote: >>>>>> >>>>>> >>>>>>> I've created CB-5403 as the über-ticket for this; various bits of it >>>>>>> are >>>>>>> >>>>>>> in >>>>>> >>>>>> subtasks. >>>>>>> >>>>>>> Once I have the tests and the JS updated, then platform owners can >>>>>>> take >>>>>>> a >>>>>>> look and see whether they have anything to do to support their own >>>>>>> >>>>>>> schemes. >>>>>> >>>>>> >>>>>>> As general guidelines, I would say: >>>>>>> >>>>>>> * If you can intercept filesystem://* URLs in native code, and return >>>>>>> arbitrary responses, then do it! It will let your platform support new >>>>>>> filesystems in the future as we come up with them. Add a couple of >>>>>>> >>>>>>> subtasks >>>>>> >>>>>> for CB-5403 for your platform and go. >>>>>>> >>>>>>> * If you can't do that, but you *can* provide access to things like >>>>>>> media >>>>>>> through special urls, then try that! You should be able to create a >>>>>>> FileSystem object that uses that URL as its root, and everything >>>>>>> should >>>>>>> work. Add a subtask for that, and see what you can do. >>>>>>> >>>>>>> * If you can't do that either, and you just want to stick with the >>>>>>> >>>>>>> file:/// >>>>>> >>>>>> urls that we are currently using, then don't do anything; nothing >>>>>>> should >>>>>>> change for you. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland < >>>>>>> iclelland@chromium.org >>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard >>>>>>> >>>>>>>> wrote: >>>>>>> >>>>>>> >>>>>>>> Ian, will there be changes that app developers will need to do? If >>>>>>>> so, >>>>>>>> >>>>>>>>> those should be clearly documented in a migration guide. If not, it >>>>>>>>> >>>>>>>>> sounds >>>>>>>> >>>>>>> >>>>>> like the [improved] tests should pass on both the old and new >>>>>>> versions, >>>>>>> >>>>>>>> which would be sweet. >>>>>>>>> >>>>>>>>> >>>>>>>> The filesystem URLs themselves will change as a result of this. The >>>>>>>> >>>>>>>> tests >>>>>>> >>>>>> >>>>>> should pass on both old and new versions, but the tests mint their own >>>>>>> >>>>>>>> >>>>>>>> URLs >>>>>>> >>>>>> >>>>>> for testing against -- each version is internally consistent, but if >>>>>>> an >>>>>>> >>>>>>>> >>>>>>>> app >>>>>>> >>>>>> >>>>>> is saving internal URLs and tries to dereference an old (file:///) url >>>>>>> >>>>>>>> >>>>>>>> with >>>>>>> >>>>>> >>>>>> the new plugin, it will probably not resolve. >>>>>>> >>>>>>>> >>>>>>>> Maybe there's a transition path here -- it might be possible to allow >>>>>>>> window.resolveLocalFileSystemURL to access files with the old URL, >>>>>>>> but >>>>>>>> never actually generate URLs. >>>>>>>> >>>>>>>> That would mean that >>>>>>>> >>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !== a, >>>>>>>> >>>>>>>> but I don't think that we depend on that as an identity. >>>>>>>> >>>>>>>> Is there work which needs to be done on the other platforms (BB, WP8, >>>>>>>> >>>>>>>> Win8, FFOS, etc) so that those platforms don't get left behind? If >>>>>>>>> so, >>>>>>>>> would it make sense to reach out to those platform owners and at >>>>>>>>> least >>>>>>>>> >>>>>>>>> get >>>>>>>> >>>>>>> >>>>>> Jira items created with some notes? >>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> Yes. I'm going to create a ticket for this, and we can add >>>>>>>> platform-specific tasks to it. >>>>>>>> >>>>>>>> Other platforms shouldn't *have* to do anything, and some platforms >>>>>>>> *can't* do anything, because they cannot access anything other than >>>>>>>> file:/// urls anyway. >>>>>>>> >>>>>>>> However, if the other platforms want to get in on the goodness, and >>>>>>>> >>>>>>>> start >>>>>>> >>>>>> >>>>>> supporting their own URL schemes (I think the BB folks can use >>>>>>> something >>>>>>> >>>>>>>> like local:// for their filesystem access), then that'll be the place >>>>>>>> to >>>>>>>> keep everything organized. >>>>>>>> >>>>>>>> I'll post back once I have that issue set up. >>>>>>>> >>>>>>>> Ian >>>>>>>> >>>>>>>> >>>>>>>> Sounds like you've built some very significant improvements. >>>>>>>> Thanks! >>>>>>>> >>>>>>>>> >>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> wrote: >>>>>>>>> >>>>>>>> >>>>>> >>>>>>> After working with the File plugin for a week or so, I've gotten it >>>>>>>>>> >>>>>>>>>> more or >>>>>>>>> >>>>>>>>> less where I want it to be on iOS, and am working through Android. >>>>>>>>>> >>>>>>>>>> Looking >>>>>>>>> >>>>>>>>> at the end result, though, I expect that over 90% of the code has >>>>>>>>>> been >>>>>>>>>> touched in some way. (It's a huge diff.) >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>> >> >