Return-Path: X-Original-To: apmail-cordova-dev-archive@www.apache.org Delivered-To: apmail-cordova-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CD8D910261 for ; Thu, 12 Dec 2013 20:17:18 +0000 (UTC) Received: (qmail 52899 invoked by uid 500); 12 Dec 2013 20:17:18 -0000 Delivered-To: apmail-cordova-dev-archive@cordova.apache.org Received: (qmail 52871 invoked by uid 500); 12 Dec 2013 20:17:18 -0000 Mailing-List: contact dev-help@cordova.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cordova.apache.org Delivered-To: mailing list dev@cordova.apache.org Received: (qmail 52863 invoked by uid 99); 12 Dec 2013 20:17:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Dec 2013 20:17:18 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of shazron@gmail.com designates 209.85.212.182 as permitted sender) Received: from [209.85.212.182] (HELO mail-wi0-f182.google.com) (209.85.212.182) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 Dec 2013 20:17:14 +0000 Received: by mail-wi0-f182.google.com with SMTP id en1so127169wid.9 for ; Thu, 12 Dec 2013 12:16:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=15EkqQ7GXgOkA3bWrA+z3FU/O7dhc/+S282LFb7ulMo=; b=J5wYA6GZrKhsWvFPNEB/XlSsLakwRCG189NiD8c8oMx77v+fzgllGHs+fKEeK9gs8d jhjVi3017ynLpzRt7Y3Mh4NXeE6XD8+eZCmxy1yGJRG8MENCVB2ieJ8VCJ5tUQG35SzV 8llHivKuy62/WTcgsCvfw/+60b5fD/BQ2wBc/qeEVDEDVCH8Y73oBgnbpKTAzleK60G9 mIdsaaqwTRLDhZ7QdZvdMZIRGPfMVkpC8/Vcctl36Z4sftRYCbT4BJaGCvJzFVHReq+Q yH6XAfHg5KaVPUjfHyoOYKlCSFri8OZu+xDWs2hlRLgk0nLuwIFcK8QEqg6jJB9MYCb2 cxBg== X-Received: by 10.194.20.230 with SMTP id q6mr8335351wje.49.1386879412823; Thu, 12 Dec 2013 12:16:52 -0800 (PST) MIME-Version: 1.0 Received: by 10.194.201.69 with HTTP; Thu, 12 Dec 2013 12:16:12 -0800 (PST) In-Reply-To: References: <52866a86.e4b0320a.3292.ffffbd5f@mx.google.com> <52A74752.4000408@silverorange.com> <52A76DF9.7050705@silverorange.com> <52A78992.6040305@silverorange.com> From: Shazron Date: Thu, 12 Dec 2013 12:16:12 -0800 Message-ID: Subject: Re: Transitioning to a better File API implementation To: "dev@cordova.apache.org" Content-Type: multipart/alternative; boundary=047d7b5d971bbf6e4d04ed5c06f4 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5d971bbf6e4d04ed5c06f4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Reminds me of the iOS LocalStorage (CDVLocalStorage plugin) bs that needed to be handled - we definitely need a migration path (automatic hopefully) if not users are in for a world of hurt... On Wed, Dec 11, 2013 at 11:33 AM, Tommy Williams wrote= : > +1 > On 12/12/2013 6:26 am, "Ian Clelland" wrote: > > > Yeah, it would definitely require some kind of migration support. Not > > suggesting that this is something that we ever actually do, but we coul= d > > give developers a bit of code that automatically looks in both places, > and > > moves files to the new location on open. Or we do it under a flag that = is > > off for existing apps (off-on-upgrade) and only enabled-by-default for > new > > apps. > > > > I agree with you: this could cause worlds of pain, with developers, and > > worse with users. We shouldn't take any steps in that direction without= a > > lot of very careful thought. For now, /Documents, sub-optimal though it > may > > be, is where Cordova puts persistent files. > > > > Ian > > > > On Wed, Dec 11, 2013 at 2:18 PM, Tommy Williams > > wrote: > > > > > Changing where PERSISTENT is located sounds like a very very bad idea= . > > > > > > I know that would cause me personally a lot of grief with existing us= er > > > data. > > > On 11/12/2013 8:58 am, "Ian Clelland" wrote: > > > > > > > On Tue, Dec 10, 2013 at 4:37 PM, Michael Gauthier < > > mike@silverorange.com > > > > >wrote: > > > > > > > > > On 10/12/13 05:02 PM, Ian Clelland wrote: > > > > > > > > > >> Yep. > > > > >> > > > > >> I think that Library is the more natural place for the HTML > > persistent > > > > >> filesystem, since it's used by an app for whatever persistent > > storage > > > it > > > > >> needs, without exposing all of it's implementation details to th= e > > > user. > > > > >> (There's lots of room for debate on this, I'm sure) > > > > >> > > > > >> The trouble is that we've historically used /Documents for > > persistent > > > > >> storage, and changing that will break apps. > > > > >> > > > > >> I'm fine with a BC break, but I don't have to maintain any lega= cy > > > > > applications. /Library does make more sense as the default for > > > > PERSISTENT. > > > > > The big problem with BC is for installed apps with existing data = on > > the > > > > > filesystem, right? > > > > > > > > > > > > Exactly. I'd hate for developers to update their plugins, and push = a > > new > > > > version of their app that seems fine -- everything works great when > you > > > > start from scratch -- but users with existing data find themselves > > > > completely locked out of it. > > > > > > > > > > > > > > > > > > One idea is to allow something like requestFilesystem(DOCUMENT), > in > > > > >> addition to PERSISTENT and TEMPORARY. Another suggestion has bee= n > to > > > add > > > > >> extra arguments -- hints -- such as "{sync: true}", or maybe in > this > > > > case > > > > >> "{purpose: documents}" to specify the attributes of the filesyst= em > > > that > > > > is > > > > >> returned. > > > > >> > > > > > > > > > > > {sync: true} is a bit tricky because /Library/Cache is not synced > but > > > > > /Library/Application Data is synced. Having a DOCUMENT type would > > match > > > > > /tmp and /Library as the top-level dirs mapping to file-system > > > constants. > > > > > > > > > > All my feedback is only from the iOS perspective though. Not sure > if > > > > these > > > > > ideas make other platforms more or less complicated. > > > > > > > > > > Cheers, > > > > > Mike > > > > > > > > > > > > > > >> > > > > >> > > > > >> On Tue, Dec 10, 2013 at 2:39 PM, Michael Gauthier < > > > > mike@silverorange.com > > > > >> >wrote: > > > > >> > > > > >> Hmm.. The two directories have different defined roles on iOS a= nd > > > both > > > > >>> are > > > > >>> normal targets for applications to read/write files. Library is > for > > > > cache > > > > >>> data or app-specific resources. Documents is for saving/loading > > > actual > > > > >>> things created by users within apps. > > > > >>> > > > > >>> See https://developer.apple.com/library/ios/documentation/ > > > > >>> > > > > > > FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview= / > > > > >>> FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW14 > > > > >>> > > > > >>> Supporting both should be a goal. > > > > >>> > > > > >>> Cheers, > > > > >>> Mike > > > > >>> > > > > >>> > > > > >>> > > > > >>> On 2013-12-10 14:39, Ian Clelland wrote: > > > > >>> > > > > >>> If you could do that before, it was probably a bug :) (You > > shouldn't > > > > be > > > > >>>> able to use getParent on the filesystem root, and the native > code > > > was > > > > >>>> supposed to be checking for the correct path prefix before > > allowing > > > > >>>> access > > > > >>>> to any files on the device) > > > > >>>> > > > > >>>> At the moment, it's not possible to do that -- the filesystems > > > should > > > > be > > > > >>>> properly isolated, and only allow access to correctly formed > urls, > > > > like > > > > >>>> filesystem://localhost/persistent/some-file-here.txt. > > > > >>>> > > > > >>>> What we *can* do easily, though, is allow a new URL scheme for > > > library > > > > >>>> assets; something like filesystem://localhost/ > > > > >>>> library/some-file-here.txt, > > > > >>>> and we can have a filesystem handler for those URLs. That'll > work > > if > > > > >>>> your > > > > >>>> use case is just "I need access to files in /Library", rather > than > > > "I > > > > >>>> need > > > > >>>> to get to them via string manipulation". > > > > >>>> > > > > >>>> I've also had some discussions about making /Library the defau= lt > > > place > > > > >>>> for > > > > >>>> the persistent filesystem, and leaving /Documents either just > for > > > > legacy > > > > >>>> apps, or making *that* the target of the special URL scheme. > > That's > > > a > > > > >>>> proposal for a different day, though. There are some pretty bi= g > > > > >>>> backwards-compatibility issues there. > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> On Tue, Dec 10, 2013 at 11:54 AM, Michael Gauthier < > > > > >>>> mike@silverorange.com > > > > >>>> > > > > >>>>> wrote: > > > > >>>>> > > > > >>>> > > > > >>>> Ian, > > > > >>>> > > > > >>>>> > > > > >>>>> With the new URLs will it be possible to write to both > /Documents > > > and > > > > >>>>> /Library for iOS apps? In the old version, the filesystem roo= t > > > > resolved > > > > >>>>> as > > > > >>>>> /Documents but it was possible to get to /Library by navigati= ng > > the > > > > the > > > > >>>>> parent dir. > > > > >>>>> > > > > >>>>> Cheers, > > > > >>>>> Mike > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> On 2013-11-15 15:19, Ian Clelland wrote: > > > > >>>>> > > > > >>>>> On Fri, Nov 15, 2013 at 1:40 PM, purplecabbage < > > > > >>>>> purplecabbage@gmail.com > > > > >>>>> > > > > >>>>>> > > > > >>>>>>> wrote: > > > > >>>>>> > > > > >>>>>> Considering the magnitude of the changes I would have > > expected > > > > that > > > > >>>>>> this > > > > >>>>>> > > > > >>>>>> was just a new file plugin. The previous version was based > on a > > > > spec, > > > > >>>>>>> and > > > > >>>>>>> if we are deviating from it we should probably maintain bot= h, > > and > > > > >>>>>>> possibly > > > > >>>>>>> even make a recommendation to the w3c. > > > > >>>>>>> I hope we at least do a major version update for this if AP= Is > > > have > > > > >>>>>>> changed. > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> Yes, definitely a major version bump, at least so that de= vs > > > > aren't > > > > >>>>>>> > > > > >>>>>> caught > > > > >>>>>> by the URL-format-change. > > > > >>>>>> > > > > >>>>>> There aren't any changes to external APIs; I've been very > > careful > > > to > > > > >>>>>> maintain conformance with the existing tests, except where > those > > > > tests > > > > >>>>>> have > > > > >>>>>> been for undocumented implementation details. The only > > app-visible > > > > >>>>>> change > > > > >>>>>> is that URLs returned by the .toURL() method will look like > > > > >>>>>> > > > > >>>>>> filesystem://localhost/persistent/my/file.jpg > > > > >>>>>> > > > > >>>>>> rather than > > > > >>>>>> > > > > >>>>>> file:///var/mobile/Applications//Documents/my/file.j= pg > > > > >>>>>> > > > > >>>>>> We are still following the spec -- in fact, this brings us > > closer > > > in > > > > >>>>>> line > > > > >>>>>> with the W3C File API spec, on these points: > > > > >>>>>> > > > > >>>>>> http://www.w3.org/TR/file-system-api/#widl-Entry-fullPath -- > > The > > > > >>>>>> fullPath > > > > >>>>>> of an entry should be the path from the root (of the HTML > > > > filesystem, > > > > >>>>>> not > > > > >>>>>> the underlying device file system). > > > > >>>>>> > > > > >>>>>> > > http://www.w3.org/TR/file-system-api/#widl-Entry-toURL-DOMString-- > > > > >>>>>> This > > > > >>>>>> is > > > > >>>>>> a note, rather than a requirement, but a filesystem url sche= me > > is > > > > >>>>>> mentioned > > > > >>>>>> there. > > > > >>>>>> > > > > >>>>>> We're considering extending the spec in one way, which shoul= d > be > > > > >>>>>> compatible > > > > >>>>>> with the spirit of the spec, and backwards-compatible with t= he > > > > >>>>>> previous > > > > >>>>>> API. That is the ability to declare new filesystem types, > beyond > > > > >>>>>> "temporary" and "persistent", so that we can access things > like > > > > device > > > > >>>>>> media and app bundle assets using the same APIs that we use > for > > > > >>>>>> accessing > > > > >>>>>> user-defined files. If we're happy with the way that works, > then > > > we > > > > >>>>>> should > > > > >>>>>> definitely propose it for inclusion in the standard (not the > > > > specific > > > > >>>>>> filesystems, perhaps, but the mechanism for requesting and > > > > interacting > > > > >>>>>> with > > > > >>>>>> them) > > > > >>>>>> > > > > >>>>>> Ian > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Sent from my iPhone > > > > >>>>>> > > > > >>>>>>> > > > > >>>>>>> On Nov 15, 2013, at 8:36 AM, Ian Clelland < > > > > iclelland@chromium.org > > > > >>>>>>> > > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>>> wrote: > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> I've created CB-5403 as the =FCber-ticket for this; variou= s > bits > > > of > > > > it > > > > >>>>>>>> are > > > > >>>>>>>> > > > > >>>>>>>> in > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>>> subtasks. > > > > >>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> Once I have the tests and the JS updated, then platform > owners > > > can > > > > >>>>>>>> take > > > > >>>>>>>> a > > > > >>>>>>>> look and see whether they have anything to do to support > their > > > own > > > > >>>>>>>> > > > > >>>>>>>> schemes. > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> As general guidelines, I would say: > > > > >>>>>>>> > > > > >>>>>>>> * If you can intercept filesystem://* URLs in native code, > and > > > > >>>>>>>> return > > > > >>>>>>>> arbitrary responses, then do it! It will let your platform > > > support > > > > >>>>>>>> new > > > > >>>>>>>> filesystems in the future as we come up with them. Add a > > couple > > > of > > > > >>>>>>>> > > > > >>>>>>>> subtasks > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>>> for CB-5403 for your platform and go. > > > > >>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> * If you can't do that, but you *can* provide access to > things > > > > like > > > > >>>>>>>> media > > > > >>>>>>>> through special urls, then try that! You should be able to > > > create > > > > a > > > > >>>>>>>> FileSystem object that uses that URL as its root, and > > everything > > > > >>>>>>>> should > > > > >>>>>>>> work. Add a subtask for that, and see what you can do. > > > > >>>>>>>> > > > > >>>>>>>> * If you can't do that either, and you just want to stick > with > > > the > > > > >>>>>>>> > > > > >>>>>>>> file:/// > > > > >>>>>>>> > > > > >>>>>>> > > > > >>>>>>> urls that we are currently using, then don't do anything; > > > nothing > > > > >>>>>>> > > > > >>>>>>>> should > > > > >>>>>>>> change for you. > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> On Fri, Nov 15, 2013 at 10:36 AM, Ian Clelland < > > > > >>>>>>>> iclelland@chromium.org > > > > >>>>>>>> wrote: > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> On Fri, Nov 15, 2013 at 10:03 AM, Marcel Kinard < > > > > >>>>>>>> cmarcelk@gmail.com > > > > >>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> wrote: > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> Ian, will there be changes that app developers will ne= ed > > to > > > > do? > > > > >>>>>>>>> If > > > > >>>>>>>>> so, > > > > >>>>>>>>> > > > > >>>>>>>>> those should be clearly documented in a migration guide. > If > > > not, > > > > >>>>>>>>>> it > > > > >>>>>>>>>> > > > > >>>>>>>>>> sounds > > > > >>>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>> like the [improved] tests should pass on both the old a= nd > > new > > > > >>>>>>> > > > > >>>>>>>> versions, > > > > >>>>>>>> > > > > >>>>>>>> which would be sweet. > > > > >>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>> The filesystem URLs themselves will change as a result = of > > > this. > > > > >>>>>>>>> The > > > > >>>>>>>>> > > > > >>>>>>>>> tests > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>> should pass on both old and new versions, but the tests > mint > > > > their > > > > >>>>>>> own > > > > >>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> URLs > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>> for testing against -- each version is internally > consistent, > > > but > > > > >>>>>>> if > > > > >>>>>>> > > > > >>>>>>>> an > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> app > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>> is saving internal URLs and tries to dereference an old > > > > (file:///) > > > > >>>>>>> url > > > > >>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> with > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>> the new plugin, it will probably not resolve. > > > > >>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> Maybe there's a transition path here -- it might be > possible > > to > > > > >>>>>>>>> allow > > > > >>>>>>>>> window.resolveLocalFileSystemURL to access files with the > old > > > > URL, > > > > >>>>>>>>> but > > > > >>>>>>>>> never actually generate URLs. > > > > >>>>>>>>> > > > > >>>>>>>>> That would mean that > > > > >>>>>>>>> > > > > >>>>>>>>> (resolveLocalFileSystemURL(a)).toURL() !=3D=3D a, > > > > >>>>>>>>> > > > > >>>>>>>>> but I don't think that we depend on that as an identity. > > > > >>>>>>>>> > > > > >>>>>>>>> Is there work which needs to be done on the other platfor= ms > > > (BB, > > > > >>>>>>>>> WP8, > > > > >>>>>>>>> > > > > >>>>>>>>> Win8, FFOS, etc) so that those platforms don't get left > > > behind? > > > > >>>>>>>>> If > > > > >>>>>>>>> > > > > >>>>>>>>>> so, > > > > >>>>>>>>>> would it make sense to reach out to those platform owner= s > > and > > > at > > > > >>>>>>>>>> least > > > > >>>>>>>>>> > > > > >>>>>>>>>> get > > > > >>>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>> Jira items created with some notes? > > > > >>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>>> Yes. I'm going to create a ticket for this, and we can > add > > > > >>>>>>>>> platform-specific tasks to it. > > > > >>>>>>>>> > > > > >>>>>>>>> Other platforms shouldn't *have* to do anything, and some > > > > platforms > > > > >>>>>>>>> *can't* do anything, because they cannot access anything > > other > > > > than > > > > >>>>>>>>> file:/// urls anyway. > > > > >>>>>>>>> > > > > >>>>>>>>> However, if the other platforms want to get in on the > > goodness, > > > > and > > > > >>>>>>>>> > > > > >>>>>>>>> start > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>>>> > > > > >>>>>>> supporting their own URL schemes (I think the BB folks ca= n > > use > > > > >>>>>>> > > > > >>>>>>>> something > > > > >>>>>>>> > > > > >>>>>>>> like local:// for their filesystem access), then that'll = be > > the > > > > >>>>>>>>> place > > > > >>>>>>>>> to > > > > >>>>>>>>> keep everything organized. > > > > >>>>>>>>> > > > > >>>>>>>>> I'll post back once I have that issue set up. > > > > >>>>>>>>> > > > > >>>>>>>>> Ian > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> Sounds like you've built some very significant > > improvements. > > > > >>>>>>>>> Thanks! > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>>> On Nov 15, 2013, at 9:29 AM, Ian Clelland < > > > > >>>>>>>>>> iclelland@chromium.org > > > > >>>>>>>>>> > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>>> wrote: > > > > >>>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>> After working with the File plugin for a week or so, I'v= e > > > gotten > > > > >>>>>>>> it > > > > >>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>>>> more or > > > > >>>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>> less where I want it to be on iOS, and am working > through > > > > >>>>>>>>>> Android. > > > > >>>>>>>>>> > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> Looking > > > > >>>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>>> at the end result, though, I expect that over 90% of t= he > > > code > > > > >>>>>>>>>> has > > > > >>>>>>>>>> > > > > >>>>>>>>>>> been > > > > >>>>>>>>>>> touched in some way. (It's a huge diff.) > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> > > > > >>>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>> > > > > >>>>>> > > > > >>>>> > > > > >>>> > > > > >>> > > > > >> > > > > > > > > > > > > > > > --047d7b5d971bbf6e4d04ed5c06f4--