cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Ginchereau <jason...@microsoft.com>
Subject RE: [DISCUSS] Faster incremental builds
Date Tue, 26 Apr 2016 16:39:02 GMT
I don't understand how adding a file list to the hook parameters will help solve this problem.
Any hook that currently fails after an incremental prepare would need to be updated either
way. It would be simpler to just update it to correctly handle the case when a file wasn't
updated by prepare.

We still don't have any real examples of hooks where this is a problem, do we? But it still
seems safest to save this change for a major version update, unfortunately.

Jason


-----Original Message-----
From: Carlos Santana [mailto:csantana23@gmail.com] 
Sent: Friday, April 22, 2016 5:03 PM
To: dev@cordova.apache.org
Subject: Re: [DISCUSS] Faster incremental builds

Why your saying is more complicated?
Complicated for hook authors? 
for the hooks that don't need they don't have to do anything just ignore the argument in context.


"cordova prepare --clean" == "cordova clean && cordova prepare"



- Carlos
@csantanapr

> On Apr 22, 2016, at 7:45 PM, Jason Ginchereau <jasongin@microsoft.com> wrote:
> 
> I don't think making the after_prepare hook API more complicated is a good design. Instead
can we just document that after_prepare hooks should be designed to handle the files being
copied only sometimes? Most of them probably are anyway, because Cordova never made an explicit
guarantee about the files being fresh every time the hook is called. Anyway I guess it could
still be considered a breaking change so maybe it should trigger a major-version bump?
> 
> I don't understand what you are suggesting with "cordova prepare --clean".
> 
> -----Original Message-----
> From: Carlos Santana [mailto:csantana23@gmail.com]
> Sent: Thursday, April 21, 2016 17:16
> To: dev@cordova.apache.org
> Subject: Re: [DISCUSS] Faster incremental builds
> 
> I guess it's fine to have default to be faster, we just need to pass the files to be
copied to feed the hook to be aware. It's kind of a break API on CLI so maybe CLI 7.x but
not sure. 
> 
> Also for folks that want the conversation approach with one command we can also have
a flag --clean "cordova prepare --clean"
> 
> 
> - Carlos
> @csantanapr
> 
>> On Apr 21, 2016, at 7:42 PM, Jason Ginchereau <jasongin@microsoft.com> wrote:
>> 
>> Thanks Carlos, that's a good point.
>> 
>> The Ionic add_platform_class hook (https://github.com/driftyco/ionic-package-hooks)
actually would be fine; it doesn't do anything if the CSS class to be added is already there.
>> 
>> But I guess it's possible some other after_prepare hooks might not be as smart. I'm
not sure what to do about that.
>> 
>> 
>> -----Original Message-----
>> From: Carlos Santana [mailto:csantana23@gmail.com]
>> Sent: Thursday, April 21, 2016 2:25 PM
>> To: dev@cordova.apache.org
>> Subject: Re: [DISCUSS] Faster incremental builds
>> 
>> The only thing to watch out is folks modified files during after_prepare event, then
they will be broken if the file have not changed and they modified/add content to the file
a second time. 
>> 
>> To make this work the hook event will need to inform batch/node and plugins hooks
on before_prepare and after_prepare the list of files that are going to be copy, so the hook
owner can know not to modified the file if their file is not I the list. 
>> 
>> One real use case is Ionic and I think MobileFirst. 
>> For MobileForst I think we don't modified I think we just recreate 
>> new file ok every prepare
>> 
>> For Ionic they edit the index.html to a class to to the body element 
>> to for platform identification
>> 
>> - Carlos
>> @csantanapr
>> 
>>> On Apr 21, 2016, at 5:12 PM, Jason Ginchereau <jasongin@microsoft.com>
wrote:
>>> 
>>> If "cordova clean" would also take care of deleting the files copied by prepare,
then I'd be confident in making prepare incremental by default. Then if anyone was ever concerned
about whether the incremental prepare was doing the right thing, "cordova clean" would be
a fairly obvious way to reset the build. I still can't think of a realistic developer scenario
where that would be necessary, but it seems like the right thing for a "clean" command to
do regardless.
>>> 
>>> So I will work on enhancing the "clean" function for each platform at the same
time I enable the incremental prepare.
>>> 
>>> Jason
>>> 
>>> -----Original Message-----
>>> From: Jason Ginchereau [mailto:jasongin@microsoft.com]
>>> Sent: Wednesday, April 20, 2016 10:00 AM
>>> To: dev@cordova.apache.org
>>> Subject: RE: [DISCUSS] Faster incremental builds
>>> 
>>> My concern with making this the default behavior is that Cordova 
>>> prepare doesn't have a corresponding "clean" operation. While there 
>>> is a "cordova clean" command, it only does a clean for the 
>>> native-code project (e.g. gradle clean), which doesn't affect the 
>>> files copied during prepare. I guess this would only be a problem if 
>>> you did something to give your source files older last-modified 
>>> times... then a cordova prepare/build/run that was incremental would 
>>> not pick up the older files, and "cordova clean" wouldn't fix it. I 
>>> was thinking a git checkout could do that, but upon investigation it 
>>> actually doesn't reset files' last-modified times. (I guess any 
>>> source control tools shouldn't, as it would mess up any kind of 
>>> incremental build system.)
>>> 
>>> Maybe "cordova clean" should also delete the files that are copied by prepare?
Or maybe there could be something like a "cordova prepare --force" flag for the rare case
when a non-incremental prepare is needed? Between those two I'd prefer making "cordova clean"
more thorough; I didn't think of that until now.
>>> 
>>> Jason
>>> 
>>> -----Original Message-----
>>> From: Steven Gill [mailto:stevengill97@gmail.com]
>>> Sent: Wednesday, April 20, 2016 9:18 AM
>>> To: dev@cordova.apache.org
>>> Subject: Re: [DISCUSS] Faster incremental builds
>>> 
>>> Thanks for sharing the stats! Looks like a great improvement!
>>> 
>>> I'm fine with this going in sans flag if we test it  bit more first.
>>> 
>>> Have your tried it with the --browserify flag?
>>>> On Apr 20, 2016 7:18 AM, "Victor Sosa" <sosah.victor@gmail.com> wrote:
>>>> 
>>>> This is an amazing improvement in Cordova performance. Thanks Jason
>>>> +1!!
>>>> 
>>>>> On Wed, Apr 20, 2016 at 9:14 AM Shazron <shazron@gmail.com> wrote:
>>>>> 
>>>>> +1 what Jesse said
>>>>> 
>>>>>> On Tuesday, April 19, 2016, Jesse <purplecabbage@gmail.com>
wrote:
>>>>>> 
>>>>>> This sounds great!
>>>>>> Personally, I don't think this belongs behind a flag, why would 
>>>>>> anyone
>>>>> not
>>>>>> want to build faster?
>>>>>> I think we just need to test more to make sure it functions as 
>>>>>> expected everywhere, and then it should just make it's way in 
>>>>>> directly, without
>>>>> the
>>>>>> overhead of addition flag code, or documentation ...
>>>>>> 
>>>>>> 
>>>>>> @purplecabbage
>>>>>> risingj.com
>>>>>> 
>>>>>> On Tue, Apr 19, 2016 at 5:58 PM, Jason Ginchereau <
>>>>> jasongin@microsoft.com
>>>>>> <javascript:;>>
>>>>>> wrote:
>>>>>> 
>>>>>>> I collected data for 4 projects with varying numbers and sizes

>>>>>>> of
>>>>> source
>>>>>>> files (mostly .js and .png):
>>>>>>>  Tiny  :     17 files, 0.24 MB (A new Cordova project created
from
>>>>> the
>>>>>>> VS template)
>>>>>>>  Small :     55 files,  7.7 MB
>>>>>>>  Medium:    242 files,   44 MB
>>>>>>>  Large : 12,923 files,  134 MB
>>>>>>> 
>>>>>>> Following are full -> incremental times for 'cordova prepare
android'
>>>>>>> commands, without any source file changes in between, averaged

>>>>>>> over 5
>>>>>> warm
>>>>>>> runs on my medium-spec Windows desktop machine with SSD:
>>>>>>> 
>>>>>>>  Tiny  :  1.98 -> 1.84 s
>>>>>>>  Small :  2.40 -> 1.94 s
>>>>>>>  Medium:  4.22 -> 2.02 s
>>>>>>>  Large : 50.24 -> 5.81 s
>>>>>>> 
>>>>>>> As you can see, larger projects see greatly improved prepare

>>>>>>> times
>>>> when
>>>>>>> using the incremental option.
>>>>>>> 
>>>>>>> Also, I confirmed that without using the --incremental option

>>>>>>> with
>>>> this
>>>>>>> change, times are not measurably different compared to the 
>>>>>>> released
>>>>>> version
>>>>>>> 6.1.1 of Cordova. So there doesn't seem to be any regression

>>>>>>> compared
>>>>> to
>>>>>>> the previous use of shelljs.cp('-R') to blindly copy files when
>>>>>> preparing.
>>>>>>> 
>>>>>>> Jason
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Steven Gill [mailto:stevengill97@gmail.com <javascript:;>]
>>>>>>> Sent: Tuesday, April 19, 2016 3:14 PM
>>>>>>> To: dev@cordova.apache.org <javascript:;>
>>>>>>> Subject: Re: [DISCUSS] Faster incremental builds
>>>>>>> 
>>>>>>> Sounds like a worthy cause. Do you have any stats on how much

>>>>>>> time is saved?
>>>>>>> 
>>>>>>> Definitely put it behind the --incremental flag to start.
>>>>>>> 
>>>>>>> On Tue, Apr 19, 2016 at 2:43 PM, Jason Ginchereau <
>>>>>> jasongin@microsoft.com <javascript:;>>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> We've had a few customers complain that the dev inner loop
for
>>>>> Cordova
>>>>>>>> apps is slow compared to native app development. So recently

>>>>>>>> I've
>>>>> been
>>>>>>>> looking at ways to optimize it. The two largest pieces of
a 
>>>>>>>> Cordova build are "prepare" and "compile" phases. While there's

>>>>>>>> not much we can realistically do to speed up the native-code

>>>>>>>> compilation (in
>>>>> which
>>>>>>>> gradle/xcodebuild/etc is invoked), it's clear that the prepare
>>>> phase
>>>>>>>> is not nearly as efficient as it could be. So I opened a
JIRA 
>>>>>>>> for that, and I have a solution in the works that I want
to get
>>>> feedback
>>>>>> on.
>>>>>>>> 
>>>>>>>> CB-11117: Preparing platforms should skip copying files which
>>>> haven't
>>>>>>>> changed<
>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f
>>>>>>>> %2fissues.apache.org
>>>>> %2fjira%2fbrowse%2fCB-11117&data=01%7c01%7cjasongi
>>>>>>>> n%40microsoft.com
>>>>> %7c297f714a60a94c3f258108d3689ff306%7c72f988bf86f141a
>>>>> f91ab2d7cd011db47%7c1&sdata=E2Ww9LusEmVkUgjXPPGnxcGz%2fMH8fWruqGWU
>>>>> r
>>>>> L
>>>>> Xq
>>>>>>>> KRk%3d> Many cordova CLI commands include a "prepare"

>>>>>>>> operation, including 'cordova build', 'cordova run', 'cordova

>>>>>>>> plugin add', and more. Every time each of those commands
runs, 
>>>>>>>> the target platform
>>>> is
>>>>>>>> "prepared", which involves copying all files from 
>>>>>>>> [<project>/www, <project>/platforms/<platform>/platform_www,
>>>>>>>> <project>/merges/<platform>] to the platform's
target www 
>>>>>>>> folder,
>>>> as
>>>>>>>> well as copying a bunch of icons and splash screens to
>>>>>> platform-specific
>>>>>>> locations.
>>>>>>>> 
>>>>>>>> For the very first prepare of a platform, all that file copying

>>>>>>>> is necessary. But most of the time after that most of the
files 
>>>>>>>> being copied have not changed and therefore don't really
need 
>>>>>>>> to be
>>>> copied
>>>>>>>> again. So the typical developer inner loop (edit a few source
>>>> files,
>>>>>>>> build and run the app, repeat) is a lot slower than it could
be
>>>> for a
>>>>>>>> Cordova project, especially one that includes a significant

>>>>>>>> number
>>>> of
>>>>>>>> source files or resources.
>>>>>>>> 
>>>>>>>> Instead, Cordova should be smart enough to skip copying of

>>>>>>>> files
>>>> that
>>>>>>>> haven't changed, based on their last-modified timestamp.
But 
>>>>>>>> there should still be a way to force a 
>>>>>>>> clean/full/non-incremental copy if desired. To preserve 
>>>>>>>> compatibility with all possible existing workflows, I'm leaning

>>>>>>>> toward keeping the full copy as the default behavior, and

>>>>>>>> enabling the optimization only with a new
>>>> --incremental
>>>>>>>> flag, at least initially. If we find the incremental option
is
>>>>> working
>>>>>>>> well for everyone, we can promote that behavior to default
in a
>>>>> future
>>>>>>> major version update.
>>>>>>>> 
>>>>>>>> See PRs here for implementation of this idea...
>>>>>>>> https://github.com/apache/cordova-lib/pull/429: Add FileUpdater
>>>>> module
>>>>>>>> to cordova-common
>>>>>>>> https://github.com/apache/cordova-android/pull/295: Use
>>>> FileUpdater
>>>>> to
>>>>>>>> optimize prepare for android platform I'll submit PRs for
>>>> cordova-ios
>>>>>>>> and cordova-windows platforms soon, as well as a cordova-cli

>>>>>>>> change
>>>>> to
>>>>>>>> enable the optimization via a new --incremental flag (assuming

>>>>>>>> we don't want to make it the default behavior).
>>>>>>>> 
>>>>>>>> Jason
>>> 
>>> --------------------------------------------------------------------
>>> - To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>>> For additional commands, e-mail: dev-help@cordova.apache.org 
>>> ТÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒ
>>> ÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒÒ
>>> Ã 
>>> ’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã
>>> ’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã’Ã
>>> ’
>>> Ò
>>> ÐÐ¥Fò
>>> Vç7V'67&–&RÂRÖÖ–âFWb×Vç7V'67&–&
>>> T6÷&F÷fæ6†RæÃ
>>> ·&pФf÷"FF—F–öæ
>>> Â6öÖÖæG2ÂRÖÖ–âFWbֆVÃÂ
>>> ‡6÷&F÷fæ6†Ræ÷&pÐ
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
>> For additional commands, e-mail: dev-help@cordova.apache.org
>> 
>> B‹KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
>> KK
>> KCB•È[œÝXœØܚX™KK[XZ[
> ˆ]‹][œÝXœØܚX™PÛܙݘK˜\XÚK›Ü™ÃB‘›ÜˆY
>> ][ۘ[ÛÛ[X[™ËK[XZ[
> ˆ]‹Z[ÛܙݘK˜\XÚK›Ü™ÃB
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
> For additional commands, e-mail: dev-help@cordova.apache.org
> 
> B‹KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK
> KCB•È[œÝXœØܚX™KK[XZ[
ˆ]‹][œÝXœØܚX™PÛܙݘK˜\XÚK›Ü™ÃB‘›ÜˆY
> ][ۘ[ÛÛ[X[™ËK[XZ[
ˆ]‹Z[ÛܙݘK˜\XÚK›Ü™ÃB

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@cordova.apache.org
For additional commands, e-mail: dev-help@cordova.apache.org

Mime
View raw message