cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Charbonneau <amcha...@gmail.com>
Subject Re: Cordova Screen Capture Plugin for Android
Date Wed, 20 Mar 2013 17:32:29 GMT
"One more thing I noticed is that you're using mCaptureCount and mFileName
from different threads. You might run into errors if multiple calls are
happening at the same time. It'd be better to pass the fileName as a
function parameter, and maybe change the counter resetting logic to occur
only on the UI thread part."

-Ah yes good point, I will fix that.

"As for why it's choppier, is it because the throughput is now higher? If
you're not testing on a multi-core device, then the multi-threading might
have an adverse affect."

- Yes throughput is about 17%  higher when spawning from the ui thread, but
what happens due to the jerky framerate is multiple captures happen on a
single render frame, then the animation skips a bunch of frames and our
next capture happens then, causing very inconsistent results.  Spawning
from the core thread seems to yield much more consistent results where each
render frame is accompanied by a capture.  This is on a multi-core Nexus
10.

-Aaron


On Wed, Mar 20, 2013 at 7:15 AM, Andrew Grieve <agrieve@chromium.org> wrote:

> Glad you've found this useful. If you have any ideas for improving the
> docs, please feel free to make suggestions or file pull requests against
> the cordova-docs repo.
>
> One more thing I noticed is that you're using mCaptureCount and mFileName
> from different threads. You might run into errors if multiple calls are
> happening at the same time. It'd be better to pass the fileName as a
> function parameter, and maybe change the counter resetting logic to occur
> only on the UI thread part.
>
> As for why it's choppier, is it because the throughput is now higher? If
> you're not testing on a multi-core device, then the multi-threading might
> have an adverse affect.
>
>
> On Mon, Mar 18, 2013 at 7:08 PM, Aaron Charbonneau <amcharbo@gmail.com
> >wrote:
>
> > Ok thought this was all closed out, but I've hit a little snag with the
> > 'spawn background thread from ui thread'.
> >
> > For some reason for the throughput case, spawning the fileIO background
> > thread directly from the ui thread causes the ui to become much less
> fluid,
> > causing more jerking in a simple animation than spawning the fileIO
> thread
> > from the WebCore thread.  The code is almost identical between the 2
> > methods, so it seems odd this would be happening.  My only thought is
> that
> > the ui thread now has to hold on to some references of that background
> > thread?  Perplexing, and not easy to find info on the interwebs about
> this.
> >
> >
> > Any thoughts?
> >
> > -Aaron
> >
> >
> > On Mon, Mar 18, 2013 at 1:57 PM, Aaron Charbonneau <amcharbo@gmail.com
> > >wrote:
> >
> > > Cool thanks for explaining that, alot of the information for Cordova
> > > plugin dev is a little out-dated, makes it hard sometimes to 'do the
> > right
> > > thing.'
> > >
> > > Ok I've made all the changes:
> > >
> > >
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> > >
> > > I'll probably go forward with this approach once I've written some more
> > > tests to make sure there's no issues, and test out the additional
> > > functionality for throughput captures.
> > >
> > > Thanks again the help and feedback, was a very good learning
> experience.
> > >  Please let me know if you see any other issues!
> > >
> > > Thanks,
> > > Aaron
> > >
> > >
> > > On Mon, Mar 18, 2013 at 1:44 PM, Andrew Grieve <agrieve@chromium.org
> > >wrote:
> > >
> > >> On Mon, Mar 18, 2013 at 4:38 PM, Aaron Charbonneau <
> amcharbo@gmail.com
> > >> >wrote:
> > >>
> > >> > "You could avoid the blocking queue if you dispatch the background
> > >> thread's
> > >> > runnable at the end of the UI thread's block."
> > >> >
> > >> > -Great idea, done.
> > >> >
> > >> >
> > >> > I traced through to see what sending a NO_RESULT PluginResult with
> > >> > keepCallback==true does, and it looks like a no-op
> > >> > (NativeToJsMessageQueue.java, addPluginResult()):
> > >> >
> > >> >         boolean noResult = result.getStatus() ==
> > >> > PluginResult.Status.NO_RESULT.ordinal();
> > >> >         boolean keepCallback = result.getKeepCallback();
> > >> >         if (noResult && keepCallback) {
> > >> >             return;
> > >> >         }
> > >> >
> > >> > I'm sure I'm missing something here, is that really a no-op?
> > >>
> > >> Yep, that's a no-op and exists only to support the previous style of
> > >> Android plugins, where the execute method *had* to return a
> > PluginResult.
> > >>
> > >>
> > >>
> > >> > Did you
> > >> > mean for me to send a PluginResult.Status.OK with keepCallBack=true?
> > >> >
> > >> That's exactly what I meant :)
> > >>
> > >>
> > >> >
> > >> >
> > >> > Thanks,
> > >> > Aaron
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Sat, Mar 16, 2013 at 6:15 AM, Andrew Grieve <
> agrieve@chromium.org>
> > >> > wrote:
> > >> >
> > >> > > You could avoid the blocking queue if you dispatch the background
> > >> > thread's
> > >> > > runnable at the end of the UI thread's block.
> > >> > >
> > >> > > I see you send a success callback as soon as the image is
> captured,
> > >> but
> > >> > > before it is saved. This would improve throughput, but it might be
> > >> even
> > >> > > more useful if you set the keepCallback flag on it and then send
> > >> another
> > >> > > success response once you have the filename / comparison results.
> > It's
> > >> > the
> > >> > > filename that the app is probably interested in.
> > >> > >
> > >> > >
> > >> > > On Fri, Mar 15, 2013 at 9:41 PM, Aaron Charbonneau <
> > >> amcharbo@gmail.com
> > >> > > >wrote:
> > >> > >
> > >> > > > Ah thanks for clarifying that.
> > >> > > >
> > >> > > > I'm not sure how I can alleviate the ui thread any more than I
> am
> > >> > > > currently.  It is already doing the bare minimum amount of work
> on
> > >> the
> > >> > > > there.  Right now the only work done on the ui thread is
> > >> > > > view.capturePicture() which is the recommended thread for doing
> > that
> > >> > work
> > >> > > > (otherwise you get a warning):
> > >> > > >
> > >> > > >
> > >> > > > mUIThreadDone = false;
> > >> > > > cordova.getActivity().runOnUiThread(new Runnable() {
> > >> > > >          public void run() {
> > >> > > >                 CordovaWebView uiThreadView = webView;
> > >> > > >                 picture = uiThreadView.capturePicture();
> > >> > > >                 mUIThreadDone = true;
> > >> > > >             }
> > >> > > > });
> > >> > > > //rest of the work is done on core thread
> > >> > > > while(!mUIThreadDone) {}
> > >> > > > //write the picture to a file
> > >> > > > ...
> > >> > > >
> > >> > > > I could do the file io and compares on a different thread than
> the
> > >> > > > core one, but for the major use case, the core thread needs to
> > wait
> > >> no
> > >> > > > matter what to ensure that the file is written and the
> comparison
> > is
> > >> > > > complete, so I wouldn't expect much gain there.
> > >> > > >
> > >> > > >
> > >> > > > However, I've been experimenting with a more threaded version
> that
> > >> > > > does the file io and comparison on a background thread as you
> > >> > > > suggested.  If you have time please check it out. While the code
> > is
> > >> > > > not as clean, I think it addresses the issues you raised Andrew.
> > >>  For
> > >> > > > the major use case, performance is still the same as expected,
> but
> > >> if
> > >> > > > we want pure throughput on captures there's a big gain from
> about
> > 3
> > >> > > > captures per second (cps) to 10 cps on a Nexus 10.
> > >> > > >
> > >> > > >
> > >> > > > Here's the new plugin, with moar threading!
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCaptureMoarThreads.java
> > >> > > >
> > >> > > >
> > >> > > > Once again, thanks for all your feedback on this, I haven't had
> > much
> > >> > > > experience with threading in java before now.
> > >> > > >
> > >> > > >
> > >> > > > -Aaron
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Mar 14, 2013 at 8:10 AM, Andrew Grieve <
> > >> agrieve@chromium.org>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > My suggestion was just to prevent the UI thread from locking
> up
> > so
> > >> > that
> > >> > > > you
> > >> > > > > app doesn't appear to be frozen when the plugin is doing its
> > >> thing.
> > >> > > > >
> > >> > > > >
> > >> > > > > On Wed, Mar 13, 2013 at 5:03 PM, Aaron Charbonneau <
> > >> > amcharbo@gmail.com
> > >> > > > > >wrote:
> > >> > > > >
> > >> > > > > > Yah that's how I have it working now :) No problem with that
> > >> > approach
> > >> > > > for
> > >> > > > > > working with Jasmine.
> > >> > > > > >
> > >> > > > > > I made reference to running them asynchronously to respond
> to
> > >> > Andrew
> > >> > > > > about
> > >> > > > > > returning immediately after the bits are captured, which in
> > the
> > >> > case
> > >> > > of
> > >> > > > > > using that with Jasmine would only see a benefit IF we could
> > run
> > >> > > > multiple
> > >> > > > > > 'it' blocks at the same time.
> > >> > > > > >
> > >> > > > > > I think however Andrew was thinking of a different use case
> > >> where
> > >> > we
> > >> > > > > aren't
> > >> > > > > > limited by a framework that requires tests to run serially,
> > and
> > >> > > results
> > >> > > > > can
> > >> > > > > > just come back as they are ready, and be evaluated all
> > together
> > >> at
> > >> > > the
> > >> > > > > end,
> > >> > > > > > or not at all if you just want the captures and no
> evaluation.
> > >> > > > > >
> > >> > > > > > -Aaron
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Wed, Mar 13, 2013 at 1:53 PM, Braden Shepherdson <
> > >> > > > braden@chromium.org
> > >> > > > > > >wrote:
> > >> > > > > >
> > >> > > > > > > As far as I know, you're correct: there's no way to move
> on
> > >> the
> > >> > > next
> > >> > > > > it()
> > >> > > > > > > block before the first one is done.
> > >> > > > > > >
> > >> > > > > > > What's the problem with just making the tests wait? Why
> does
> > >> the
> > >> > > next
> > >> > > > > > it()
> > >> > > > > > > block need to start early? You can make your call to Java,
> > use
> > >> > > > waitFor
> > >> > > > > to
> > >> > > > > > > wait until the callback with the data is called, run your
> > >> > > > expectations,
> > >> > > > > > and
> > >> > > > > > > then move on to the next it() block. If the next it()
> block
> > is
> > >> > > using
> > >> > > > > the
> > >> > > > > > > result of this one, then they should be one it() block.
> > >> > > > > > >
> > >> > > > > > > Braden
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > On Wed, Mar 13, 2013 at 4:46 PM, Aaron Charbonneau <
> > >> > > > amcharbo@gmail.com
> > >> > > > > > > >wrote:
> > >> > > > > > >
> > >> > > > > > > > Hi Braden,
> > >> > > > > > > > Sorry maybe I am overlooking something.  The way I
> > >> understand
> > >> > it
> > >> > > is
> > >> > > > > > that
> > >> > > > > > > a
> > >> > > > > > > > previous 'it' block needs to complete before the next
> one
> > >> can
> > >> > > run.
> > >> > > > >  So
> > >> > > > > > I
> > >> > > > > > > > have various tests setup like this:
> > >> > > > > > > >
> > >> > > > > > > > it("renders something", function() {
> > >> > > > > > > > //draw something
> > >> > > > > > > > ....
> > >> > > > > > > > captureAndCompare(); //call the native function
> > >> > > > > > > >  waitsFor(function() {
> > >> > > > > > > > return captureComplete; //set by the native function
> > >> callback
> > >> > > > > > > > }, "capture never completed", 10000);
> > >> > > > > > > > runs(function() {
> > >> > > > > > > > expect(captureResult).toBe(0); //expects difference to
> be
> > 0
> > >> > > > > > > > });
> > >> > > > > > > > });
> > >> > > > > > > > //next 'it' test
> > >> > > > > > > > ...
> > >> > > > > > > >
> > >> > > > > > > > The expect block needs to wait until there is a result
> > >> returned
> > >> > > > from
> > >> > > > > > the
> > >> > > > > > > > native captureAndCompare call.  Therefore we use a
> > >> > waitsFor/runs
> > >> > > > > block
> > >> > > > > > > > there.
> > >> > > > > > > >
> > >> > > > > > > > But say we wanted to allow Javascript to run immediately
> > >> after
> > >> > > > > calling
> > >> > > > > > > that
> > >> > > > > > > > native function.  The only thing you need to wait for is
> > >> that
> > >> > > Java
> > >> > > > > > > grabbed
> > >> > > > > > > > the bits, and the rest of the processing will be done
> 'at
> > >> some
> > >> > > > point
> > >> > > > > in
> > >> > > > > > > the
> > >> > > > > > > > future.'  So as Andrew suggested you would return as
> soon
> > as
> > >> > the
> > >> > > > bits
> > >> > > > > > are
> > >> > > > > > > > captured, and allow JS to run again.  The problem is
> that
> > >> our
> > >> > > > expect
> > >> > > > > > > block
> > >> > > > > > > > needs the results, so that has to wait, the only other
> > >> thing to
> > >> > > do
> > >> > > > > > would
> > >> > > > > > > be
> > >> > > > > > > > to run the next 'it' block.
> > >> > > > > > > >
> > >> > > > > > > > This is the part I didn't think was possible, can the
> next
> > >> 'it'
> > >> > > > block
> > >> > > > > > be
> > >> > > > > > > > run while the previous 'it' is still waiting to execute
> > the
> > >> > > > 'expect'
> > >> > > > > > > block?
> > >> > > > > > > >  Or is there another approach all together that would
> > allow
> > >> > this?
> > >> > > > > > > >
> > >> > > > > > > > Thanks,
> > >> > > > > > > > Aaron
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > On Wed, Mar 13, 2013 at 1:03 PM, Braden Shepherdson <
> > >> > > > > > braden@chromium.org
> > >> > > > > > > > >wrote:
> > >> > > > > > > >
> > >> > > > > > > > > Jasmine does support asynchronous tests, and waiting
> an
> > >> > > arbitrary
> > >> > > > > > > period
> > >> > > > > > > > or
> > >> > > > > > > > > until some condition is true. Why does that not work
> for
> > >> > these
> > >> > > > > tests?
> > >> > > > > > > > >
> > >> > > > > > > > > Braden
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > On Wed, Mar 13, 2013 at 3:42 PM, Aaron Charbonneau <
> > >> > > > > > amcharbo@gmail.com
> > >> > > > > > > > > >wrote:
> > >> > > > > > > > >
> > >> > > > > > > > > > Quick follow up question for the return values
> stuff:
> > >> > > > > > > > > > Does it make sense to have my functions called from
> > >> execute
> > >> > > > > > > (capture()
> > >> > > > > > > > > and
> > >> > > > > > > > > > captureAndCompare()) return an error string if one
> is
> > >> > > > encountered
> > >> > > > > > and
> > >> > > > > > > > > pass
> > >> > > > > > > > > > that back in callbackContext.error, then have
> execute
> > >> > return
> > >> > > > > true?
> > >> > > > > > > > > >
> > >> > > > > > > > > > Or is there a preferred way to get errors from sub
> > >> > functions
> > >> > > > back
> > >> > > > > > > into
> > >> > > > > > > > > > Javascript?
> > >> > > > > > > > > >
> > >> > > > > > > > > > Thanks!
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau
> <
> > >> > > > > > > > amcharbo@gmail.com
> > >> > > > > > > > > > >wrote:
> > >> > > > > > > > > >
> > >> > > > > > > > > > > Thanks Andrew these are great suggestions!
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > About not needing the busy-wait in getScreenBits:
> > >> > > > > > > > > > > I'm not a fan of busy-waiting either.
> > >> > > > > > > > > > > Perhaps it's a side effect of Jasmine requiring
> > tests
> > >> to
> > >> > be
> > >> > > > run
> > >> > > > > > and
> > >> > > > > > > > > > > evaluated synchronously that I decided to use a
> > >> busy-wait
> > >> > > in
> > >> > > > > > > > > > getScreenBits.
> > >> > > > > > > > > > >  If the Runnable calls
> CallbackContext.success/error
> > >> once
> > >> > > the
> > >> > > > > > bits
> > >> > > > > > > > are
> > >> > > > > > > > > > > captured (but not written to file) on the
> UIThread,
> > >> the
> > >> > > > > > Javascript
> > >> > > > > > > > side
> > >> > > > > > > > > > > will proceed to run before it has an image
> > url/compare
> > >> > > > result,
> > >> > > > > > most
> > >> > > > > > > > > > likely
> > >> > > > > > > > > > > causing that test to fail.  As far as I know
> Jasmine
> > >> > > doesn't
> > >> > > > > > allow
> > >> > > > > > > > for
> > >> > > > > > > > > > > kicking off a bunch of tests and then gather all
> the
> > >> > > results
> > >> > > > at
> > >> > > > > > the
> > >> > > > > > > > end
> > >> > > > > > > > > > and
> > >> > > > > > > > > > > output pass/fails (each 'expect' function needs to
> > be
> > >> > > within
> > >> > > > an
> > >> > > > > > > 'it'
> > >> > > > > > > > > > block,
> > >> > > > > > > > > > > essentially that means one test must be evaluated
> > >> before
> > >> > > the
> > >> > > > > next
> > >> > > > > > > can
> > >> > > > > > > > > > > begin).
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > I can see the usefulness of allowing the capture,
> > file
> > >> > io,
> > >> > > > and
> > >> > > > > > > > > comparison
> > >> > > > > > > > > > > to be able to run asynchronously from the
> Javascript
> > >> > > though.
> > >> > > > In
> > >> > > > > > the
> > >> > > > > > > > > case
> > >> > > > > > > > > > > where you have your own test framework that CAN
> kick
> > >> off
> > >> > a
> > >> > > > > bunch
> > >> > > > > > of
> > >> > > > > > > > > tests
> > >> > > > > > > > > > > and just gather the results as they come in and
> do a
> > >> > batch
> > >> > > > > > > evaluation
> > >> > > > > > > > > at
> > >> > > > > > > > > > > the end, or in the case where you don't want to do
> > any
> > >> > > > > evaluation
> > >> > > > > > > in
> > >> > > > > > > > > your
> > >> > > > > > > > > > > app and just want to get captures as fast as you
> > can,
> > >> > > > > > asynchronous
> > >> > > > > > > > > could
> > >> > > > > > > > > > > increase performance there.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > I could try to implement it this way too, if you
> > think
> > >> > > > that's a
> > >> > > > > > > worth
> > >> > > > > > > > > > > while use case, maybe just provide an
> 'asynchronous'
> > >> flag
> > >> > > in
> > >> > > > > > > > > > CaptureOptions?
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > For doing the fileIO/compare using
> > >> cordova.getThreadPool,
> > >> > > is
> > >> > > > > > there
> > >> > > > > > > a
> > >> > > > > > > > > > > benefit to doing that if I still plan having the
> > >> > operation
> > >> > > > > block
> > >> > > > > > > the
> > >> > > > > > > > > > > Javascript?  For asynchronous mode I see why that
> > >> makes
> > >> > > > sense.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > For the return values, I'll make those changes
> right
> > >> > away!
> > >> > > > > >  Thanks
> > >> > > > > > > > > again
> > >> > > > > > > > > > > for providing your insight, hopefully I understood
> > it
> > >> > > > > correctly.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > -Aaron
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <
> > >> > > > > > > agrieve@chromium.org
> > >> > > > > > > > > > >wrote:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >> Had a glance at your code. Things you may want to
> > >> > address:
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> while(!mUIThreadDone) {}
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> You shouldn't need to busy-wait. Pass the
> > >> > CallbackContext
> > >> > > to
> > >> > > > > the
> > >> > > > > > > > > > Runnable,
> > >> > > > > > > > > > >> and call success/error on it whenever you're
> done.
> > No
> > >> > need
> > >> > > > to
> > >> > > > > > have
> > >> > > > > > > > the
> > >> > > > > > > > > > web
> > >> > > > > > > > > > >> thread wait for it. Even better would be to use a
> > >> second
> > >> > > > > > Runnable
> > >> > > > > > > to
> > >> > > > > > > > > do
> > >> > > > > > > > > > >> the
> > >> > > > > > > > > > >> image compare / file IO on a background thread
> via
> > >> > > > > > > > > > cordova.getThreadPool()
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> The return value to execute() should only be
> false
> > >> if an
> > >> > > > > invalid
> > >> > > > > > > > > command
> > >> > > > > > > > > > >> was passed. It's not meant to indicate the
> success
> > of
> > >> > the
> > >> > > > > > > operation.
> > >> > > > > > > > > You
> > >> > > > > > > > > > >> should be returning true for it.
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> Instead of return "success"/"", it might simplify
> > >> > things a
> > >> > > > bit
> > >> > > > > > to
> > >> > > > > > > > > > return a
> > >> > > > > > > > > > >> boolean of true/false
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <
> > >> > > > > > > > lorin.beer.dev@gmail.com
> > >> > > > > > > > > > >> >wrote:
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >> > all the same, very impressive and faster then I
> > >> > > expected.
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> > - Lorin
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron
> > Charbonneau
> > >> <
> > >> > > > > > > > > > amcharbo@gmail.com
> > >> > > > > > > > > > >> > >wrote:
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >> > > Thanks Lorin.
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > I did a quick test for throughput on an
> > animating
> > >> > > > canvas.
> > >> > > > >  I
> > >> > > > > > > was
> > >> > > > > > > > > > able
> > >> > > > > > > > > > >> to
> > >> > > > > > > > > > >> > > grab 30 frames within a 5 seconds which
> equates
> > >> to
> > >> > > > around
> > >> > > > > > > 166ms
> > >> > > > > > > > > per
> > >> > > > > > > > > > >> > capture
> > >> > > > > > > > > > >> > > on a Galaxy Nexus.  Unfortunately not fast
> > >> enough to
> > >> > > > > > automate
> > >> > > > > > > > > > >> something
> > >> > > > > > > > > > >> > > like video.  As with any kind of capturing
> > >> system,
> > >> > > often
> > >> > > > > the
> > >> > > > > > > > > > overhead
> > >> > > > > > > > > > >> of
> > >> > > > > > > > > > >> > > copying bits around is enough to slow down
> the
> > >> > > operation
> > >> > > > > of
> > >> > > > > > > the
> > >> > > > > > > > > app.
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > -Aaron
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer
> <
> > >> > > > > > > > > > >> lorin.beer.dev@gmail.com
> > >> > > > > > > > > > >> > > >wrote:
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> > > > Hey Aaron,
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > very cool stuff, looking forward to
> checking
> > it
> > >> > out!
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > Question: any performance hit on the app by
> > >> using
> > >> > > > > Capture?
> > >> > > > > > > > > What's
> > >> > > > > > > > > > >> the
> > >> > > > > > > > > > >> > > > expected throughput on images taken in this
> > way
> > >> > > > (given a
> > >> > > > > > > > > > particular
> > >> > > > > > > > > > >> > > device,
> > >> > > > > > > > > > >> > > > say Galaxy Nexus)?
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > - Lorin
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron
> > >> > Charbonneau
> > >> > > <
> > >> > > > > > > > > > >> > amcharbo@gmail.com
> > >> > > > > > > > > > >> > > > >wrote:
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > > > > Glad you like it :)
> > >> > > > > > > > > > >> > > > > Yes in fact the actual capture makes use
> of
> > >> > > > > > > > > > View.capturePicture()
> > >> > > > > > > > > > >> > which
> > >> > > > > > > > > > >> > > > > actually grabs the entire document, then
> > that
> > >> > can
> > >> > > be
> > >> > > > > > > clipped
> > >> > > > > > > > > > down
> > >> > > > > > > > > > >> to
> > >> > > > > > > > > > >> > > the
> > >> > > > > > > > > > >> > > > > size/location of a specific element.
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > > -Aaron
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal
> > >> Mocny <
> > >> > > > > > > > > > >> mmocny@chromium.org>
> > >> > > > > > > > > > >> > > > > wrote:
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > > > > Aaron,
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > I haven't even begun looking at your
> > >> > > > implementation,
> > >> > > > > > but
> > >> > > > > > > > I'm
> > >> > > > > > > > > > >> just
> > >> > > > > > > > > > >> > > going
> > >> > > > > > > > > > >> > > > > to
> > >> > > > > > > > > > >> > > > > > say it: this is awesome!
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > First question: When Capturing a DOM
> > >> element,
> > >> > > can
> > >> > > > > you
> > >> > > > > > > > > capture
> > >> > > > > > > > > > >> > 'body'
> > >> > > > > > > > > > >> > > to
> > >> > > > > > > > > > >> > > > > > grab it&children for a "full content
> > >> > > screenshot",
> > >> > > > or
> > >> > > > > > > does
> > >> > > > > > > > it
> > >> > > > > > > > > > >> have
> > >> > > > > > > > > > >> > to
> > >> > > > > > > > > > >> > > > be a
> > >> > > > > > > > > > >> > > > > > specific single element?
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > -Michal
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron
> > >> > > > Charbonneau <
> > >> > > > > > > > > > >> > > amcharbo@gmail.com
> > >> > > > > > > > > > >> > > > > > >wrote:
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > > > > Greetings!  My name is Aaron
> > Charbonneau,
> > >> > > happy
> > >> > > > to
> > >> > > > > > be
> > >> > > > > > > a
> > >> > > > > > > > > new
> > >> > > > > > > > > > >> > member
> > >> > > > > > > > > > >> > > of
> > >> > > > > > > > > > >> > > > > the
> > >> > > > > > > > > > >> > > > > > > mailing list!
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > I have been developing a sceenshot
> > plugin
> > >> > for
> > >> > > > > > Cordova
> > >> > > > > > > to
> > >> > > > > > > > > > help
> > >> > > > > > > > > > >> > > > > facilitate
> > >> > > > > > > > > > >> > > > > > > automation testing and debugging of
> > >> Cordova
> > >> > > > apps,
> > >> > > > > > and
> > >> > > > > > > I
> > >> > > > > > > > > > would
> > >> > > > > > > > > > >> > love
> > >> > > > > > > > > > >> > > > some
> > >> > > > > > > > > > >> > > > > > > feedback on it.  Currently Cordova
> > >> provides
> > >> > a
> > >> > > > > bunch
> > >> > > > > > of
> > >> > > > > > > > > > native
> > >> > > > > > > > > > >> > > > functions
> > >> > > > > > > > > > >> > > > > > > that allow you to do some cool stuff,
> > but
> > >> > not
> > >> > > > much
> > >> > > > > > > > > > >> functionality
> > >> > > > > > > > > > >> > to
> > >> > > > > > > > > > >> > > > > test
> > >> > > > > > > > > > >> > > > > > > the apps that make use of them.
>  Being
> > >> able
> > >> > to
> > >> > > > > take
> > >> > > > > > a
> > >> > > > > > > > > > capture
> > >> > > > > > > > > > >> of
> > >> > > > > > > > > > >> > > the
> > >> > > > > > > > > > >> > > > > > screen
> > >> > > > > > > > > > >> > > > > > > from within you app is a great way to
> > >> > automate
> > >> > > > > > testing
> > >> > > > > > > > or
> > >> > > > > > > > > > get
> > >> > > > > > > > > > >> > > > > additional
> > >> > > > > > > > > > >> > > > > > > information for debugging an issue.
> > >>  Since
> > >> > > there
> > >> > > > > is
> > >> > > > > > no
> > >> > > > > > > > > > >> > > > > > > Javascript mechanism for taking
> screen
> > >> > > captures
> > >> > > > > the
> > >> > > > > > > > > solution
> > >> > > > > > > > > > >> > would
> > >> > > > > > > > > > >> > > > have
> > >> > > > > > > > > > >> > > > > > to
> > >> > > > > > > > > > >> > > > > > > be native, which fits nicely into the
> > >> "gap"
> > >> > > that
> > >> > > > > > > > > > >> Cordova/Phonegap
> > >> > > > > > > > > > >> > > > > > bridges.
> > >> > > > > > > > > > >> > > > > > >  Any medium to large scale app can
> > >> benefit
> > >> > > > greatly
> > >> > > > > > > from
> > >> > > > > > > > > > >> > automation
> > >> > > > > > > > > > >> > > > > > testing
> > >> > > > > > > > > > >> > > > > > > and any app can benefit from an extra
> > >> > > debugging
> > >> > > > > > tool,
> > >> > > > > > > > and
> > >> > > > > > > > > > >> that is
> > >> > > > > > > > > > >> > > > what
> > >> > > > > > > > > > >> > > > > I
> > >> > > > > > > > > > >> > > > > > > hope this screenshot plugin can help
> > >> > achieve.
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Currently the plugin offers 2
> > functions:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Capture():
> > >> > > > > > > > > > >> > > > > > > * Take a capture of the current view,
> > >> write
> > >> > > that
> > >> > > > > > > capture
> > >> > > > > > > > > to
> > >> > > > > > > > > > a
> > >> > > > > > > > > > >> > .png
> > >> > > > > > > > > > >> > > > file
> > >> > > > > > > > > > >> > > > > > > with the specified file name and sub
> > >> > directory
> > >> > > > of
> > >> > > > > > the
> > >> > > > > > > > > sdcard
> > >> > > > > > > > > > >> > > > (fallback
> > >> > > > > > > > > > >> > > > > to
> > >> > > > > > > > > > >> > > > > > > emulated sdcard in the case there
> isn't
> > >> an
> > >> > > > sdcard
> > >> > > > > > > > mounted)
> > >> > > > > > > > > > >> > > > > > > * Able to create a sub-screenshot
> with
> > a
> > >> > > > specified
> > >> > > > > > > > > rectangle
> > >> > > > > > > > > > >> in
> > >> > > > > > > > > > >> > > order
> > >> > > > > > > > > > >> > > > > to
> > >> > > > > > > > > > >> > > > > > > block out ui elements that may be
> > >> variable,
> > >> > > and
> > >> > > > > also
> > >> > > > > > > > save
> > >> > > > > > > > > > >> space.
> > >> > > > > > > > > > >> > > > > > > * Can take captures of images/dom
> > >> elements
> > >> > > > > > (including
> > >> > > > > > > > > > canvas)
> > >> > > > > > > > > > >> > that
> > >> > > > > > > > > > >> > > > are
> > >> > > > > > > > > > >> > > > > > > lager than the actual screen size
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > CaptureAndCompare():
> > >> > > > > > > > > > >> > > > > > > * All the functionality of Capture()
> > >> > > > > > > > > > >> > > > > > > * Perform a comparison between the
> > >> captured
> > >> > > > image
> > >> > > > > > and
> > >> > > > > > > a
> > >> > > > > > > > > > >> baseline
> > >> > > > > > > > > > >> > > > image
> > >> > > > > > > > > > >> > > > > > > located at the specified location in
> > >> either
> > >> > > the
> > >> > > > > > assets
> > >> > > > > > > > > > folder
> > >> > > > > > > > > > >> or
> > >> > > > > > > > > > >> > > the
> > >> > > > > > > > > > >> > > > > > > sdcard.
> > >> > > > > > > > > > >> > > > > > > * User can specify per color channel
> > >> > > tolerances
> > >> > > > as
> > >> > > > > > > well
> > >> > > > > > > > as
> > >> > > > > > > > > > >> total
> > >> > > > > > > > > > >> > > > pixel
> > >> > > > > > > > > > >> > > > > > > tolerances to avoid false positives
> for
> > >> the
> > >> > > > > > inevitable
> > >> > > > > > > > > > >> rendering
> > >> > > > > > > > > > >> > > > > > > differences across devices.
> > >> > > > > > > > > > >> > > > > > > * Optionally output a png file that
> > >> contains
> > >> > > the
> > >> > > > > > > > > differences
> > >> > > > > > > > > > >> > > between
> > >> > > > > > > > > > >> > > > > the
> > >> > > > > > > > > > >> > > > > > > actual and the baseline for
> > >> debugging/triage
> > >> > > > > > purposes,
> > >> > > > > > > > two
> > >> > > > > > > > > > >> modes:
> > >> > > > > > > > > > >> > > > > binary
> > >> > > > > > > > > > >> > > > > > > diff (all failing pixels appear as
> > solid
> > >> > > white)
> > >> > > > or
> > >> > > > > > the
> > >> > > > > > > > > true
> > >> > > > > > > > > > >> > > > differences
> > >> > > > > > > > > > >> > > > > > > between pixels.
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > If you can spare some time, I would
> > love
> > >> it
> > >> > if
> > >> > > > you
> > >> > > > > > > could
> > >> > > > > > > > > > take
> > >> > > > > > > > > > >> a
> > >> > > > > > > > > > >> > > look
> > >> > > > > > > > > > >> > > > at
> > >> > > > > > > > > > >> > > > > > the
> > >> > > > > > > > > > >> > > > > > > api and parameters I have defined to
> > make
> > >> > sure
> > >> > > > > they
> > >> > > > > > > > adhere
> > >> > > > > > > > > > to
> > >> > > > > > > > > > >> > > Cordova
> > >> > > > > > > > > > >> > > > > > > plugin best practices.  The most
> > crucial
> > >> > part
> > >> > > > > would
> > >> > > > > > be
> > >> > > > > > > > in
> > >> > > > > > > > > > the
> > >> > > > > > > > > > >> > > plugin
> > >> > > > > > > > > > >> > > > > > itself
> > >> > > > > > > > > > >> > > > > > > at ScreenCapture.java:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
> > >> > > > > > > > > > >> > > > > > > and captureAndCompare() are the two
> > >> exposed
> > >> > > > > > functions)
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > I'm also interested to know you
> > thoughts
> > >> on
> > >> > > it's
> > >> > > > > > > usage.
> > >> > > > > > > >  I
> > >> > > > > > > > > > >> > started
> > >> > > > > > > > > > >> > > > with
> > >> > > > > > > > > > >> > > > > > the
> > >> > > > > > > > > > >> > > > > > > mobile-spec testing framework and put
> > >> some
> > >> > > quick
> > >> > > > > > > > rendering
> > >> > > > > > > > > > >> tests
> > >> > > > > > > > > > >> > > into
> > >> > > > > > > > > > >> > > > > the
> > >> > > > > > > > > > >> > > > > > > Autotest section as a viable use
> case.
> > >>  In
> > >> > > order
> > >> > > > > to
> > >> > > > > > > get
> > >> > > > > > > > > the
> > >> > > > > > > > > > >> > WebView
> > >> > > > > > > > > > >> > > > > into
> > >> > > > > > > > > > >> > > > > > > the state I wanted to capture, I had
> to
> > >> > > > implement
> > >> > > > > a
> > >> > > > > > > wait
> > >> > > > > > > > > > >> timer to
> > >> > > > > > > > > > >> > > > allow
> > >> > > > > > > > > > >> > > > > > the
> > >> > > > > > > > > > >> > > > > > > view to update before taking the
> > capture.
> > >> > >  Once
> > >> > > > > the
> > >> > > > > > > wait
> > >> > > > > > > > > > >> function
> > >> > > > > > > > > > >> > > > > > > completes, the capture can be taken
> and
> > >> > > > everything
> > >> > > > > > > from
> > >> > > > > > > > > > there
> > >> > > > > > > > > > >> is
> > >> > > > > > > > > > >> > > > > callback
> > >> > > > > > > > > > >> > > > > > > based.  I use the Jasmine
> > >> waitsFor()/runs()
> > >> > > > blocks
> > >> > > > > > to
> > >> > > > > > > > make
> > >> > > > > > > > > > the
> > >> > > > > > > > > > >> > > tests
> > >> > > > > > > > > > >> > > > > run
> > >> > > > > > > > > > >> > > > > > > synchronously.  For some validation
> > >> testing
> > >> > I
> > >> > > > > made a
> > >> > > > > > > > > compare
> > >> > > > > > > > > > >> > > function
> > >> > > > > > > > > > >> > > > > in
> > >> > > > > > > > > > >> > > > > > > Javascript to run against the native
> > >> > compare.
> > >> > > > >  Turns
> > >> > > > > > > out
> > >> > > > > > > > > > Java
> > >> > > > > > > > > > >> > runs
> > >> > > > > > > > > > >> > > > alot
> > >> > > > > > > > > > >> > > > > > > faster than Javascript (surprise!)
> thus
> > >> the
> > >> > > > native
> > >> > > > > > > > compare
> > >> > > > > > > > > > is
> > >> > > > > > > > > > >> > about
> > >> > > > > > > > > > >> > > > > 2-5x
> > >> > > > > > > > > > >> > > > > > > faster.  All in all the process is
> > fairly
> > >> > > quick,
> > >> > > > > the
> > >> > > > > > > > full
> > >> > > > > > > > > > >> > > > > > captureAndCompare
> > >> > > > > > > > > > >> > > > > > > with 3 file io's takes about 233ms
> on a
> > >> > Nexus
> > >> > > > One,
> > >> > > > > > and
> > >> > > > > > > > > > ~100ms
> > >> > > > > > > > > > >> on
> > >> > > > > > > > > > >> > a
> > >> > > > > > > > > > >> > > > > Nexus
> > >> > > > > > > > > > >> > > > > > 10
> > >> > > > > > > > > > >> > > > > > > (256x256 image size).
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Anyways here is the usage:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > And here's the JS wrapper for the
> > plugin
> > >> > > calls:
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Thanks for your time, and I look
> > forward
> > >> to
> > >> > > ANY
> > >> > > > > > > feedback
> > >> > > > > > > > > > >> positive
> > >> > > > > > > > > > >> > > or
> > >> > > > > > > > > > >> > > > > > > negative.
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > > > Thanks,
> > >> > > > > > > > > > >> > > > > > > Aaron
> > >> > > > > > > > > > >> > > > > > >
> > >> > > > > > > > > > >> > > > > >
> > >> > > > > > > > > > >> > > > >
> > >> > > > > > > > > > >> > > >
> > >> > > > > > > > > > >> > >
> > >> > > > > > > > > > >> >
> > >> > > > > > > > > > >>
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message