cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Charbonneau <amcha...@gmail.com>
Subject Re: Cordova Screen Capture Plugin for Android
Date Wed, 13 Mar 2013 19:42:37 GMT
Quick follow up question for the return values stuff:
Does it make sense to have my functions called from execute (capture() and
captureAndCompare()) return an error string if one is encountered and pass
that back in callbackContext.error, then have execute return true?

Or is there a preferred way to get errors from sub functions back into
Javascript?

Thanks!


On Wed, Mar 13, 2013 at 12:14 PM, Aaron Charbonneau <amcharbo@gmail.com>wrote:

> Thanks Andrew these are great suggestions!
>
> About not needing the busy-wait in getScreenBits:
> I'm not a fan of busy-waiting either.
> Perhaps it's a side effect of Jasmine requiring tests to be run and
> evaluated synchronously that I decided to use a busy-wait in getScreenBits.
>  If the Runnable calls CallbackContext.success/error once the bits are
> captured (but not written to file) on the UIThread, the Javascript side
> will proceed to run before it has an image url/compare result, most likely
> causing that test to fail.  As far as I know Jasmine doesn't allow for
> kicking off a bunch of tests and then gather all the results at the end and
> output pass/fails (each 'expect' function needs to be within an 'it' block,
> essentially that means one test must be evaluated before the next can
> begin).
>
> I can see the usefulness of allowing the capture, file io, and comparison
> to be able to run asynchronously from the Javascript though. In the case
> where you have your own test framework that CAN kick off a bunch of tests
> and just gather the results as they come in and do a batch evaluation at
> the end, or in the case where you don't want to do any evaluation in your
> app and just want to get captures as fast as you can, asynchronous could
> increase performance there.
>
> I could try to implement it this way too, if you think that's a worth
> while use case, maybe just provide an 'asynchronous' flag in CaptureOptions?
>
> For doing the fileIO/compare using cordova.getThreadPool, is there a
> benefit to doing that if I still plan having the operation block the
> Javascript?  For asynchronous mode I see why that makes sense.
>
> For the return values, I'll make those changes right away!  Thanks again
> for providing your insight, hopefully I understood it correctly.
>
> -Aaron
>
>
> On Wed, Mar 13, 2013 at 8:04 AM, Andrew Grieve <agrieve@chromium.org>wrote:
>
>> Had a glance at your code. Things you may want to address:
>>
>> while(!mUIThreadDone) {}
>>
>> You shouldn't need to busy-wait. Pass the CallbackContext to the Runnable,
>> and call success/error on it whenever you're done. No need to have the web
>> thread wait for it. Even better would be to use a second Runnable to do
>> the
>> image compare / file IO on a background thread via cordova.getThreadPool()
>>
>> The return value to execute() should only be false if an invalid command
>> was passed. It's not meant to indicate the success of the operation. You
>> should be returning true for it.
>>
>> Instead of return "success"/"", it might simplify things a bit to return a
>> boolean of true/false
>>
>>
>> On Tue, Mar 12, 2013 at 4:51 PM, Lorin Beer <lorin.beer.dev@gmail.com
>> >wrote:
>>
>> > all the same, very impressive and faster then I expected.
>> >
>> > - Lorin
>> >
>> >
>> > On Tue, Mar 12, 2013 at 1:46 PM, Aaron Charbonneau <amcharbo@gmail.com
>> > >wrote:
>> >
>> > > Thanks Lorin.
>> > >
>> > > I did a quick test for throughput on an animating canvas.  I was able
>> to
>> > > grab 30 frames within a 5 seconds which equates to around 166ms per
>> > capture
>> > > on a Galaxy Nexus.  Unfortunately not fast enough to automate
>> something
>> > > like video.  As with any kind of capturing system, often the overhead
>> of
>> > > copying bits around is enough to slow down the operation of the app.
>> > >
>> > > -Aaron
>> > >
>> > >
>> > > On Tue, Mar 12, 2013 at 11:58 AM, Lorin Beer <
>> lorin.beer.dev@gmail.com
>> > > >wrote:
>> > >
>> > > > Hey Aaron,
>> > > >
>> > > > very cool stuff, looking forward to checking it out!
>> > > >
>> > > > Question: any performance hit on the app by using Capture? What's
>> the
>> > > > expected throughput on images taken in this way (given a particular
>> > > device,
>> > > > say Galaxy Nexus)?
>> > > >
>> > > > - Lorin
>> > > >
>> > > >
>> > > > On Tue, Mar 12, 2013 at 11:30 AM, Aaron Charbonneau <
>> > amcharbo@gmail.com
>> > > > >wrote:
>> > > >
>> > > > > Glad you like it :)
>> > > > > Yes in fact the actual capture makes use of View.capturePicture()
>> > which
>> > > > > actually grabs the entire document, then that can be clipped
down
>> to
>> > > the
>> > > > > size/location of a specific element.
>> > > > >
>> > > > > -Aaron
>> > > > >
>> > > > >
>> > > > > On Tue, Mar 12, 2013 at 11:27 AM, Michal Mocny <
>> mmocny@chromium.org>
>> > > > > wrote:
>> > > > >
>> > > > > > Aaron,
>> > > > > >
>> > > > > > I haven't even begun looking at your implementation, but
I'm
>> just
>> > > going
>> > > > > to
>> > > > > > say it: this is awesome!
>> > > > > >
>> > > > > > First question: When Capturing a DOM element, can you capture
>> > 'body'
>> > > to
>> > > > > > grab it&children for a "full content screenshot", or
does it
>> have
>> > to
>> > > > be a
>> > > > > > specific single element?
>> > > > > >
>> > > > > > -Michal
>> > > > > >
>> > > > > >
>> > > > > > On Tue, Mar 12, 2013 at 1:58 PM, Aaron Charbonneau <
>> > > amcharbo@gmail.com
>> > > > > > >wrote:
>> > > > > >
>> > > > > > > Greetings!  My name is Aaron Charbonneau, happy to
be a new
>> > member
>> > > of
>> > > > > the
>> > > > > > > mailing list!
>> > > > > > >
>> > > > > > > I have been developing a sceenshot plugin for Cordova
to help
>> > > > > facilitate
>> > > > > > > automation testing and debugging of Cordova apps, and
I would
>> > love
>> > > > some
>> > > > > > > feedback on it.  Currently Cordova provides a bunch
of native
>> > > > functions
>> > > > > > > that allow you to do some cool stuff, but not much
>> functionality
>> > to
>> > > > > test
>> > > > > > > the apps that make use of them.  Being able to take
a capture
>> of
>> > > the
>> > > > > > screen
>> > > > > > > from within you app is a great way to automate testing
or get
>> > > > > additional
>> > > > > > > information for debugging an issue.  Since there is
no
>> > > > > > > Javascript mechanism for taking screen captures the
solution
>> > would
>> > > > have
>> > > > > > to
>> > > > > > > be native, which fits nicely into the "gap" that
>> Cordova/Phonegap
>> > > > > > bridges.
>> > > > > > >  Any medium to large scale app can benefit greatly
from
>> > automation
>> > > > > > testing
>> > > > > > > and any app can benefit from an extra debugging tool,
and
>> that is
>> > > > what
>> > > > > I
>> > > > > > > hope this screenshot plugin can help achieve.
>> > > > > > >
>> > > > > > > Currently the plugin offers 2 functions:
>> > > > > > >
>> > > > > > > Capture():
>> > > > > > > * Take a capture of the current view, write that capture
to a
>> > .png
>> > > > file
>> > > > > > > with the specified file name and sub directory of the
sdcard
>> > > > (fallback
>> > > > > to
>> > > > > > > emulated sdcard in the case there isn't an sdcard mounted)
>> > > > > > > * Able to create a sub-screenshot with a specified
rectangle
>> in
>> > > order
>> > > > > to
>> > > > > > > block out ui elements that may be variable, and also
save
>> space.
>> > > > > > > * Can take captures of images/dom elements (including
canvas)
>> > that
>> > > > are
>> > > > > > > lager than the actual screen size
>> > > > > > >
>> > > > > > > CaptureAndCompare():
>> > > > > > > * All the functionality of Capture()
>> > > > > > > * Perform a comparison between the captured image and
a
>> baseline
>> > > > image
>> > > > > > > located at the specified location in either the assets
folder
>> or
>> > > the
>> > > > > > > sdcard.
>> > > > > > > * User can specify per color channel tolerances as
well as
>> total
>> > > > pixel
>> > > > > > > tolerances to avoid false positives for the inevitable
>> rendering
>> > > > > > > differences across devices.
>> > > > > > > * Optionally output a png file that contains the differences
>> > > between
>> > > > > the
>> > > > > > > actual and the baseline for debugging/triage purposes,
two
>> modes:
>> > > > > binary
>> > > > > > > diff (all failing pixels appear as solid white) or
the true
>> > > > differences
>> > > > > > > between pixels.
>> > > > > > >
>> > > > > > > If you can spare some time, I would love it if you
could take
>> a
>> > > look
>> > > > at
>> > > > > > the
>> > > > > > > api and parameters I have defined to make sure they
adhere to
>> > > Cordova
>> > > > > > > plugin best practices.  The most crucial part would
be in the
>> > > plugin
>> > > > > > itself
>> > > > > > > at ScreenCapture.java:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/src/org/apache/cordova/plugin/ScreenCapture.java(capture()
>> > > > > > > and captureAndCompare() are the two exposed functions)
>> > > > > > >
>> > > > > > > I'm also interested to know you thoughts on it's usage.
 I
>> > started
>> > > > with
>> > > > > > the
>> > > > > > > mobile-spec testing framework and put some quick rendering
>> tests
>> > > into
>> > > > > the
>> > > > > > > Autotest section as a viable use case.  In order to
get the
>> > WebView
>> > > > > into
>> > > > > > > the state I wanted to capture, I had to implement a
wait
>> timer to
>> > > > allow
>> > > > > > the
>> > > > > > > view to update before taking the capture.  Once the
wait
>> function
>> > > > > > > completes, the capture can be taken and everything
from there
>> is
>> > > > > callback
>> > > > > > > based.  I use the Jasmine waitsFor()/runs() blocks
to make the
>> > > tests
>> > > > > run
>> > > > > > > synchronously.  For some validation testing I made
a compare
>> > > function
>> > > > > in
>> > > > > > > Javascript to run against the native compare.  Turns
out Java
>> > runs
>> > > > alot
>> > > > > > > faster than Javascript (surprise!) thus the native
compare is
>> > about
>> > > > > 2-5x
>> > > > > > > faster.  All in all the process is fairly quick, the
full
>> > > > > > captureAndCompare
>> > > > > > > with 3 file io's takes about 233ms on a Nexus One,
and ~100ms
>> on
>> > a
>> > > > > Nexus
>> > > > > > 10
>> > > > > > > (256x256 image size).
>> > > > > > >
>> > > > > > > Anyways here is the usage:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/autotest/tests/rendering.tests.js
>> > > > > > >
>> > > > > > > And here's the JS wrapper for the plugin calls:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/Charbs09/Cordova-Mobile-Spec-ScreenCapture/blob/master/assets/www/screencapture.js
>> > > > > > >
>> > > > > > > Thanks for your time, and I look forward to ANY feedback
>> positive
>> > > or
>> > > > > > > negative.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > Aaron
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message