incubator-cvs mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <>
Subject [Incubator Wiki] Trivial Update of "SensSoftProposal" by LewisJohnMcgibbney
Date Mon, 23 May 2016 18:38:11 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Incubator Wiki" for change notification.

The "SensSoftProposal" page has been changed by LewisJohnMcgibbney:

  Any tool that people use to accomplish a task can be instrumented; once instrumented, those
tools can be used to report how they were used to perform that task. Software tools are ubiquitous
interfaces for people to interact with data and other technology that can be instrumented
for such a purpose. Tools are different than web pages or simple displays, however; they are
not simply archives for information. Rather, they are ways of interfacing with and manipulating
data and other technology. There are numerous consumer solutions for understanding how people
move through web pages and displays (e.g., Google Analytics, Adobe Omniture). There are far
fewer options for understanding how software tools are used. This requires understanding how
users integrate a tool’s functionality into usage strategies to perform tasks, how users
sequence the functionality provided them, and deeper knowledge of how users understand the
features of software as a cohesive tool. The Software as a Sensor™ Project is designed to
address this gap, providing the public an agile, cost-efficient solution for improving software
tool design, implementation, and usability.
  == Software as a Sensor™ Project Overview ==
- [[attachment:userale_figure_1.png]]
+ {{attachment:userale_figure_1.png||align="left"}}
  Figure 1. User ALE Elastic Back End Schema, with Transfer Protocols.
  Funded through the DARPA XDATA program and other sources, the Software as a Sensor™ Project
provides an open source (ALv2.0) solution for instrumenting software tools developed for the
web so that when users interact with it, their behavior is captured. User behavior, or user
activities, are captured and time-stamped through a simple application program interface (API)
called User Analytic Logging Engine (User ALE). User ALE’s key differentiator is the schema
that it uses to collect information about user activities; it provides sufficient context
to understand activities within the software tool’s overall functionality. User ALE captures
each user initiated action, or event (e.g., hover, click, etc.), as a nested action within
a specific element (e.g., map object, drop down item, etc.), which are in turn nested within
element groups (e.g., map, drop down list) (see Figure 1). This information schema provides
sufficient context to understand and disambiguate user events from one another. In turn, this
enables myriad analysis possibilities at different levels of tool design and more utility
to end-user than commercial services currently offer. 
  Once instrumented with User ALE, software tools become human signal sensors in their own
right. Most importantly, the data that User ALE collects is owned outright by adopters and
can be made available to other processes through scalable Elastic infrastructure and easy-to-manage
Restful APIs. 
  Distill is the analytic framework of the Software as a Sensor™ Project, providing (at
release) segmentation and graph analysis metrics describing users’ interactions with the
application to adopters. The segmentation features allow adopters to focus their analyses
of user activity data based on desired data attributes (e.g., certain interactions, elements,
etc.), as well as attributes describing the software tool users, if that data was also collected.
Distill’s usage and usability metrics are derived from a representation of users’ sequential
interactions with the application as a directed graph. This provides an extensible framework
for providing insight as to how users integrate the functional components of the application
to accomplish tasks.
- [[attachment:userale_figure_2.png]]
+ {{attachment:userale_figure_2.png||align="left"}}
  Figure 2. Software as a Sensor™ System Architecture with all components.
  The Test Application Portal (TAP) provides a single point of interface for adopters of the
Software as a Sensor™ project. Through the Portal, adopters can register their applications,
providing version data and permissions to others for accessing data. The Portal ensures that
all components of the Software as a Sensor™ Project have the same information. The Portal
also hosts a number of python D3 visualization libraries, providing adopters with a customizable
“dashboard” with which to analyze and view user activity data, calling analytic processes
from Distill.
  Finally, the Subject Tracking and Online User Testing (STOUT) application, provides support
for HCI/UX researchers that want to collect data from users in systematic ways or within experimental
designs. STOUT supports user registration, anonymization, user tracking, tasking (see Figure
3), and data integration from a variety of services. STOUT allows adopters to perform human
subject review board compliant research studies, and both between- and within-subjects designs.
Adopters can add tasks, surveys and questionnaires through 3rd party services (e.g., SurveyMonkey).
STOUT tracks users’ progress by passing a unique user IDs to other services, allowing researchers
to trace progress by passing a unique user IDs to other services, allowing researchers to
trace form data and User ALE logs to specific users and task sets (see Figure 4).
- [[attachment:userale_figure_3.png]]
+ {{attachment:userale_figure_3.png||align="left"}}
  Figure 3. STOUT assigns participants subjects to experimental conditions and ensures the
correct task sequence. STOUT’s Django back end provides data on task completion, this can
be used to drive other automation, including unlocking different task sequences and/or achievements.
- [[attachment:userale_figure_4.png]]
+ {{attachment:userale_figure_4.png||align="left"}}
  Figure 4. STOUT User Tracking. Anonymized User IDs (hashes) are concatenated with unique
Task IDs. This “Session ID” is appended to URLs (see Highlighted region), custom variable
fields, and User ALE, to provide and integrated user testing data collection service.
  STOUT also provides for data polling from third party services (e.g., SurveyMonkey) and
integration with python or R scripts for statistical processing of data collected through
STOUT. D3 visualization libraries embedded in STOUT allow adopters to view distributions of
quantitative data collected from form data (see Figure 5).
- [[attachment:userale_figure_5.png]]
+ {{attachment:userale_figure_5.png||align="left"}}
  Figure 5. STOUT Visualization. STOUT gives experimenters direct and continuous access to
automatically processed research data.
  == Insights from User Activity Logs ==
@@ -81, +91 @@

  The Software as a Sensor™ Project is ultimately designed to address the wide gaps between
current best practices in software user testing and trends toward agile software development
practices. Like much of the applied psychological sciences, user testing methods generally
borrow heavily from basic research methods. These methods are designed to make data collection
systematic and remove extraneous influences on test conditions. However, this usually means
removing what we test from dynamic, noisy—real-life—environments. The Software as a Sensor™
Project is designed to allow for the same kind of systematic data collection that we expect
in the laboratory, but in real-life software environments, by making software environments
data collection platforms. In doing so, we aim to not only collect data from more realistic
environments, and use-cases, but also to integrate the test enterprise into agile software
development process. 
  Our vision for The Software as a Sensor™ Project is that it provides software developers,
HCI/UX researchers, and project managers a mechanism for continuous, iterative usability testing
for software tools in a way that supports the flow (and schedule) of modern software development
practices—Iterative, Waterfall, Spiral, and Agile. This is enabled by a few discriminating
- [[attachment:userale_figure_6.png]]
+ {{attachment:userale_figure_6.png||align="left"}}
  Figure 6. Version to Version Testing for Agile, Iterative Software Development Methods.
The Software as a Sensor™ Project enables new methods for collecting large amounts of data
on software tools, deriving insights rapidly to inject into subsequent iterations
   * Insights enabling software tool usability assessment and improvement can be inferred
directly from interactions with the tool in “real-world” environments. This is a sea-change
in thinking compared to canonical laboratory approaches that seek to artificially isolate
extraneous influences on the user and the software. The Software as a Sensor™ Project enables
large scale, remote, opportunities for data collection with minimal investment and no expensive
lab equipment (or laboratory training). This allows adopters to see how users will interact
with their technology in their places of work, at home, etc. 
@@ -237, +249 @@

     * Kyle Finley ( 
-    * Lewis John McGibbney
+    * Lewis John McGibbney (
  == Sponsors ==
@@ -245, +257 @@

   * Lewis McGibbney (NASA/JPL)
  === Nominated Mentors ===
-  * Paul Ramirez
+  * Paul Ramirez (NASA/JPL)
-  * Lewis John McGibbney
+  * Lewis John McGibbney (NASA/JPL)
-  * Chris Mattmann
+  * Chris Mattmann (NASA/JPL)
  == Sponsoring Entity ==
  The Apache Incubator
+ == References ==
+ Mariano, L. J., Poore, J. C., Krum, D. M., Schwartz, J. L., Coskren, W. D., & Jones,
E. M. (2015). Modeling Strategic Use of Human Computer Interfaces with Novel Hidden Markov
Models. [Methods]. Frontiers in Psychology, 6. doi: 10.3389/fpsyg.2015.00919
+ Poore, J., Webb, A., Cunha, M., Mariano, L., Chapell, D., Coskren, M., & Schwartz, J.
(2016). Operationalizing Engagement with Multimedia as User Coherence with Context. IEEE Transactions
on Affective Computing, PP(99), 1-1. doi: 10.1109/taffc.2015.2512867

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message