Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 317AE200B40 for ; Fri, 17 Jun 2016 03:20:54 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 2E75B160A52; Fri, 17 Jun 2016 01:20:54 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 25BC0160A51 for ; Fri, 17 Jun 2016 03:20:51 +0200 (CEST) Received: (qmail 36786 invoked by uid 500); 17 Jun 2016 01:20:50 -0000 Mailing-List: contact general-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@incubator.apache.org Delivered-To: mailing list general@incubator.apache.org Received: (qmail 36774 invoked by uid 99); 17 Jun 2016 01:20:50 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Jun 2016 01:20:50 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 148BA180290 for ; Fri, 17 Jun 2016 01:20:50 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.198 X-Spam-Level: * X-Spam-Status: No, score=1.198 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id maY_1-ldnNfw for ; Fri, 17 Jun 2016 01:20:37 +0000 (UTC) Received: from mail-io0-f174.google.com (mail-io0-f174.google.com [209.85.223.174]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 486AC5F39A for ; Fri, 17 Jun 2016 01:20:36 +0000 (UTC) Received: by mail-io0-f174.google.com with SMTP id t74so58060875ioi.0 for ; Thu, 16 Jun 2016 18:20:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc; bh=Sbsj8YF4nN7vRSmDL+0F1HoA1TumTLzA0U167wn3OsQ=; b=Qe2Krm3PBuOEmvd1H34YusxD+kDWmgl55XY7O214wejDGbOvpr+NTbIPGq7WKDJ6D4 0d4oPjw/ePpPoqxY0gijfleWEZmPwNvc+FTvmpprmETiVZd3uSHB4lGZZaMBa+mis7bX +KX28XrYu+2MlZ4Ra4YlKJeyemrW2pNmG3unqPssddz7P2IxfJyXTzQ1VDJspwFrFovD o/NSNas3KS5d6KNsGi8adIwaVF7X8pT9VjFueyja2AiYtw+uN/Jht5/jSWyt3o9t/85u nioh2RsKq0ZFl0ph/RvFB6v0TpTpz4UZApoEyp7JSlZ4/s7lz2h/sPftUjwKNRJQuTyx PLgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=Sbsj8YF4nN7vRSmDL+0F1HoA1TumTLzA0U167wn3OsQ=; b=jSau72Su8pLxGNrl+GNqR3NQelgoM+UkqR8PI5ejhcdR5+03XCTlszzzG56MdgyFmi SjeqlcTffO9KYZFTpFDRWqUxY6HwuCYldR7W527mQwI0rrSDI1PyURmnTZZljtBhkrye Hm9NYB8lAiICqJcYwY6rX1J0T8KmPYDBiXo0chErvJ3c0j2ybdDBqbIAm+8mFo5f/AiG TYGFaPFe3xsqlRsfjL+ONXvWu+Jx3wyPC8k0Z/dFP94fKbpQrynFNwneKLOJnmdvG4bf 3S2qf1IeX9xGa4cS/YeOnOfCs+xQ2nbq2OlCwAaRmu8fhI28h8qcNVtLjsS/Zm40Ej+4 irRw== X-Gm-Message-State: ALyK8tIQqzaVbrJP1tem2jVDmcud8o/Q6Bs+HHyPyBMccgTyjUBDxL9R3rCv0JHUU7dj+864cXC8MhgYf2dkuA== X-Received: by 10.107.26.85 with SMTP id a82mr12715276ioa.13.1466126428993; Thu, 16 Jun 2016 18:20:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.31.201 with HTTP; Thu, 16 Jun 2016 18:20:28 -0700 (PDT) From: Lewis John Mcgibbney Date: Thu, 16 Jun 2016 18:20:28 -0700 Message-ID: Subject: [RESULT] WAS Re: [VOTE] Accept SensSoft into the Apache Incubator WAS Re: [VOTE] Accept To: general@incubator.apache.org Cc: jpoore@draper.com Content-Type: multipart/alternative; boundary=001a113fd586ff025505356f2836 archived-at: Fri, 17 Jun 2016 01:20:54 -0000 --001a113fd586ff025505356f2836 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi general@, 72 hours has come and gone I am therefore closing off this VOTE'ing thread. Thank you to everyone that was able to VOTE. [4] +1 Accept SensSoft into the Apache Incubator Lewis John Mcgibbney* Chris A Mattmann* Tom Barber* Henry Saptura* [0] +/-0 Not overly bothered either way [0] -1 DO NOT accept SensSoft into the Apache Incubator (please state why) *IPMC binding So great news, SensSoft is accepted into the Incubator :) :) :) I'll progress with workflow. Thanks folks. Lewis On Mon, Jun 13, 2016 at 3:07 PM, Lewis John Mcgibbney < lewis.mcgibbney@gmail.com> wrote: > Title here should be [VOTE] Accept SensSoft into the Apache Incubator > > On Mon, Jun 13, 2016 at 10:55 AM, Lewis John Mcgibbney < > lewis.mcgibbney@gmail.com> wrote: > >> Hi general@, >> Since I am back from a bit of vacation and it seems the discussion has >> died down, I am now calling a vote on accepting SensSoft into the Apache >> Incubator. >> >> For those who are interested the DISCUSS thread can be found at >> https://s.apache.org/senssoft_discuss >> >> This vote will run for the usual 72 hours. >> >> Please VOTE as follows >> >> [ ] +1 Accept SensSoft into the Apache Incubator >> [ ] +/-0 Not overly bothered either way >> [ ] -1 DO NOT accept SensSoft into the Apache Incubator (please state wh= y) >> >> Thanks everyone who contributed to DISCUSS and is able to participate in >> VOTE >> >> Best >> Lewis >> P.S. Here is my +1 >> >> ##################### >> >> =3D SensSoft Proposal =3D >> >> =3D=3D Abstract =3D=3D >> The Software as a Sensor=E2=84=A2 (SensSoft) Project offers an open-sour= ce >> (ALv2.0) >> software tool usability testing platform. It includes a number of >> components that work together to provide a platform for collecting data >> about user interactions with software tools, as well as archiving, >> analyzing and visualizing that data. Additional components allow for >> conducting web-based experiments in order to capture this data within a >> larger experimental framework for formal user testing. These components >> currently support Java Script-based web applications, although the schem= a >> for =E2=80=9Clogging=E2=80=9D user interactions can support mobile and d= esktop >> applications, as well. Collectively, the Software as a Sensor Project >> provides an open source platform for assessing how users interacted with >> technology, not just collecting what they interacted with. >> >> =3D=3D Proposal =3D=3D >> The Software as a Sensor=E2=84=A2 Project is a next-generation platform = for >> analyzing how individuals and groups of people make use of software tool= s >> to perform tasks or interact with other systems. It is composed of a >> number >> of integrated components: >> * User Analytic Logging Engine (User ALE) refers to a simple Applicatio= n >> Program Interface (API) and backend infrastructure. User ALE provides >> =E2=80=9Cinstrumentation=E2=80=9D for software tools, such that each use= r interaction >> within the application can be logged, and sent as a JSON message to an >> Elasticsearch/Logstash/Kibana (Elastic Stack) backend. >> * The API provides a robust schema that makes user activities human >> readable, and provides an interpretive context for understanding that >> activity=E2=80=99s functional relevance within the application. The sche= ma >> provides >> highly granular information best suited for advanced analytics. This >> hierarchical schema is as follows: >> * Element Group: App features that share function (e.g., map group) >> * Element Sub: Specific App feature (e.g., map tiles) >> * Element Type: Category of feature (e.g., map) >> * Element ID: [attribute] id >> * Activity: Human imposed label (e.g., =E2=80=9Csearch=E2=80=9D) >> * Action: Event class (e.g., zoom, hover, click) >> * The API can either be manually embedded in the app source code, or >> implemented automatically by inserting a script tag in the source code. >> * Users can either setup up their own Elastic stack instance, or use >> Vagrant, a virtualization environment, to deploy a fully configured >> Elastic >> stack instance to ship and ingest user activity logs and visualize their >> log data with Kibana. >> * RESTful APIs allow other services to access logs directly from >> Elasticsearch. >> * User ALE allows adopters to own the data they collect from users >> outright, and utilize it as they see fit. >> * Distill is an analytics stack for processing user activity logs >> collected through User ALE. Distill is fully implemented in Python, >> dependent on graph-tool to support graph analytics and other external >> python libraries to query Elasticsearch. The two principle functions of >> Distill are segmentation and graph analytics: >> * Segmentation allows for partitioning of the available data along >> multiple axes. Subsets of log data can be selected via their attributes = in >> User ALE (e.g. Element Group or Activity), and by users/sessions. Disti= ll >> also has the capability to ingest and segment data by additional >> attributes >> collected through other channels (e.g. survey data, demographics).This >> allows adopters to focus their analysis of log data on precisely the >> attributes of their app (or users) they care most about. >> * Distill=E2=80=99s usage metrics are derived from a probabilistic >> representation of the time series of users=E2=80=99 interactions with th= e elements >> of the application. A directed network is constructed from the >> representation, and metrics from graph theory (e.g. betweenness >> centrality, >> in/out-degree of nodes) are derived from the structure. These metrics >> provide adopters ways of understanding how different facets of the app a= re >> used together, and they capture canonical usage patterns of their >> application. This broad analytic framework provides adopters a way to >> develop and utilize their own metrics >> * The Test Application Portal (TAP) provides a single, user-friendly >> interface to Software as a Sensor=E2=84=A2 Project components, including >> visualization functionality for Distill Outputs leveraging Django, React= , >> and D3.js. It has two key functions: >> * It allows adopters to register apps, providing metadata regarding >> location, app name, version, etc., as well as permissions regarding who >> can >> access user data. This information is propagated to all other components >> of >> the larger system. >> * The portal also stages visualization libraries that make calls to >> Distill. This allows adopters to analyze their data as they wish to; it= =E2=80=99s >> =E2=80=9Cdashboard=E2=80=9D feel provides a way to customize their views= with >> adopter-generated widgets (e.g., D3 libraries) beyond what is included i= n >> the initial open source offering. >> * The Subject Tracking and Online User Testing (STOUT) application is a= n >> optional component that turns Software as a Sensor=E2=84=A2 Technology i= nto a >> research/experimentation enterprise. Designed for psychologists and HCI/= UX >> researchers, STOUT allows comprehensive human subjects data protection, >> tracking, and tasking for formal research on software tools. STOUT is >> primarily python, with Django back-end for authentication, permissions, >> and >> tracking, MongoDB for databasing, and D3 for visualization. STOUT includ= es >> a number of key features: >> * Participants can register in studies of software tools using their >> own >> preferred credentials. As part of registration, participants can be >> directed through human subjects review board compliant consent forms >> before >> study enrollment. >> * STOUT stores URLs to web/network accessible software tools as well = as >> URLs to third party survey services (e.g., surveymonkey), this allows >> adopters to pair software tools with tasks, and collect survey data and >> comments from participants prior to, during, or following testing with >> software tools. >> * STOUT tracks participants=E2=80=99 progress internally, and by appe= nding a >> unique identifier, and task identifier to URLs. This information can be >> passed to other processes (e.g., User ALE) allowing for disambiguation >> between participants and tasks in experiments on the open web. >> * STOUT supports between and within-subjects experimental designs, wi= th >> random assignment to experimental conditions. This allows for testing >> across different versions of applications. >> * STOUT can also use Django output (e.g., task complete) to automate >> other processes, such as automated polling applications serving 3rd part= y >> form data APIs (e.g.,SurveyMonkey), and python or R scripts to provide >> automated post-processing on task or survey data. >> * STOUT provides adopters a comprehensive dashboard view of data >> collected and post-processed through its extensions; in addition to user >> enrollment, task completion, and experiment progress metrics, STOUT allo= ws >> adopters to visualize distributions of scores collected from task and >> survey data. >> >> Each component is available through its own repository to support organi= c >> growth for each component, as well as growth of the whole platform=E2=80= =99s >> capabilities. >> >> =3D=3D Background and Rationale =3D=3D >> Any tool that people use to accomplish a task can be instrumented; once >> instrumented, those tools can be used to report how they were used to >> perform that task. Software tools are ubiquitous interfaces for people t= o >> interact with data and other technology that can be instrumented for suc= h >> a >> purpose. Tools are different than web pages or simple displays, however; >> they are not simply archives for information. Rather, they are ways of >> interfacing with and manipulating data and other technology. There are >> numerous consumer solutions for understanding how people move through we= b >> pages and displays (e.g., Google Analytics, Adobe Omniture). There are f= ar >> fewer options for understanding how software tools are used. This requir= es >> understanding how users integrate a tool=E2=80=99s functionality into us= age >> strategies to perform tasks, how users sequence the functionality provid= ed >> them, and deeper knowledge of how users understand the features of >> software >> as a cohesive tool. The Software as a Sensor=E2=84=A2 Project is designe= d to >> address this gap, providing the public an agile, cost-efficient solution >> for improving software tool design, implementation, and usability. >> >> =3D=3D Software as a Sensor=E2=84=A2 Project Overview =3D=3D >> >> {{attachment:userale_figure_1.png}} >> >> Figure 1. User ALE Elastic Back End Schema, with Transfer Protocols. >> >> Funded through the DARPA XDATA program and other sources, the Software a= s >> a >> Sensor=E2=84=A2 Project provides an open source (ALv2.0) solution for >> instrumenting >> software tools developed for the web so that when users interact with it= , >> their behavior is captured. User behavior, or user activities, are >> captured >> and time-stamped through a simple application program interface (API) >> called User Analytic Logging Engine (User ALE). User ALE=E2=80=99s key >> differentiator is the schema that it uses to collect information about >> user >> activities; it provides sufficient context to understand activities with= in >> the software tool=E2=80=99s overall functionality. User ALE captures eac= h user >> initiated action, or event (e.g., hover, click, etc.), as a nested actio= n >> within a specific element (e.g., map object, drop down item, etc.), whic= h >> are in turn nested within element groups (e.g., map, drop down list) (se= e >> Figure 1). This information schema provides sufficient context to >> understand and disambiguate user events from one another. In turn, this >> enables myriad analysis possibilities at different levels of tool design >> and more utility to end-user than commercial services currently offer. >> Once instrumented with User ALE, software tools become human signal >> sensors >> in their own right. Most importantly, the data that User ALE collects is >> owned outright by adopters and can be made available to other processes >> through scalable Elastic infrastructure and easy-to-manage Restful APIs. >> Distill is the analytic framework of the Software as a Sensor=E2=84=A2 P= roject, >> providing (at release) segmentation and graph analysis metrics describin= g >> users=E2=80=99 interactions with the application to adopters. The segmen= tation >> features allow adopters to focus their analyses of user activity data >> based >> on desired data attributes (e.g., certain interactions, elements, etc.), >> as >> well as attributes describing the software tool users, if that data was >> also collected. Distill=E2=80=99s usage and usability metrics are derive= d from a >> representation of users=E2=80=99 sequential interactions with the applic= ation as a >> directed graph. This provides an extensible framework for providing >> insight >> as to how users integrate the functional components of the application t= o >> accomplish tasks. >> >> {{attachment:userale_figure_2.png}} >> >> Figure 2. Software as a Sensor=E2=84=A2 System Architecture with all com= ponents. >> >> The Test Application Portal (TAP) provides a single point of interface f= or >> adopters of the Software as a Sensor=E2=84=A2 project. Through the Porta= l, >> adopters >> can register their applications, providing version data and permissions = to >> others for accessing data. The Portal ensures that all components of the >> Software as a Sensor=E2=84=A2 Project have the same information. The Por= tal also >> hosts a number of python D3 visualization libraries, providing adopters >> with a customizable =E2=80=9Cdashboard=E2=80=9D with which to analyze an= d view user >> activity data, calling analytic processes from Distill. >> Finally, the Subject Tracking and Online User Testing (STOUT) applicatio= n, >> provides support for HCI/UX researchers that want to collect data from >> users in systematic ways or within experimental designs. STOUT supports >> user registration, anonymization, user tracking, tasking (see Figure 3), >> and data integration from a variety of services. STOUT allows adopters t= o >> perform human subject review board compliant research studies, and both >> between- and within-subjects designs. Adopters can add tasks, surveys an= d >> questionnaires through 3rd party services (e.g., SurveyMonkey). STOUT >> tracks users=E2=80=99 progress by passing a unique user IDs to other ser= vices, >> allowing researchers to trace progress by passing a unique user IDs to >> other services, allowing researchers to trace form data and User ALE log= s >> to specific users and task sets (see Figure 4). >> >> {{attachment:userale_figure_3.png}} >> >> Figure 3. STOUT assigns participants subjects to experimental conditions >> and ensures the correct task sequence. STOUT=E2=80=99s Django back end p= rovides >> data on task completion, this can be used to drive other automation, >> including unlocking different task sequences and/or achievements. >> >> {{attachment:userale_figure_4.png}} >> >> Figure 4. STOUT User Tracking. Anonymized User IDs (hashes) are >> concatenated with unique Task IDs. This =E2=80=9CSession ID=E2=80=9D is = appended to URLs >> (see Highlighted region), custom variable fields, and User ALE, to provi= de >> and integrated user testing data collection service. >> >> STOUT also provides for data polling from third party services (e.g., >> SurveyMonkey) and integration with python or R scripts for statistical >> processing of data collected through STOUT. D3 visualization libraries >> embedded in STOUT allow adopters to view distributions of quantitative >> data >> collected from form data (see Figure 5). >> >> {{attachment:userale_figure_5.png}} >> >> Figure 5. STOUT Visualization. STOUT gives experimenters direct and >> continuous access to automatically processed research data. >> >> =3D=3D Insights from User Activity Logs =3D=3D >> >> The Software as a Sensor=E2=84=A2 Project provides data collection and a= nalytic >> services for user activities collected during interaction with software >> tools. However, the Software as a Sensor=E2=84=A2 Project emerged from y= ears of >> research focused on the development of novel, reliable methods for >> measuring individuals=E2=80=99 cognitive state in a variety of contexts. >> Traditional approaches to assessment in a laboratory setting include >> surveys, questionnaires, and physiology (Poore et al., 2016). Research >> performed as part of the Software as a Sensor=E2=84=A2 project has shown= that the >> same kind of insights derived from these standard measurement approaches >> can also be derived from users=E2=80=99 behavior. Additionally, we have = explored >> insights that can only be gained by analyzing raw behavior collected >> through software interactions (Mariano et al., 2015). The signal >> processing >> and algorithmic approaches resulting from this research have been >> integrated into the Distill analytics stack. This means that adopters wi= ll >> not be left to discern for themselves how to draw insights from the data >> they gather about their software tools, although they will have the >> freedom >> to explore their own methods as well. >> Insights from user activities provided by Distill=E2=80=99s analytics fr= amework >> fall under two categories, broadly classified as functional workflow and >> usage statistics: >> Functional workflow insights tell adopters how user activities are >> connected, providing them with representations of how users integrate th= e >> application=E2=80=99s features together in time. These insights are info= rmative >> for >> understanding the step-by-step process by which users interact with >> certain >> facets of a tool. For example, questions like =E2=80=9Chow are my users, >> constructing plots?=E2=80=9D are addressable through workflow analysis. = Workflows >> provide granular understanding of process level mechanics and can be >> modeled probabilistically through a directed graph representation of the >> data, and by identification of meaningful sub-sequences of user activiti= es >> actually observed in the population. Metrics derived provide insight abo= ut >> the structure and temporal features of these mechanics, and can help >> highlight efficiency problems within workflows. For example, workflow >> analysis could help identify recursive, repetitive behaviors, and might = be >> used to define what =E2=80=9Cfloundering=E2=80=9D looks like for that pa= rticular tool. >> Functional workflow analysis can also support analyses with more breadth= . >> Questions like, =E2=80=9Chow are my users integrating my tools=E2=80=99 = features into a >> cohesive whole? Are they relying on the tool as a whole or just using ve= ry >> specific parts of it?=E2=80=9D Adopters will be able to explore how user= s think >> about software as cohesive tools and examine if users are relying on >> certain features as central navigation or analytic features. This allows >> for insights into whether tools are designed well enough for users to >> understand that they need to rely on multiple features together. >> Through segmentation, adopters can select the subset of the data -softwa= re >> element, action, user demographics, geographic location, etc.- they want >> to >> analyze. This will allow them to compare, for example, specific user >> populations against one another in terms of how they integrate software >> functionality. Importantly, the graph-based analytics approach provides = a >> flexible representation of the time series data that can capture and >> quantify canonical usage patterns, enabling direct comparisons between >> users based on attributes of interest. Other modeling approaches have be= en >> utilized to explore similar insights and may be integrated at a later da= te >> (Mariano, et al., 2015). >> Usage statistics derive metrics from simple frequentist approaches to >> understanding, coarsely, how much users are actually using applications. >> This is different from simple =E2=80=9Ctraffic=E2=80=9D metrics, however= , which assess how >> many users are navigating to a page or tool. Rather usage data provides >> insight on how much raw effort (e.g., number of activities) is being >> expended while users are interacting with the application. This provides >> deeper insight into discriminating =E2=80=9Cvisitors=E2=80=9D from =E2= =80=9Cusers=E2=80=9D of software >> tools. Moreover, given the information schema User ALE provides, adopter= s >> will be able to delve into usage metrics related to specific facets of >> their application. >> Given these insights, different sets of adopters=E2=80=94software develo= pers, >> HCI/UX researchers, and project managers=E2=80=94may utilize The Softwar= e as a >> Sensor=E2=84=A2 Project for a variety different use cases, which may inc= lude: >> * Testing to see if users are interacting with software tools in expect= ed >> or unexpected ways. >> * Understanding how much users are using different facets of different >> features in service of planning future developments. >> * Gaining additional context for translating user/customer comments int= o >> actionable software fixes. >> * Understanding which features users have trouble integrating to guide >> decisions on how to allocate resources to further documentation. >> * Understanding the impact that new developments have on usability from >> version to version. >> * Market research on how users make use of competitors=E2=80=99 applica= tions to >> guide decisions on how to build discriminating software tools. >> * General research on Human Computer Interaction in service of refining >> UX >> and design principles. >> * Psychological science research using software as data collection >> platforms for cognitive tasks. >> >> =3D=3D Differentiators =3D=3D >> >> The Software as a Sensor=E2=84=A2 Project is ultimately designed to addr= ess the >> wide gaps between current best practices in software user testing and >> trends toward agile software development practices. Like much of the >> applied psychological sciences, user testing methods generally borrow >> heavily from basic research methods. These methods are designed to make >> data collection systematic and remove extraneous influences on test >> conditions. However, this usually means removing what we test from >> dynamic, >> noisy=E2=80=94real-life=E2=80=94environments. The Software as a Sensor= =E2=84=A2 Project is >> designed >> to allow for the same kind of systematic data collection that we expect = in >> the laboratory, but in real-life software environments, by making softwa= re >> environments data collection platforms. In doing so, we aim to not only >> collect data from more realistic environments, and use-cases, but also t= o >> integrate the test enterprise into agile software development process. >> Our vision for The Software as a Sensor=E2=84=A2 Project is that it prov= ides >> software developers, HCI/UX researchers, and project managers a mechanis= m >> for continuous, iterative usability testing for software tools in a way >> that supports the flow (and schedule) of modern software development >> practices=E2=80=94Iterative, Waterfall, Spiral, and Agile. This is enabl= ed by a >> few >> discriminating facets: >> >> {{attachment:userale_figure_6.png}} >> >> Figure 6. Version to Version Testing for Agile, Iterative Software >> Development Methods. The Software as a Sensor=E2=84=A2 Project enables n= ew methods >> for collecting large amounts of data on software tools, deriving insight= s >> rapidly to inject into subsequent iterations >> >> * Insights enabling software tool usability assessment and improvement >> can >> be inferred directly from interactions with the tool in =E2=80=9Creal-wo= rld=E2=80=9D >> environments. This is a sea-change in thinking compared to canonical >> laboratory approaches that seek to artificially isolate extraneous >> influences on the user and the software. The Software as a Sensor=E2=84= =A2 Project >> enables large scale, remote, opportunities for data collection with >> minimal >> investment and no expensive lab equipment (or laboratory training). This >> allows adopters to see how users will interact with their technology in >> their places of work, at home, etc. >> >> * Insights are traceable to the software itself. Traditionally laborato= ry >> measures=E2=80=94questionnaires, interviews, and physiology=E2=80=94coll= ect data that is >> convenient for making inferences about psychological states. However, it >> is >> notoriously difficult to translate this data into actionable =E2=80=9Cge= t-well=E2=80=9D >> strategies in technology development. User ALE=E2=80=99s information sch= ema is >> specifically designed to dissect user interaction within the terminology >> of >> application design, providing a familiar nomenclature for software >> developers to interpret findings with. >> >> * Granular data collection enables advanced modeling and analytics. Use= r >> ALE=E2=80=99s information schema dissects user interaction by giving con= text to >> activity within the functional architecture of software tools. Treating >> each time-series of user activity as a set of events nested within >> functional components provides sufficient information for a variety of >> modeling approaches that can be used to understand user states (e.g., >> engagement and cognitive load), user workflows (e.g., sub-sequences), an= d >> users=E2=80=99 mental models of how software tool features can be integr= ated (in >> time) to perform tasks. In contrast, commercial services such as Google >> Analytics and Adobe Analytics (Omniture) provide very sparse options for >> describing events. They generally advocate for using =E2=80=9Cboiler pla= te=E2=80=9D event >> sets that are more suited to capturing count data for interactions with >> specific content (e.g., videos, music, banners) and workflows through >> =E2=80=9Cmarketplace=E2=80=9D like pages. User ALE provides content agno= stic approaches >> for >> capturing user activities by letting adopters label them in domain >> specific >> ways that give them context. This provides a means by which identical us= er >> activities (e.g. click, select, etc.) can be disambiguated from each oth= er >> based on which functional sub-component of the tool they have been >> assigned >> to. >> >> * Adopter-generated content, analytics and data ownership. The Software >> as >> a Sensor=E2=84=A2 Project is a set of open-source products built from ot= her >> open-source products. This project will allow adopters to generate their >> own content easily, using open source analytics and visualization >> capabilities. By design, we also allow adopters to collect and manage >> their >> own data with support from widely used open source data architectures >> (e.g., Elastic). This means that adopters will not have to pay for >> additional content that they can develop themselves to make use of the >> service, and do not have to expose their data to third party commercial >> services. This is useful for highly proprietary software tools that are >> designed to make use of sensitive data, or are themselves sensitive. >> >> =3D=3D Current Status =3D=3D >> >> All components of the Software as a Sensor=E2=84=A2 Project were origina= lly >> designed and developed by Draper as part of DARPA=E2=80=99s XDATA projec= t, >> although >> User ALE is being used on other funded R&D projects, including DARPA >> RSPACE, AFRL project, and Draper internally funded projects. >> Currently, only User ALE is publically available, however, the Portal, >> Distill, and STOUT will be publically available in the May/June 2016 >> time-frame. The last major release of User ALE was May, 2015. All >> components are currently maintained in separate repositories through >> GitHub >> (github.com/draperlaboratory). >> Currently, only software tools developed with Javascript are supported. >> However, we are currently working on pythonQT implementations for User A= LE >> that will support many desktop applications. >> >> =3D=3D Meritocracy =3D=3D >> The current developers are familiar with meritocratic open source >> development at Apache. Apache was chosen specifically because we want to >> encourage this style of development for the project. >> >> =3D=3D Community =3D=3D >> The Software as a Sensor=E2=84=A2 Project is new and our community is no= t yet >> established. However, community building and publicity is a major thrust= . >> Our technology is generating interest within industry, particularly in t= he >> HCI/UX community, both Aptima and Charles River Analytics, for example a= re >> interested in being adopters. We have also begun publicizing the project >> to >> software development companies and universities, recently hosting a publ= ic >> focus group for Boston, MA area companies. >> We are also developing communities of interested within the DoD and >> Intelligence community. The NGA Xperience Lab has expressed interest in >> becoming a transition partner as has the Navy=E2=80=99s HCIL group. We a= re also >> aggressively pursuing adopters at AFRL=E2=80=99s Human Performance Wing,= Analyst >> Test Bed. >> During incubation, we will explicitly seek to increase our adoption, >> including academic research, industry, and other end users interested in >> usability research. >> >> =3D=3D Core Developers =3D=3D >> The current set of core developers is relatively small, but includes >> Draper >> full-time staff. Community management will very likely be distributed >> across a few full-time staff that have been with the project for at leas= t >> 2 >> years. Core personnel can be found on our website: >> http://www.draper.com/softwareasasensor >> >> =3D=3D Alignment =3D=3D >> The Software as a Sensor=E2=84=A2 Project is currently Copyright (c) 201= 5, 2016 >> The >> Charles Stark Draper Laboratory, Inc. All rights reserved and licensed >> under Apache v2.0. >> >> =3D=3D Known Risks =3D=3D >> >> =3D=3D=3D Orphaned products =3D=3D=3D >> There are currently no orphaned products. Each component of The Software >> as >> a Sensor=E2=84=A2 Project has roughly 1-2 dedicated staff, and there is >> substantial >> collaboration between projects. >> >> =3D=3D=3D Inexperience with Open Source =3D=3D=3D >> Draper has a number of open source software projects available through >> www.github.com/draperlaboratory. >> >> =3D=3D Relationships with Other Apache Products =3D=3D >> Software as a Sensor=E2=84=A2 Project does not currently have any depend= ences on >> Apache Products. We are also interested in coordinating with other >> projects >> including Usergrid, and others involving data processing at large scales= , >> time-series analysis and ETL processes. >> >> =3D=3D Developers =3D=3D >> The Software as a Sensor=E2=84=A2 Project is primarily funded through co= ntract >> work. There are currently no =E2=80=9Cdedicated=E2=80=9D developers, how= ever, the same >> core >> team does work will continue work on the project across different >> contracts >> that support different features. We do intend to maintain a core set of >> key >> personnel engaged in community development and maintenance=E2=80=94in th= e future >> this may mean dedicated developers funded internally to support the >> project, however, the project is tied to business development strategy t= o >> maintain funding into various facets of the project. >> >> =3D=3D Documentation =3D=3D >> Documentation is available through Github; each repository under the >> Software as a Sensor=E2=84=A2 Project has documentation available throug= h wiki=E2=80=99s >> attached to the repositories. >> >> =3D=3D Initial Source =3D=3D >> Current source resides at Github: >> * https://github.com/draperlaboratory/user-ale (User ALE) >> * https://github.com/draperlaboratory/distill (Distill) >> * https://github.com/draperlaboratory/stout (STOUT and Extensions) >> * https://github.com/draperlaboratory/ >> >> =3D=3D External Dependencies =3D=3D >> Each component of the Software as a Sensor=E2=84=A2 Project has its own >> dependencies. Documentation will be available for integrating them. >> >> =3D=3D=3D User ALE =3D=3D=3D >> * Elasticsearch: https://www.elastic.co/ >> * Logstash: https://www.elastic.co/products/logstash >> * Kibana (optional): https://www.elastic.co/products/kibana >> =3D=3D=3D STOUT =3D=3D=3D >> * Django: https://www.djangoproject.com/ >> * django-axes >> * django-custom-user >> * django-extensions >> * Elasticsearch: https://www.elastic.co/ >> * Gunicorn: http://gunicorn.org/ >> * MySQL-python: https://pypi.python.org/pypi/MySQL-python >> * Numpy: http://www.numpy.org/ >> * Pandas: http://pandas.pydata.org/ >> * psycopg2: http://initd.org/psycopg/ >> * pycrypto: https://www.dlitz.net/software/pycrypto/ >> * pymongo: https://api.mongodb.org/python/current/ >> * python-dateutil: https://labix.org/python-dateutil >> * pytz: https://pypi.python.org/pypi/pytz/ >> * requests: http://docs.python-requests.org/en/master/ >> * six: https://pypi.python.org/pypi/six >> * urllib3: https://pypi.python.org/pypi/urllib3 >> * mongoDB: https://www.mongodb.org/ >> * R (optional): https://www.r-project.org/ >> =3D=3D=3D Distill =3D=3D=3D >> * Flask: http://flask.pocoo.org/ >> * Elasticsearch-dsl: https://github.com/elastic/elasticsearch-dsl-py >> * graph-tool: https://git.skewed.de/count0/graph-tool >> * OpenMp: http://openmp.org/wp/ >> * pandas: http://pandas.pydata.org/ >> * numpy: http://www.numpy.org/ >> * scipy: http://www.numpy.org/ >> =3D=3D=3D Portal =3D=3D=3D >> * Django: https://www.djangoproject.com/ >> * React: https://facebook.github.io/react/ >> * D3.js: https://d3js.org/ >> >> =3D=3D=3D GNU GPL 2 =3D=3D=3D >> >> >> =3D=3D=3D LGPL 2.1 =3D=3D=3D >> >> >> =3D=3D=3D Apache 2.0 =3D=3D=3D >> >> >> =3D=3D=3D GNU GPL =3D=3D=3D >> >> >> =3D=3D Required Resources =3D=3D >> * Mailing Lists >> * priv...@senssoft.incubator.apache.org >> * d...@senssoft.incubator.apache.org >> * comm...@senssoft.incubator.apache.org >> >> * Git Repos >> * https://git-wip-us.apache.org/repos/asf/User-ALE.git >> * https://git-wip-us.apache.org/repos/asf/STOUT.git >> * https://git-wip-us.apache.org/repos/asf/DISTILL.git >> * https://git-wip-us.apache.org/repos/asf/TAP.git >> >> * Issue Tracking >> * JIRA SensSoft (SENSSOFT) >> >> * Continuous Integration >> * Jenkins builds on https://builds.apache.org/ >> >> * Web >> * http://SoftwareasaSensor.incubator.apache.org/ >> * wiki at http://cwiki.apache.org >> >> =3D=3D Initial Committers =3D=3D >> The following is a list of the planned initial Apache committers (the >> active subset of the committers for the current repository on Github). >> >> * Joshua Poore (jpo...@draper.com) >> * Laura Mariano (lmari...@draper.com) >> * Clayton Gimenez (cgime...@draper.com) >> * Alex Ford (af...@draper.com) >> * Steve York (sy...@draper.com) >> * Fei Sun (f...@draper.com) >> * Michelle Beard (mbe...@draper.com) >> * Robert Foley (rfo...@draper.com) >> * Kyle Finley (kfin...@draper.com) >> * Lewis John McGibbney (lewi...@apache.org) >> >> =3D=3D Affiliations =3D=3D >> * Draper >> * Joshua Poore (jpo...@draper.com) >> * Laura Mariano (lmari...@draper.com) >> * Clayton Gimenez (cgime...@draper.com) >> * Alex Ford (af...@draper.com) >> * Steve York (sy...@draper.com) >> * Fei Sun (f...@draper.com) >> * Michelle Beard (mbe...@draper.com) >> * Robert Foley (rfo...@draper.com) >> * Kyle Finley (kfin...@draper.com) >> >> * NASA JPL >> * Lewis John McGibbney (lewi...@apache.org) >> >> =3D=3D Sponsors =3D=3D >> >> =3D=3D=3D Champion =3D=3D=3D >> * Lewis McGibbney (NASA/JPL) >> >> =3D=3D=3D Nominated Mentors =3D=3D=3D >> * Paul Ramirez (NASA/JPL) >> * Lewis John McGibbney (NASA/JPL) >> * Chris Mattmann (NASA/JPL) >> >> =3D=3D Sponsoring Entity =3D=3D >> The Apache Incubator >> >> =3D=3D References =3D=3D >> >> Mariano, L. J., Poore, J. C., Krum, D. M., Schwartz, J. L., Coskren, W. >> D., >> & Jones, E. M. (2015). Modeling Strategic Use of Human Computer Interfac= es >> with Novel Hidden Markov Models. [Methods]. Frontiers in Psychology, 6. >> doi: 10.3389/fpsyg.2015.00919 >> Poore, J., Webb, A., Cunha, M., Mariano, L., Chapell, D., Coskren, M., & >> Schwartz, J. (2016). Operationalizing Engagement with Multimedia as User >> Coherence with Context. IEEE Transactions on Affective Computing, PP(99)= , >> 1-1. doi: 10.1109/taffc.2015.2512867 >> >> >> >> -- >> Lewis >> > > > > -- > *Lewis* > --=20 *Lewis* --001a113fd586ff025505356f2836--