Python Packages and Libraries I Use Often:

• • SciPy

  • NumPy

  • Pillow

  • Django, or Bottle/Flask for lighter weight web

  • SQLAlchemy

  • matplotlib (sometime Seaborn or Bokeh though)

  • Twisted

  • Kivy for stand alone or mobile apps

  • nltk

  • nose

  • SymPy

  • Pattern

  • Pandas

  • Machine Learning Specific

    • Keras, scikit-learn, TensorFlow

  • Cython

JavaScript Packages, Frameworks, and Libraries I Use Often:

• • React (interested in moving towards Polymer though)

  • Node.js

  • Underscore

  • MathJS

  • d3.js

  • Chart.js

  • lodash

  • webpack

  • Sass

from DARPA eXplainable AI Program - COGLE, work done at PARC with Mark Stefik, Bob Krivacic, Jacob Le, and Les Nelson

Klarigo (Esperanto for explanation) provides an explanation interface for users to gain insight into and to develop mental models about choices between potential future drone mission performance outcomes the underlying AI system is making in final performance selection. 

This work is addressing explanation need questions for:

• What options is the AI considering? Why?

• What is the AI not considering? Why?

from DARPA eXplainable AI Program - COGLE, work done at PARC with Mark Stefik, Bob Krivacic, Jacob Le, and Les Nelson

Exigisi (Greek for explanation) provides an interface for an UAS (Unmanned Aerial System) machine learned drone to explain its decisions made along a mission flight. The mission is segmented into different parts each with a different goal and consideration of salient factors. The user can interactively examine the flight or a set of flights interrogating the desiderata of the drone and subsequent task behaviors. Examinations across sets of flights reveals patterns of behavior and drone stimulus-response preferences.

This work is addressing explanation need questions for:

• What actions was the AI considering? Why?

• What did the AI consider to be important at the time of each action decision? This is known as salience.

from DARPA eXplainable AI Program - COGLE, work done at PARC with Jacob Le

Micro Air Vehicle Simulation comes in a variety of forms. Named after the MAVLink protocol that defines the communication core, MAVSim started as a wrapper around ArduPilot, but has evolved into several version that allow the spectrum of use from actual drone to highly sped-up simulated drones suitable for Deep Reinforcement Learning. The PARC XAI work is exclusively performed in the autonomy domain using MAVSim.

More about MAVSim

Funded by PARC / XIG Explore & Incubate (2012-2014)

Fittle+ Project funded by NSF / NIH Smart and Connected Health (2013-2017)

NutriWalking Project funded by PARC / XIG Explore (2015-2016)

Role: Lead Engineer / Designer / Architect / Researcher in a 6-9 person team

Tech: iOS native app (Obj-C), Android native app, Django/Python Backend

Fittle is a mobile behavior change platform and health challenge exchange. Users choose from a variety of health challenges to accomplish goals via a unique intelligent coaching agent and personalized, engaging user experiences. Fittle transitioned out of research to commercial elements at Xerox at the beginning of 2015.

Fittle+ is the variant of Fittle used to engage the Kaiser-Permanante Hawaii population as part of an NSF/NIH SMart and Connected Health grant.

NutriWalking is a fork post commercial split of Fittle used for continued research on adaptive coaching.

Project funded by PARC / NSF / NSF REU (2013-2016)

Role: Lead Engineer / Designer / Architect / Researcher in a 2-3 person team

Tech: iOS native app (Obj-C), Parse Backend

Mobile Surveys or Surveying the Mob is a project that explores the utility of using visual answers to questions in order to gain more truthful responses. This tool was used to deliver the Ten Item Personality Inventory (TIPI) and compared experimentally to text and other visual forms. MobSurv experiments revealed distinct impacts in response due to context, which could be appropriately leveraged, and an increase in enjoyment and interaction time for users.

Funded by XIG Explore (2013-2014)

Role: Lead Engineer / Designer / Architect / Researcher in a 3-5 person team

Tech: Android native app, Django/Python Backend

Crowdsourcing is emerging as an important work paradigm, enabling low-cost execution of large numbers of small, online tasks that can be accomplished independently. Many business problems, however, (1) require action in the physical world, and (2) require coordination between individual tasks. This project will develop a new crowdsourcing paradigm, called CrowdOrchestration, to address such problems. Our approach is based on the growth of mobile social media.

The focus of this project is on emergent tasks that have a full or partial real-world, physical component, such as transportation of business documents that are either sensitive or require real signatures quickly between offices (e.g., biz to lawyer or banker). Where courier services often work by hub and spoke systems with often deep queues, it could be more efficient and cost effective to pay (partial) gas costs for someone who is already making a similar trip between the desired locations (or very close-by). Those individuals could be the crowd workers in the proposed system. Other business cases could involve other highly-coordinated tasks such as physical inspections (e.g., property, machinery, items at auction), use of humans in crowd control at big events (e.g., Black Friday sales), use of humans as mobile sensor platforms for identifying crowd sentiment at events and activities, and so forth.

Funded by DARPA (2007-2011)

Role: Research Supervisor for D. Hunter Hale

Tech: C++ using the Computational Geometry Algorithms Library

A novel algorithm developed for decomposing world-space into arbitrary sided high-order polyhedrons for use as navmeshes or other techniques requiring 3D world spatial decomposition. The 3-dimensional Volumetric Adaptive Space Filling Volumes (VASFV) algorithm works by seeding world-space with a series of unit cubes. Each cube is then provided with the opportunity to grow to its maximum extent before encountering an obstruction. VASFV implements an automatic sub- dividing system to convert cubes into higher-order polyhedrons while still maintaining the convex property. This allows for the generation of navigation meshes with high degrees of coverage while still allowing the use of large navigation regions—providing for easier agent navigation in virtual worlds. Compared to the Space-filling Volumes and Automatic Path Node Generation navigation mesh decomposition methods, VASFV provides more complete coverage and a less complex navigation mesh. 

PASFV is the planar (2D) version of VASFV.

DASFV, as demonstrated in the video to the left, is the dynamic, realtime decomposition of the environment version. This can be performed in both 2D (planar) and 3D (volumetric).

Funded by NSF (2002-2006) grant ITR 0121297.

Role: Chief Scientist for a rotating group of 4-6 faculty and 8-18 student researchers

Tech: C++ using CORBA across a network of machines and FLTK for interfaces

The MavHome Smart Home project is a multi-disciplinary research project at the University of Texas at Arlington focused on the creation of an intelligent home environment. Our approach is to view the smart home as an intelligent agent that perceives its environment through the use of sensors, and can act upon the environment through the use of actuators. The home has certain overall goals, such as minimizing the cost of maintaining the home and maximizing the comfort of its inhabitants. In order to meet these goals, the house must be able to reason about and adapt to provided information.

American Airlines

Role: Software lead for a 5 person cross-functional  development team 

The Enterprise Data Publisher (EDP) took Passenger Name Records (PNRs) from the batch processing on magnetic tape era to real time integration with the SABRE system. It was written in C++ and utilized a CORBA connection to the SABRE system. Internally it allowed subscribers to register the frequency, buffer size, and delivery format (e.g., MQ, TXT, XML, CORBA Object, etc.) including filtering/processing rules for information delivery.

Motorola Global Telecommunications Solution Sector

Role: Software engineer responsible for requirements analysis, solution design, implementation, and testing. Promoted to Channel Element Services (CES) Team Lead.

Common Base Transceiver Station Platform (CBTSP)

As part of the CBTSP greater team, we developed a comprehensive requirements document to share with our call processing customers that reflected a more detailed view of the 3GPP. Using our CBTSP requirements document, we developed high and low level engineering design documents. As CES team lead, we implemented and delivered the 1.0 version of Channel Element Services for CBTSP 1.0 a month early then led the code review teams for dozens of software parts for the other 3 major components of CBTSP helping the entire project to release on time. CBTSP was the first middleware C++ project at Motorola.

Some parts of the original Channel Element Services live on in every Motorola base station in use today. This is software that has been used in multi-trillions of mobile calls and data transactions globally. 

Lockheed-Martin Missiles and Fire Control, Virtual Reality Systems Integration Lab

Role: Software engineer responsible for requirements analysis, solution design ,implementation, and testing in support of high-end military simulation systems .

Future Scout was an reconfigurable engineering simulator consisting of multiple pods (one for each person in a vehicle) that allowed for experimentation and mission rehearsal of different vehicle configurations. This was used by the US and UK Armies to determine the actual requirements for scout vehicles as part of their future force initiatives.

I worked specifically on software involving laser illumination and targeting, designed hardware cable integration and controller management rack mounted units, and develop low-level ADIO drivers for custom controllers. 

Delivered a full engineering reconfigurable simulator for ground vehicles to the U.S. Army and to the U.K. Lockheed-Martin Research Organization in Farnborough, England (for use by the UK Army).

Funded by Shadow Corporation (2011-2012)

Role: Technical and Research Advisor

The save energy initiative will engage our city in reducing energy consumption by 20%, leveraging new SmartGrid technology and interactive digital displays to make energy usage more salient and connected to the everyday lives of local citizens. As part of this effort, the companies in a couple dozen office buildings in the city have agreed to share their energy usage information with the project, and many of these buildings will house digital displays that will have near real-time information about current and recent past energy consumption. We have built Greener Challenge, a mobile and web-based game to motivate citizens to report the actions they’ve taken each day to save energy and contribute to the city's goals. Through the Greener Challenge interface, city professionals register as part of a building or cooperative group, and can challenge one another to take certain green actions such as turning off lights, carpooling, setting the thermostat closer to the outside temperature, or bringing their own coffee cups to work. When participants make energy-conscious decisions and report their actions online or through smart phones, they earn garden gold, and can use it to arrange or add items to their own Greener Challenge garden. Players can enter snapshots of their gardens to the Greener Challenge Showcase, where they can rate and comment on one another’s gardens. Actions reported to Greener Challenge will be tallied along with those taken via the building kiosks in the City, and popular Greener Challenge gardens may sometimes be displayed on the central displays in city buildings.

Funded by DARPA (2007-2011)

Role: Technical and Research Advisor for Fritz Heckel

BehaviorShop: An Intuitive Interface for Interactive Character Design 

Artificial Intelligence for interactive characters is held back by the difficulty of actually creating those characters. There is a need for intuitive tools which simplify the process of building complex AI characters. In this paper, we present BehaviorShop, a tool for building interactive characters, and the results of a study to test the effectiveness and ease of use of this tool. The majority (80%) of test subjects were able to successfully create a complex character using BehaviorShop. Our results show success with this interface, and the advantage of using the subsumption architecture as a paradigm for character creation. 

Funded by DARPA (2007-2011)

Role: Technical and Research Advisor

First/third-person training simulations in virtual environments have become increasingly used; however, authoring intelligent virtual agents to populate these environments presents a large authorial burden. Our work focuses on building tools to enable rapid creation of intelligent agents for first/third-person game-like environments that enable users with no programming knowledge to develop interactive agents. This is made possible using an intuitive agent architecture known as behavior-based control combined with a user interface employing natural language-like agent specification and an interactive testing during agent development. We present the results of a study indicating that users with no programming experience can successfully design agents using our tool — defined as creating an agent that would carry out at least 80% of role-specific baseline behaviors — after only minimal training in the interface. 

Role: Technical and Research Advisor for Priyesh Dixit

The Common Games Understanding and Learning Toolkit

Creating an interactive 3D environment can be challenging, in particular because every user (player) will interact with the environment in their own unique manner. Gaining feedback through playtesting is a common method for evaluating an interactive experience. However, the information gathered from playtesting is often subjective and hard to interpret. Visual representation of logged game player data can provide insight in order to discover patterns and behavior that would otherwise be hidden. In this paper, we explored the use of visual data mining and automated data processing to process the logged player data and present the results of finding five unique phenomena in our dataset of 3079 players. These phenomena include pirouettes, flusters, jumpers, learning, and emergent behavior. Using the visual data mining process, we can find occurrences of such behavioral patterns in order to understand player behavior and improve the interactive experience. 

Funded by DARPA Transfer Learning program

Role: Technical and Research Advisor

We extended the open source Quake 3 Arena code base released by Id Software to include modification friendly code segments, art asset blueprints, external agent control and perception interfaces, game interaction logging, and a tool set that allows the creation of test beds for artificial agents. Our current modification work in the Urban Combat Testbed using the Q3 Mod Base is currently being used to study transfer learning in human and agent players and as an AI teaching tool.

UCT was in use in over six major US universities to develop research agents in the exploration of transfer learning which can bootstrap agent knowledge from a source domain to a target domain reducing the amount of learning necessary by an agent and improving solution convergence time. Human player and agent work is being performed using UCT. UCT is also a part of NRLs TIELT.