Adjunct Reality Systems

Introduction

Proposal for a wearable system providing a visual field overlay, synchronized with the wearer's head position and the geometric position of the outside world.

System Components

Eyepiece

Glasses with IR emitter, cameras (RGB + IR), accelerometers, and laser-projected visual displays, see the following for current technologies:

• WII Remote (IR camera, accelerometers)
• see-through video eyepieces [1]
• 'Spy' eyeglass cameras [2]

Pointing Device

Passive device with and IR reflective component. Used by the eyepiece to determine the user's pointing locations.

Multiple pointing locations are supported.

Processing System

Internet connected, wireless system. Should be light enough to fit in a hat/helm (with eyepiece attached, like a motorcyclist's "brain bucket" with attached glasses) or embedded in the glasses.

Battery life must be >8 hours.

Perceived response time to user motion (head or pointing device) must be under 20ms. Anticipatory computation is allowed to close the latency gap.

Positioning System

Accelerometers for small deltas, but most position information is provided by bar-coded IR reflective tape.

The IR emitter on the device 'blinks' at a pseudo random rate, allowing the removal of constant non-mapped IR sources (candles, incandescent lights, etc).

IR reflective tape on walls and surfaces:

• Passive
• Cheaper than active positioning devices
• Can embed bar codes or other graphic elements
• Can be filtered out of the visual overlay.

System Use Cases

Covert Office

You can have a completely empty office, where even external physical documents are 'scanned in' by the user's cameras.

All user information is 'pasted' on blank walls and desks, and only those who are wearing the system and have the appropriate permissions can see the documents.

Game Room

Full-room gaming in a warehouse space, where all of the reality is replaced by the imaging overlay.

Crime Scene Recovery

Tag and photograph each item in the scene, with markers automatically generated and placed in the user's FOV. The entire crime scene can be reconstructed from the walk through, and the processing software can notify the user of missed areas on-scene.

Transparent Cockpit

Could be used by pilots to replace floor and rear views with external camera views, artificial horizons and landscapes, etc.

If the system fails, overlay is removed.

Prototype Project

Human Resources

All human resources should be planned as full-time employees, to be able to meet project deadlines.

• One manager/administrator
  • Project management, payment handing, etc.
  • Must have accounting credentials
• One hardware engineer
  • Hardware design, outsourcing manufacture, etc.
• One firmware engineer
  • Implements custom ASICs and firmware for embedded hardware devices
• One user interface software engineer
  • Design and implement ARS user interface
• One 3D spatial interface engineer
  • Implement 3D tracking and pointer prediction software using IR portion of the cameras
• One image processing software engineer
  • Implement RGB image acquisition and stitching over the 3D environment.
  • Will use IR tags in the environment to simplify the process.

Engineering Resources

• Hardware lab
  • Solder station (scope, picks, etc) for rework
  • Logic analyzers
  • Data recorders
• NAS Storage
  • 100TB (test images, video, etc)
• Compute farm
  • 20 2Ghz/4Gb compute blades, in a parallel cluster
    • For automated testing, and simulation of future compute cores for the final product

Project Planning

Project planning shall be milestone based.

Each milestone will have a budget and a deadline. If that budget, plus 50%, is overrun, the project will be canceled. If the milestone deadline is not reached by the agreed upon date plus one month, excepting unusual circumstances, the project will be canceled.

M1: Planning

Cost: $0

• Secure funding for the expected prototype completion costs plus 50% overrun.
• Secure employee agreements for all required positions.
• All employees must agree to and sign off on the prototype requirement and architecture specifications.
• All employees must agree to and sign off on prototype milestone exit requirements

M2: Software Prototype

Cost: TBD

• Secure facilities for hardware lab, NAS, and compute farm
  • NOTE: This should be a shared access area, that all employees have access to.
• Complete the following administrative activities:
  • Purchase and configure the NAS storage
  • Purchase and configure the compute farm
  • Set up the development/corporate Wiki
  • Set up the bug tracking system
  • Keep M2 on track.
• Complete the following hardware activities:
  • Investigate what vendors are immediately available for all architected hardware components.
  • Support software prototype investigation
• Complete the following firmware activities:
  • Support software prototype investigation
• Complete the following software activities:
  • Using an IR emitter and camera, demonstrate the ability to track multiple pieces of IR reflective tape in the environment. Tape may be made 'unique' by encoding them with barcodes of non-reflective regions. (bullseye barcodes highly suggested)
  • Using accelerometers mounted to a conventional camera, demonstrate the ability to keep a 0.5 meter virtual sphere overlayed into the center of the room as the camera moves around it. Latency issues and "drift" are acceptable at this milestone, but a plan must exists for reducing them to acceptable limit by the next milestone.
  • Demonstrate a prototype GUI (using an existing multi-touch interface such as WII remotes or an iPhone) that corresponds to the prototype specification.

M3: Hulking Prototype

Goal: Demonstrate all the technology, even if it isn't wearable by a mere mortal. Cost: TBD

• Administrative activities:
  • Ensure all employees are paid for milestone M2
  • Keep M3 on track
  • Discover investor opportunities (trade shows, angel investors, etc)
• Hardware activities:
  • Schematics for wearable prototype
  • Select manufacturer for wearable prototype
  • Select vendors for wearable prototype
  • Build 'hulking prototype' out of commercial, off the shelf components
  • Mount 'hulking prototype' on a articulated mannequin if not light enough to be wearable
• Software activities
  • Get all software components to run on the 'hulking prototype'
  • Demonstrate all of the 'Core Demonstrations' from the ARS Prototype Specification

M4: Investor Prototype

Goal: Demonstrate a wearable version of the technology. Cost: TDB

• Administrative activities:
  • Ensure all employees are paid for milestone M3
  • Keep M4 on track
  • Line up investors for M5 activites
• Hardware activities:
  • Source, test, and assemble wearable prototypes (minimum 10)
  • Set up agreements for full external manufacturing.
• Software activities:
  • Get all software components to run on the wearable prototype
  • Demonstrate all of the 'Core Demonstrations' from the ARS Prototype Specification
  • Demonstrate at least two of the 'Market Demonstrations' from the ARS Prototype Specification

M5: Production Run Investment

Goal: Generate enough investment for a market segment testing production run of the prototypes Cost: TDB

• Administrative activities:
  • Ensure all employees are paid for milestone M4
  • Keep M5 on track
  • Generate enough investment for a production run of x1000 units.
• Hardware activities:
  • Validate external manufacturing
  • Begin work on Rev 2 hardware designs, using lessons learned from Rev 1
• Software activities:
  • Fully debug the system for release to the marketing tests.

M6: ???

To be determined by market forces.