Effects of Surface Characteristics on Alignment of Real
Effects of Surface Characteristics on Alignment of Real and Graphic Objects in Stereoscopic Augmented Reality Environments Ming Hou Ergonomics in Teleoperation and Control (ETC) Laboratory Department of Mechanical and Industrial Engineering University of Toronto January 6th, 2003 3D Measurement in Unmodelled World Virtual Tape Measure Problem: Unknown relationship between real and virtual objects in Augmented
Reality displays Research on Real World Targets Virtual Pointer LINE Video Image Cylinder AREA VOLUME Hemisphere Pseudo-Transparency Phenomenon Real Object (in Video) Virtual pointer behind real surface Virtual pointer in front of surface
Transparency Effect Breakdown of Fusion + Fusion Conflict Conflict between Binocular Disparity and Occlusion Cues Virtual pointer behind real surface Virtual pointer in front of surface Fuse Real Surface Fuse Virtual Pointer Theory of Surface Interaction: Fusion Breakdown depends on Texture Density Virtual pointer behind real surface Virtual pointer in front of surface Low Texture Density High Texture Density
Research Motivation Is this conflict really significant? If yes, can it be used as an extra cue for detecting interactions between real and virtual objects, and thus locating the real objects in AR environment more easily and accurately? Main Hypotheses #1 More accurate to indicate position on (curved) surface with high texture density (HTD) than with low texture density (LTD) HTD better than LTD
Main Hypotheses #2 Orientation of observer relative to target surface will have influence Top View Top View Real Surface Stereo Camera Real Surface Stereo Camera Virtual Pointer Virtual Pointer Centre different from Off-centre Main Hypotheses #3 Form of Virtual Pointer (VP) will have impact on alignment
performance LINE AREA VOLUME Main Hypotheses #4 Binocular disparity (i.e. crossed vs 0 vs uncrossed) will affect alignment performance C ND UC No disparity (ND) > Crossed (C) or Uncrossed (UC) Experimental Investigation of AR Surface Effects
(Measurement) #1 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Orientation Placement Error between perceived target and its actual position on real surface #2 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Form
Placement Error + Confidence Rating + Preference Rating #3 5x5 Texture Density Surface Orientation Placement Error + Angular Error between estimated normal and real surface normal Subjective Comparisons Ease of Use Transparency Ease of Fusion 6 Images
(15 paired comparisons) Texture Density High Low Virtual Pointer Form LINE AREA VOLUME Methodology Factorial Expt. # Design Independent
Variables Dependent Variables (Measurement) Expt. #1 Texture Density Surface Orientation 2x2x3x3 Binocular Disparity VP Orientation Placement Error between perceived target and its actual position on real surface Expt. #2 Texture Density
Surface Orientation 2x2x3x3 Binocular Disparity VP Form Placement Error + Confidence Rating + Preference Rating Expt. #3 5x5 Texture Density Surface Orientation Placement Error + Angular Error between estimated normal and real surface normal Placement Error vs Texture Density - 1
M ean Placem en t Erro r (cm ) VP Placem ent Error vs Texture Density (Error Bar = 95% CI, F(1,11) = 11.14, P = 0.007) Farther 1.6 1.2 0.8 0.4 0 -0.4 \ Closer \ Cylinder Surface High
\ \ Low Texture Density Main Experimental Results - 1 M e a n P la c e m e n t E r r o r ( c m ) Placement Error vs Texture Density - 2 Placement Error along Surface Normal vs Texture Density (Error Bar = 95% CI, F(4,40) = 41.34, P < 0.001) 1.6 1.4 1.2 1 0.8 0.6
0.4 0.2 0 0 10 20 30 40 Texture Density (%) Main Experimental Results - 2 Placement Error vs Surface Orientation - 1 M e a n P la c e m en t E rro r (c m ) VP Placem e nt Error vs Targe t Pos ition (Error Bar = 95% CI, F(1,11) = 98.19, P < 0.001)
Farther 1.6 1.2 0.8 0.4 0 -0.4 Closer Ce ntre Right Targe t Pos ition (relative to obs e rve r) Main Experimental Results - 3 M e a n P la c e m e n t E rro r ( c m ) Placement Error vs Surface Orientation - 2
12 Place m e nt Error along Surface Norm al vs Targe t Pos ition (Error Bar = 95% CI, F(4,40) = 11.90, P < 0.001) 2 1.8 1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0 3 4 5
1 2 3 4 5 Targe t Pos ition on the Sphe re Main Experimental Results - 4 Angular Error vs Surface Orientation M e a n A lt it u d e E rro r ( d e g re e ) Altitude Error vs Targe t Pos ition on He m is phe re (Error Bar = 95% CI, F(4, 40) = 36.72, P < 0.001)
Main Experimental Results - 5 Subjective Comparison Results w.r.t. Ease of Use and Ease of Fusion Volume VP better than other VPs, regardless of texture density w.r.t. Transparency Volume VP + Highly textured least transparent combination Main Experimental Results - 6 Summary of Results - 1 Texture Density High better than Low Surface Orientation Centre
different from Off-Centre Summary of Results - 2 VP Form VOLUME > AREA > LINE (subjective comparisons) Binocular Disparity C ND UC Conclusions - 1 Perceptual conflict does exist when real
and virtual objects interact in 3D AR environments (a model proposed) Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects Optimal density value for Random Dot texture pattern was found as Engineering Solution for accurate 3D measurement R-V Interaction Interaction Process Process Model Model R-V Perception Cognition / Decision Making (Stereo Matching) (Stereo Matching + Cue Conflict Resolution)
behind Fusion difficulty VP behind, or at, real surface? at Fused Image? No fusion difficulty at (Transparency) No Stereo Matching :
VP in front of real surface? VP behind, or at, real surface? behind Yes User Localisation Achieved Adjustment (Breakdown/Conflict) External World Display of virtual pointer (VP)
superimposed upon real object surface AR Display VP Controller Conclusions - 1 Perceptual conflict does exist when real and virtual objects interact in 3D AR environments (a model proposed) Perceptual conflict can be used as extra depth cue to indicate interaction between real and virtual objects Optimal density value for Random Dot texture pattern found as Engineering Solution for accurate 3D measurement Conclusions - 2 Target position does affect alignment task: centrally located targets benefit performance, but have disadvantage when along the line of sight
Volumetric stereo graphic cursor (more fusable features along three dimensions) is subjectively the most favoured VP Pseudo-transparency contributes literature of depth perception cues (shapefrom-texture and stereo) Implications for for AR AR Interface Interface Design Design Implications Random Dot Stereogram enhances 3D alignment performance Perceptual conflicts can be used as extra depth cue to detect real object position 3D VP better than other VPs Perceptual errors always exist Limitations Implementation
Display Mode : stationary display without motion parallax and motion perspective Binocular Disparity : confounded with size cue and resolution Scope VP Design: line thickness of wire-frame VP Texture Pattern: square Random Dot pattern Future Work/Impact Work/Impact -- 11 Future Near Term Research Projected lighting : more practical Motion parallax with Video-HMD Computational vision may alleviate some error (being investigated) Projected lighting with random dot
texture pattern Simulated Projector Stereo Cameras Future Work/Impact - 2 Long Term Interests Integration of Computer Assisted Object Detection in AR Displays See-through HMD AR for Dismounted Soldiers in the Battlefield (e.g., Perceptual Conflicts, Navigational Aids, etc.) Other Human Computer Interaction (HCI) Topics (e.g., Integration of Electronic Information with
Human-Machine Systems, etc.) Acknowledgement Dr. Julius Grodski at Defence Research & Development Canada (DRDC) Toronto, Prof. Allison B. Sekuler and Prof. Paul Milgram at University of Toronto Dr. Stephen Ellis at NASA and Prof. Stanley Hamstra at University of Toronto Natural Sciences and Engineering Research Council (NSERC) Doctoral Scholarship Institute of Robotics and Intelligent Systems (IRIS), Canada Ontario Graduate Scholarship (OGS) Experimental #1 Main Result P la c e m e n t E r r o r (c m ) Farther 1 0
VP Placem e nt Error vs Surface Texture (Error Bar = 95% CI, F(1,9) = 619.71, P< 0.001) \ \ Cylinder Surface \ -1 -2 -3 -4 -5 -6 -7 -8 -9 Closer
High Low Texture Density \ VP Form and Orientation in 1st Experiment : Vertical Diagonal Observer Horizontal Interaction in Experiment # 1 VP Placement vs Texture and Target Position Placement Error (cm)
(Error Bar = +/- 1SD, F(1,9) = 246.33, P< 0.001) 1 0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -10 -11 High Low Centre Right Angular Displacement of Target Normal
Interaction between Surface Texture (High vs Low) and Target Position (Center vs Right) Paired Comparison Result Mean Z score 0 1.15 1.71 2.26 3.16 1 2 34
5 6 0 0.25 0.89 2.97 3.82 4.37 6 5 4 Ease of Use Image # Transparency Image #
4 2.26 6 Measurement of Placement Error along Surface Normal Estimated Target Surface Normal Real Surface Target Positive Error Positive error shows the estimated target is inside the sphere along surface normal M e a n P la c e m e n t E r r o r ( c m ) Placement Error vs Texture Density for Experiments 2 and 3
Placement Error along Surface Normal vs Texture Density (Error Bar = 95% CI, F(4,40) = 41.34, P < 0.001) 1.6 Experiment 2 Experiment 3 1.4 1.2 1 0.8 0.6 0.4 0.2 0 0 10 20
30 Texture Density (%) 40 Definition of Angular Error Y Altitude Error : Angular distance between estimated normal TP and real surface normal TN N P ' Q
Azimuth Error : Angular Error between horizontal projection OR and OS ' T O S R ' '
Z X Angular Bias in Spherical Coordinate Y N E T Bias Area S Z X Angular bias tilted upwards from real surface normal (TN) Example of Distribution of Altitude and Azimuth
135 180 -20 -30 Azimuth Error (degree) Angular bias between estimated and real surface normal Ground Truth Measurements in Real Scene Cylinder Stimulus Side View Stereo Cameras Top
View Iron plate Cylinder Calibration object Real World Origin (0,0,0) Real Distance in Depth (Z) Registration Verification: Measurement of a Pin and a Cube Stereo Cameras Calibration Target Calibration Cube
P e r c e n t a g e o f D is t a n c e ( D ) f o r 1 P ix e l C h a n g e in H o r iz o n t a l D is p a r it y VP Resolution for Experiment 2 VPVP Resolution and Tests Re s olution (Pe rceAccuracy ntage of Re al Dis tance vs One Pixe l Error of V P in Dis play) 8.00% 7.00% 6.00% 5.00% 4.00% 3.00% 2.00%
1.00% 0.00% 0 1 2 3 4 Dis tance of Obje ct from Cam e ras (m ) 5 Retinal Disparity in Stereoscopic Display for One Pixel Separation (exaggerated) ZD Stereoscopic Display Monitor
C b Apparent position of point C, due to 1 pixel horizontal disparity e 2e P ) a b 2arctg ( ) 2arctg ( ) d 2d P = 1 pixel separation XD Viewers Eye a 2e
b /2 d Psychophysical Standard for Texture Density Control Spatial scale (size) Homogeneity (spatial regularity, density is approximately constant over the surface) Isotropy (no orientation bias, equally to be oriented in all directions) compression Practical Augmented Reality Example Virtual Tape Measure for Minimally Invasive Surgery Distance between point 1 and point 2 is 7mm
Coordinate of Point 1 (2.3, 14.7, 96.2) Coordinate of Point 2 (1.8, 14.4, 95.8)
Canada has 10 provinces ... Forests in Canada are a major natural resource with an abundance of timber, which is harvested in Canada to be used by its own people as well as traded with other countries around the world....
M27 The Dumbbell Nebula Photo by Richard Kinsey M57 The Ring Nebula Photo by Jerry Lodriguss Ursa Major, The Great Bear M97 The Owl Nebula Photo by Gary Breneman M97 Perseus M76 The Little Dumbbell Photo by Ken Crawford M76...
* * The equipment and areas closest to the patient are the most contaminated and considered the "Hot Zone". As you move further from the patient, surfaces are less contaminated. Starting with the bed will allow adequate contact time with...
Fossils! Pollen and foraminifera Trends in glacial-interglacial pacing There are important shifts over the last 2.75 million years in the dominance of different orbital cycles over the glacial cycles As background to this discussion: How has the intensity of the...
Assessments and Mitigation Plans are Reviewed by CIO or designate at MS A, B, and C, and other events to determine supportability. Procedures specified at high level, recommended risk assessment tasks included as an Appendix. Certification in U.S. and Host...
Young leaves may become dark green and twisted or misshapen Steely wool in sheep Photo 1 www.progressivegardens.com Photo 2 www.marc.asci.ncsu.edu Manganese toxicity Deficiency rare on Australian soils Toxicity more common on acid soils Older leaves with brown spots surrounded by...
MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. 5/1/2017 1:51 PM. A lap around Azure HDInsight and Cosmos DB: Open Source Analytics + NoSQL. Andrew Liu. Program Manager. Azure OSSA + NoSQL. P4010.
Ready to download the document? Go ahead and hit continue!