基于可见光逆反射通信的移动增强现实定向交互

李景聿; 陈拓潮; 李明哲; 徐栩海; 张铖; 徐子晨; 刘譞哲; 黄罡; 冯云; 许辰人

doi:10.12263/DZXB.20220677

您当前的位置：

首页 >

文章列表页 >

基于可见光逆反射通信的移动增强现实定向交互

学术论文 | 更新时间：2025-12-11

- 基于可见光逆反射通信的移动增强现实定向交互
- Directional Interaction for Mobile Augmented Reality Based on Visible Light Backscatter Communication
- 电子学报 2024年52卷第11期页码：3643-3656
- 作者机构：
  
  1.北京大学计算机学院，北京 100871
  2.华盛顿大学计算机科学与技术学院，美国西雅图 98195
  3.卡内基梅隆大学人机交互研究所，美国匹兹堡 15213
  4.康奈尔大学信息科学学院，美国伊萨卡14853
  5.南昌大学数学与计算机学院，江西南昌 330031
  6.北京大学第三医院，北京 100191
- 作者简介：
  
  [ "李景聿男，1996年出生，山东烟台人．本科毕业于北京邮电大学，现为北京大学计算机学院博士一年级.主要研究方向为移动增强现实系统、光逆反射通信. E-mail： motren909@pku.edu.cn" ]
  [ "陈拓潮男，1999年出生，浙江温州人．本科毕业于北京大学，现为华盛顿大学计算机科学与技术学院博士三年级.主要研究方向为无线通信系统. E-mail： ctc1998@uw.edu" ]
  [ "李明哲男，1995年出生.本科毕业于多伦多大学，现为卡内基梅隆大学人机交互研究所博士四年级.主要研究方向为人机交互. E-mail： mingzhe2@andrew.cmu.edu" ]
  [ "徐栩海男，1995年出生.本科毕业于清华大学，现为华盛顿大学信息学院博士五年级.主要研究方向为基于传感器和机器学习的行为感知. E-mail： xuhaixu@uw.edu" ]
  [ "张铖男，1985年出生.现为康奈尔大学计算与信息学院助理教授.主要研究方向为人机交互和感知.E-mail： chengzhang@cornell.edu" ]
  [ "徐子晨男，1985年出生.现为现任南昌大学数学与计算机学院副院长、计算中心主任.主要从事包括数据密集计算、智能计算、高能效计算及分布式数据存储等数据系统软件可靠性及可持续性等方面的教研工作. E-mail： xuz@ncu.edu.cn" ]
  [ "刘譞哲男，北京大学计算机学院副教授、博士生导师.主要研究方向为面向移动计算、大数据、服务计算等网络化应用模式的新型系统软件.中国电子学会会员编号：E190015296M. E-mail： liuxuanzhe@pku.edu.cn" ]
  [ "黄罡男，1975年10月生，湖南人.北京大学计算机学院副教授、博士生导师.主要研究方向为自适应软件计算反射原理、运行时模型驱动方法、自适应系统的软件定义方法等.中国电子学会会员编号：E190013031S. E-mail： hg@pku.edu.cn" ]
  [ "冯云女，北京大学第三医院主任医师、博士生导师.主要研究方向为角膜病、角膜内皮移植、眼外伤前节重建、白内障玻璃体术后各类眼表疾病. E-mail： fengyun@bjmu.edu.cn" ]
  [ "许辰人男，1985年11月生，上海人.北京大学计算机科学技术系“博雅青年学者”长聘副教授、软硬件协同体系结构（SOAR）实验室主任.主要研究方向为高速无线网络体系架构、低功耗可见光通信系统和情感计算应用.中国电子学会会员编号：E190026855S. E-mail： chenren@pku.edu.cn" ]
- 基金信息：
  
  国家自然科学基金(62022005;62061146001)
- DOI：10.12263/DZXB.20220677
  中图分类号： TP399;TN929
- 收稿：2022-06-12，
  
  修回：2022-08-15，
  
  纸质出版：2024-11-25
- 稿件说明：
移动端阅览
李景聿, 陈拓潮, 李明哲, 等. 基于可见光逆反射通信的移动增强现实定向交互[J]. 电子学报, 2024, 52(11): 3643-3656.

LI Jing-yu, CHEN Tuo-chao, LI Ming-zhe, et al. Directional Interaction for Mobile Augmented Reality Based on Visible Light Backscatter Communication[J]. Acta Electronica Sinica, 2024, 52(11): 3643-3656.
李景聿, 陈拓潮, 李明哲, 等. 基于可见光逆反射通信的移动增强现实定向交互[J]. 电子学报, 2024, 52(11): 3643-3656. DOI：10.12263/DZXB.20220677

LI Jing-yu, CHEN Tuo-chao, LI Ming-zhe, et al. Directional Interaction for Mobile Augmented Reality Based on Visible Light Backscatter Communication[J]. Acta Electronica Sinica, 2024, 52(11): 3643-3656. DOI：10.12263/DZXB.20220677

摘要

移动增强现实应用中，用户频繁地与环境中不同的智能物体交互完成任务，其完成效率和用户体验由所采用的定向交互技术决定.然而，从交互手段上来说，现阶段定向交互依赖于Wi-Fi、BLE等无线技术，其信号全向传播的特点使其不能利用用户的空间环境（即位置和方向）来缩短互动时间，带来了不必要的时间成本；从交互界面上来说，现阶段基于视觉的界面存在可靠性低、扩展性低等问题，进一步限制了系统的通用性.本文介绍了 RetroAR——基于可见光逆反射通信的定向交互系统.RetroAR利用了光的定向传播特性来保留用户的空间环境，并依靠逆反射通信实现用户与目标设备之间无连接的快速定向交互.系统实验表明，RetroAR最远可在4 m距离支持100°视角的交互，同时实现厘米级的六自由度（6-Degrees Of Freedom，6-DoF）三维跟踪.用户研究表明，与基于Wi-Fi的解决方案相比，RetroAR将非接触式控制的交互时间减少了2倍，并且具有更好的用户体验.RetroAR借助可见光逆反射通信来利用用户空间环境，保持交互过程中的直观性.用户可以“所指即所控”的方式与多个目标进行互动，实现类似自然交互的快速定向交互.

Abstract

In mobile augmented reality applications

users interact with smart objects in proximity to finish collaboration or interaction tasks

whose efficiency and user experience are determined by the underlying directional interaction technology. However

current directional interaction technologies at this stage are inefficient. For interaction means

they rely on wireless technologies such as Wi-Fi and BLE

which propagates omni-directionally and thus cannot use the user's spatial context (i.e.

location and direction) to shorten the interaction time

bringing unnecessary effort. In terms of interaction interface

current vision-based interfaces suffer from low reliability and low scalability

which further limits the adaptability and efficiency of the system. To address this issue

we developed RetroAR: an optical-sensing solution that leverages visible light backscatter communication to serve for directional interaction with intelligent objects on commodity smartphones. RetroAR exploits the directional propagation property of light to preserve the user's spatial context

which enables fast connection-free directional interaction between the user and the target devices. RetroAR instruments objects with custom retro-reflective markers called ViTags. When users interact with these smart objects

these ViTags are used to communicate with the camera on the mobile reader by backscattering the flashlight beams. We first conducted a system evaluation which showed that RetroAR could work reliably at a distance up to 4 meters and a view angle up to 100 degrees

and is able to achieve 6-DoF 3D tracking with an error as low as 1 cm on translation and 4.7 degrees on rotation. To evaluate how our system performs in terms of interaction

we then conducted a user study with 12 participants

which demonstrated that RetroAR improves the interaction time of MAR contactless control by at least two times compared to Wi-Fi-based solutions. RetroAR utilizes the user's spatial environment with visible light backscatter communication to maintain the intuitiveness of the interaction process. Users can interact with multiple targets in a point-and-control manner

which reduces interaction costs and brings a natural and intuitive interaction experience.

关键词

Keywords

references

林倞 , 杨珂 , 王涌天 , 等 . 移动增强现实系统的关键技术研究 [J ] . 中国图象图形学报 , 2009 , 14 ( 3 ): 560 - 564 .

LIN L , YANG K , WANG Y T , et al . Key issues study for mobile augmented reality system [J ] . Journal of Image and Graphics , 2009 , 14 ( 3 ): 560 - 564 . (in Chinese)

陈靖 , 王涌天 , 郭俊伟 , 等 . 增强现实技术在智能手机上的应用 [J ] . 电子科技大学学报 , 2010 , 39 ( S1 ): 80 - 84 .

CHEN J , WANG Y T , GUO J W , et al . Application of augmented reality technology in smart phone [J ] . Journal of University of Electronic Science and Technology of China , 2010 , 39 ( S1 ): 80 - 84 . (in Chinese)

王同聚 . 虚拟和增强现实(VR/AR)技术在教学中的应用与前景展望 [J ] . 数字教育 , 2017 , 3 ( 1 ): 1 - 10 .

WANG T J . Application and prospect forecast of VR/AR technology in education [J ] . Digital Education , 2017 , 3 ( 1 ): 1 - 10 . (in Chinese)

汤晓颖 , 区雪兰 , 谢丹妮 . 基于AR技术的广府家具"纸上博物馆" 设计应用与研究 [J ] . 包装工程 , 2018 , 39 ( 4 ): 115 - 122 .

TANG X Y , OU X L , XIE D N . Design application and research of "museum on paper" of Cantonese furniture based on AR technology [J ] . Packaging Engineering , 2018 , 39 ( 4 ): 115 - 122 . (in Chinese)

PERLI S D , AHMED N , KATABI D . PixNet: Interference-free wireless links using LCD-camera pairs [C ] // Proceedings of the Sixteenth Annual International Conference on Mobile Computing and Networking . New York : ACM , 2010 : 137 - 148 .

DE FREITAS A A , NEBELING M , CHEN X ' , et al . Snap-to-it: A user-inspired platform for opportunistic device interactions [C ] // Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems . New York : ACM , 2016 : 5909 - 5920 .

DE FREITAS A A , NEBELING M , RANITHANGAM A S K K , et al . Bluewave: Enabling opportunistic context sharing via bluetooth device names [C ] // Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems . New York : ACM , 2016 : 38 - 49 .

LI J T , LIU A L , SHEN G B , et al . Retro-VLC: Enabling battery-free duplex visible light communication for mobile and IoT applications [C ] // Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications . New York : ACM , 2015 : 21 - 26 .

XU X Y , SHEN Y , YANG J R , et al . PassiveVLC: Enabling practical visible light backscatter communication for battery-free IoT applications [C ] // Proceedings of the 23rd Annual International Conference on Mobile Computing and Networking . New York : ACM , 2017 : 180 - 192 .

HEUN V , KASAHARA S , MAES P . Smarter objects: Using AR technology to program physical objects and their interactions [C ] // CHI '13 Extended Abstracts on Human Factors in Computing Systems . New York : ACM , 2013 : 961 - 966 .

TANG L Z W , ANG K S , AMIRUL M , et al . Augmented reality control home (ARCH) for disabled and elderlies [C ] // 2015 IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP) . Piscataway : IEEE , 2015 : 1 - 2 .

JO D , KIM G J . ARIoT: Scalable augmented reality framework for interacting with Internet of Things appliances everywhere [J ] . IEEE Transactions on Consumer Electronics , 2016 , 62 ( 3 ): 334 - 340 .

HUO K , CAO Y Z , YOON S H , et al . Scenariot: Spatially mapping smart things within augmented reality scenes [C ] // Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . New York : ACM , 2018 : 1 - 13 .

韩玉仁 , 李铁军 , 杨冬 . 增强现实中三维跟踪注册技术概述 [J ] . 计算机工程与应用 , 2019 , 55 ( 21 ): 26 - 35 .

HAN Y R , LI T J , YANG D . Overview of 3D tracking registration technology in augmented reality [J ] . Computer Engineering and Applications , 2019 , 55 ( 21 ): 26 - 35 . (in Chinese)

REKIMOTO J . Matrix: A realtime object identification and registration method for augmented reality [C ] // Proceedings . 3rd Asia Pacific Computer Human Interaction . Piscataway : IEEE , 1998 : 63 - 68 .

CHOY C B , STARK M , CORBETT-DAVIES S , et al . Enriching object detection with 2D-3D registration and continuous viewpoint estimation [C ] // 2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway : IEEE , 2015 : 2512 - 2520 .

GAO Q H , WAN T R , TANG W , et al . Object registration in semi-cluttered and partial-occluded scenes for augmented reality [J ] . Multimedia Tools and Applications , 2019 , 78 ( 11 ): 15079 - 15099 .

王笛 , 胡辽林 . 基于双目视觉的改进特征立体匹配方法 [J ] . 电子学报 , 2022 , 50 ( 1 ): 157 - 166 .

WANG D , HU L L . Improved feature stereo matching method based on binocular vision [J ] . Acta Electronica Sinica , 2022 , 50 ( 1 ): 157 - 166 . (in Chinese)

GAO Q H , WAN T R , TANG W , et al . A stable and accurate marker-less augmented reality registration method [C ] // 2017 International Conference on Cyberworlds . Piscataway : IEEE , 2017 : 41 - 47 .

MICROSOFT . Microsoft hololens | mixed reality technology for business [EB/OL ] . ( 2020 )[2022 ] . https://www.microsoft.com/en-us/hololens/ https://www.microsoft.com/en-us/hololens/ .

GOOGLE . ARCore - google developers [EB/OL ] . ( 2020 )[2022 ] . https://developers.google.com/ar https://developers.google.com/ar .

GARRIDO-JURADO S , MUÑOZ-SALINAS R , MADRID-CUEVAS F J , et al . Automatic generation and detection of highly reliable fiducial markers under occlusion [J ] . Pattern Recognition , 2014 , 47 ( 6 ): 2280 - 2292 .

KAWSAR F , RUKZIO E , KORTUEM G . An explorative comparison of magic lens and personal projection for interacting with smart objects [C ] // Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services . New York : ACM , 2010 : 157 - 160 .

JUNG J D A , VOGEL D . Methods for intentional encoding of high capacity human-designable visual markers [C ] // Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . New York : ACM , 2018 : 1 - 10 .

COSTANZA E , HUANG J . Designable visual markers [C ] // Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . New York : ACM , 2009 : 1879 - 1888 .

AHUJA K , PAREDDY S , XIAO R , et al . LightAnchors: Appropriating point lights for spatially-anchored augmented reality interfaces [C ] // Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology . New York : ACM , 2019 : 189 - 196 .

YANG J , LANDAY J A . InfoLED: Augmenting LED indicator lights for device positioning and communication [C ] // Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology . New York : ACM , 2019 : 175 - 187 .

SUN Y B , ARMENGOL-URPI A , REDDY KANTAREDDY S N , et al . Magic hand: Interact with IOT devices in augmented reality environment [C ] // 2019 IEEE Conference on Virtual Reality and 3D User Interfaces . Piscataway : IEEE , 2019 : 1738 - 1743 .

卢霆威 , 王泽平 , 刘梦 , 等 . 基于可见光通信技术的全双工以太网通信系统设计 [J ] . 电子学报 , 2022 , 50 ( 1 ): 45 - 53 .

LU T W , WANG Z P , LIU M , et al . Full-duplex indoor ethernet communication system based on visible light communication [J ] . Acta Electronica Sinica , 2022 , 50 ( 1 ): 45 - 53 . (in Chinese)

GUMMADI R , WETHERALL D , GREENSTEIN B , et al . Understanding and mitigating the impact of RF interference on 802.11 networks [J ] . ACM SIGCOMM Computer Communication Review , 2007 , 37 ( 4 ): 385 - 396 .

YANG J J . Apparatus and Method for Suppressing Interference Caused by Coexistence of WiMAX and Wi-Fi [M ] . New York : Google Patents , 2013 .

SHAO S H , KHREISHAH A , ELGALA H . Pixelated VLC-backscattering for self-charging indoor IOT devices [J ] . IEEE Photonics Technology Letters , 2017 , 29 ( 2 ): 177 - 180 .

AMIRI I S , ALI J . Data signal processing via Manchester coding-decoding method using chaotic signals generated by PANDA ring resonator [J ] . Chinese Optics Letters , 2013 , 11 ( 4 ): 41901 - 41904 .

TAO Q , ZHONG C J , LIN H , et al . Symbol detection of ambient backscatter systems with Manchester coding [J ] . IEEE Transactions on Wireless Communications , 2018 , 17 ( 6 ): 4028 - 4038 .

CANNY J . A Computational approach to edge detection [M ] // Readings in computer Vision . Amsterdam : Elsevier , 1987 : 184 - 203 .

SUZUKI S , BE K . Topological structural analysis of digitized binary images by border following [J ] . Computer Vision, Graphics, and Image Processing , 1985 , 30 ( 1 ): 32 - 46 .

DUDA R O , HART P E . Use of the Hough transformation to detect lines and curves in pictures [J ] . Communications of the ACM , 1972 , 15 ( 1 ): 11 - 15 .

GRAHAM R L . An efficient algorith for determining the convex hull of a finite planar set [J ] . Information Processing Letters , 1972 , 1 ( 4 ): 132 - 133 .

LEPETIT V , MORENO-NOGUER F , FUA P . EPnP: An accurate O(n) solution to the PnP problem [J ] . International Journal of Computer Vision , 2009 , 81 ( 2 ): 155 - 166 .

PERSSON M , NORDBERG K . Lambda twist: An accurate fast robust perspective three point (P3P) solver [C ] // Proceedings of the European Conference on Computer Vision . New York : ACM , 2018 : 318 - 332 .

WILLSON R G , SHAFER S A . Perspective projection camera model for zoom lenses [EB/OL ] . ( 2022 )[2022 ] . https://www.spiedigitallibrary.org/conference-proceedings -of-spie/2252/0000/Perspective-projection-camera-model-for-zoom-lenses/10.1117/12.169831.full https://www.spiedigitallibrary.org/conference-proceedings-of-spie/2252/0000/Perspective-projection-camera-model-for-zoom-lenses/10.1117/12.169831.full .

YANG X F , HUANG Y M , GAO F . A simple camera calibration method based on sub-pixel corner extraction of the chessboard image [C ] // 2010 IEEE International Conference on Intelligent Computing and Intelligent Systems . Piscataway : IEEE , 2010 : 688 - 692 .

LEPETIT V , MORENO-NOGUER F , FUA P . EPnP: An accurate O(n) solution to the PnP problem [EB/OL ] . ( 2022 )[2022 ] . https://link.springer.com/article/10.1007/s11263-008-0152- https://link.springer.com/article/10.1007/s11263-008-0152- .

OPENCV . Detection of ArUco markers [EB/OL ] . ( 2022 )[2022 ] . https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_ detection.html https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html .

MAHMOUD M S , MOHAMAD A A H . A study of efficient power consumption wireless communication techniques/modules for Internet of Things (IoT) applications [J ] . Advances in Internet of Things , 2016 , 6 ( 2 ): 19 - 29 .

浏览量

178

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据