资源描述
基于视觉的增强现实系统的研究 孙磊：（P2）增强现实技术(Augmented Reality,简称 AR)是 20 世纪 90 年代初兴起的一种计算机图形图像处理技术，它是将计算机产生的附加信息融合到使用者所看到的真实世界景象中，增强了使用者对真实场景的感知能力。由于其虚实结合的优点，在医疗、工业等领域得到具有广阔的应用发展前景。本论文的研究目的是将增强现实技术应用医疗外科手术导航中，使得医生能够医生在计算机屏幕上同时看到病人体表和内部的病灶组织，提高手术的完成质量。目前，我国在这方面的研究还处在起步阶段。 （P8）增强现实(AR)是随着虚拟现实(VR)技术的发展而产生的，因此两者间存在着不可分割的密切关系，增强现实(AR)系统采用了一些与虚拟现实(VR)技术相同的硬件技术，但两者最关键的不同之处8在于：虚拟现实是用软件模拟出的虚拟世界代替真实世界；而增强现实则是在真实世界的背景中加入补充的虚拟信息。可见，增强现实(AR)不仅继承了虚拟现实(VR)的优点，同时又具有虚实结合的特点。它致力于把计算机带入到用户的真实世界中，允许用户在与实际对象交互的过程中，从计算机接收到关于这些对象的各种辅助信息。在医学应用中，增强现实系统可以利用计算机将人体结构解剖数据可视化处理，并将其虚拟物体影像实时准确地显示在一个患者身体的局部位置，这样外科医生通过显示器将同时看到患者身体的局部外形与其内部解剖结构，帮助手术医生准确定位手术部位，提高其临床技能9-10。如果说，虚拟现实系统是试图把世界送入使用者的计算机，那么增强现实系统则是要把计算机带进使用者的真实的工作环境，从而在虚拟环境与真实世界之间的架起了一座桥梁。 （P10）14 增强现实技术的应用 由于增强现实系统既有虚拟的成份，同时也有现实世界的真实环境，在虚拟环境与真实世界之间建立了桥梁，因此，增强现实技术具有广阔的应用发展前景。增强现实系统将成为一种新型的媒介，其应用领域主要有以下方面： 1医学领域：可以将通过 CT 或 MRI 扫描获得的人体透视三维图形叠加在相应的身体部位, 外科医生在给病人动手术的过程中，看到配准到病人身体上的 CT 或者 MRI 图像,对手术过程进行指导。不仅如此，增强现实还可用于虚拟人体解剖图、虚拟人体功能、虚拟手术模拟、远程手术等，并且可以用于康复医疗23-25（P65）5.6.2 配准实验二：人头模型的配准 配准实验一验证了验证了本文的虚实配准算法，在实验一的基础上，将三维人头模型作为研究对象，模拟手术过程，对虚拟人头模型与真实场景图像进行配准，并从不同角度对虚实图像进行配准。 5.6.2.1 实验数据来源 虚实配准过程中需要的实验数据主要包括两个部分：虚拟模型的三维数据和真实场景的二维图像。计算机要重建出虚拟三维模型所需要医疗切片数据，本课题中采用的是 CT 数据；而真实场景图像是由摄像机拍摄到的 768580 的24位的真彩色图像。 5.6.2.2 实验过程与数据分析 1首先在人头模型上贴上 13 个特征点，分别位于人头的特征位置：下颚、鼻尖、脸颊两侧、耳朵、头顶等位置，如图 5-26 中圆圈标记所示，用序号 0 到 12 标记。2通过摄像机 A 和 B 构成的双目定位系统对 13 个标志点定位，进行10 次实验，获得图 5-26 中标记点 0-12 在真实空间中的三维坐标，10 次实验数据的平均值见表 5-16。 3摄像机 C 拍摄人头模型场景图像，对其拍摄的场景失真图像进行畸变校正。 4利用步骤 3 中立体定位出的标志点的 3D 坐标，结合摄像机 C 拍摄的图像坐标，完成对摄像机 C 的标定，获得人头模型的三维空间点到摄像机成像平面的投影关系，使得人头表面的标志点配准到摄像机 C 拍摄的真实场景图像上，同时将虚拟模型上的其他所有点均与真实景像配准，最后将虚实配准的增强图像送到显示器，实现增强现实目的。 以下同样用距离误差方法分析标志点和标志点以外的其他任意点的配准情况，比较由变换矩阵计算得到的空间点的 2D 投影坐标和真实场景图像中对应点的实际 2D 坐标之间的距离误差，从而测试本文的基于视觉的虚实配准系统的配准精度，具体的距离误差计算公式见配准实验一中公式（5-2）。 增强现实应用技术研究_姚远（p18）是一个典型的交叉学科,它的研究范围十分广泛,涉及到诸如信号处理、计算机图形和图像处理、人机界面和心理学、移动计算、计算机网络、分布式计算、信息获取和信息可视化,以及新型显示器和传感器的设计等各个领域。系统不需要显示完整的虚拟场景,但是需要分析大量的定位数据和场景信息,以保证由计算机生成的虚拟物体可以精确的定位在真实场景中。以基于视频标记定位的系统为例,系统中一般都含四个基本的步骤如图获取真实场景信息对真实场景和场景位置信息进行分析生成虚拟景物合并视频或直接显示。这种方式首先利用视频流中的标志物获得场景位置信息。然后图形系统根据相机位置和从真实场景中获得的定位信息计算虚拟物体坐标到相机视平面的仿射变换,通过变换矩阵在视平面上绘制虚拟物体。最后依靠输出设备显示合成场景。输出设备、跟踪与定位技术、交互技术和真实与虚拟之间的合并技术是实现一个系统的基本支撑技术。(p106)当前技术所有研究都基本围绕于如何构造准确与高质量的虚拟一真实混合环境,以实现更加自然的人机交互界面。应用已经在医疗、工程、教育和娱乐领域显示出越来越广泛的应用前景。而提高环境的显示质量与降低的应用的开发难度是推动技术普及应用重要因素。本文前几章首先在系统阐述研究中基本技术和关键问题的基础上,从提高环境的显示质量入手,提出与场景的阴影生成方法和真实场景光源检测方法,利用最基本的定位标记设备来实现虚拟物体与真实场景的一致性渲染。实际上,应用在设备配置、显示、稳定性和开发过程的复杂性是阻碍技术普及和实用化的主要障碍。为解决这些问题,推动基于环境的交互系统在教育、娱乐和工业等领域中的普及应用,本文提出了基于场景管理的软件框架。以简化系统的设计和开发过程。智能隐形眼镜_健康的_守护天使_刘霞另外，帕维兹研发出的这些隐形眼镜也能直接将图片显示在视线范围内，为佩戴者最终体验到增强现实（也被称为混合现实，指通过电脑技术将虚拟的信息应用到真实世界，使真实环境和虚拟物体实时地叠加到同一个画面或空间同时存在）创造出了平视显示效果。包含小型发光二极管（LED）阵列的隐形眼镜也使各种数字信息能通过眼镜直接向佩戴者显示。目前，手机上已经实现了这种增强现实，无数的软件应用程序将数字信息重重叠叠地放置在展示我们周围环境信息的图片上，让物理世界和网络世界有效地交融在一起。然而，要想在隐形眼镜上实现这一点并不容易，不过，帕维兹的传感器基本做到了。现在，其实验室已经在隐形眼镜显示屏方面取得了进展，他们已经研发出红色和蓝色的微型 LED，只差绿色就齐全了；也制出了具有三维光学器件的眼镜，其外形类似于用来观看三维电影的护目镜。帕维兹打算将光学器件和 LED 结合在同一个隐形眼镜中，他表示，LED 将被组建成网格模式，当显示屏关掉时，也不会干扰正常视力。A Projection-based Medical Augmented Reality SystemJiann-Der Lee*, Hao-Che Lee, Chieh-Tsai Wu Shin-Tseng Lee(p1)In this paper, we present a projection-based medical augmented reality (AR) system for craniofacial application. With the aid of image registration and geometrical correction, the deformation of the projected image on a curved surface like cheek or forehead can be successfully recovered. From the experimental result, it is shown that the overlapped target region is up to 98% after the correction process and this demonstrates the effectiveness of the proposed system. (p1)Nowadays, augmented reality technology has been widely used in medical applications because the target region of interest can be directly projected on the patients body. This kind of display manner is helpful for surgeons to verify the position of important targets, and then the accuracy of the operation is significantly increased. In past, some remarkable approaches were proposed to achieve this goal. For example, Fischer et al. 1 used a special tag board to obtain the relationships of world coordinate system and camera coordinate system, and then transform a Magnetic Resonance Imaging (MRI) from image coordinate system to world coordinate system and displayed it on the computer screen. Bichlmeier et al 2 presents a method for the use of Augmented Reality (AR) for the convergence of improved perception of 3D medical imaging data (mimesis) to the patients own anatomy (in-situ) incorporating the physicians intuitive multisensory interaction by head-mounted display (HMD). However, it is uncomfortable for physicians to wear the HMD during the operation. In this approach, we use the projector to project the target tissue selected by the surgeon on the craniofacial Computed Tomography (CT) to achieve AR display. That is, the surgeon does not need to wear any device during the operation. In the past, there are various approaches to geometrically correct the deformation of a projected image on a nonplanar surface 3. But they are not suitable to correct the distortion of curved surface. In our case, since the shape of the skull is nonplanar, it is necessary to perform curved surface correction before projection. Applying Augmented Reality to Enable Automated and Low-Cost Data Capture from Medical DevicesDaniel Chamberlain Adrian Jimenez-Galindo Richard Ribn Fletcher (p1)As an alternative to building custom electronic devices that connect to mobile phones (via Bluetooth or USB), we present a new approach using Augmented Reality (AR) and machine vision to digitally recognize a biomedical device and capture readings automatically. In the context of developing countries, this approach enables easy integration with low-cost devices, without the need for designing any electronics or obtaining new FDA regulatory approval. As an example, we illustrate the use of AR with a peak flow meter, a device used in the diagnosis and treatment of respiratory disease. In our mobile application, the AR graphic overlay is used to provide feedback to patients and doctors by displaying personalized reference values. Comparing the automated readings from this device to manual readings, our mobile application had a mean error of 5.8 L/min and a correlation of 0.99. A small user study was also conducted in an India field clinic with three health staff (two nurses and a doctor). Following one minute of instruction, the automated readings from the participants had a mean error of 5.5 L/min and a correlation of 0.99 compared to manual readings, with a median task duration of 17.5 seconds. This small case study illustrates how AR can be used to capture medical device data on a mobile phone and help automate the data recording tasks performed by health workers in developing countries. This technology can also be used in developed countries, enabling patients to automatically record readings from similar devices at home using their smart phones. (p2)Many biomedical devices produce analog readings that are never recorded in a digital form. As a result, analysis is difficult and hand-written measurements can be lost, prone to error, or even falsified. A typical engineering solution to this problem would be to design electronic versions of these devices that automatically record values. The problem with this approach is that it increases the complexity and cost of medical devices, in addition to requiring new manufacturing and obtaining regulatory approvals. Given this need, we suggest the use of machine vision and Augmented Reality (AR) to automatically capture measurements from existing medical devices. Since the first AR systems were developed in the early 1980s 2, AR has been used in medicine, primarily focused either on developing training programs 5, 10or enabling physicians to see real-time data captured from medical sensing technologies 3, 6, 9. By using AR to digitize measurements, we can make use of pre-existing and pre-approved medical devices without requiring any electronics. In this paper, we present the peak flow meter as a case study, but this approach is generally applicable to other devices, especially in contexts where data is already being collected but not aggregated or analyzed in a systematic way. While this approach requires a smartphone, such phones are being rapidly adopted in both the developing and developed world 1. Peak flow meters are inexpensive, mechanical devices that are used to assess patient lung health. These devices measure a patients peak expiratory flow rate (PEFR). While useful for other pulmonary diseases, one of the primary uses of a peak flow meter is to monitor the treatment of asthma in patients and evaluate changes in their lung health. (p2)2.2 Augmented Reality Application The augmented reality application was developed for Android smartphones using the Vuforia Augmented Reality SDK. The application allows the user to enter their age, height, and gender. Using this information, the application computes the personalized reference values for comparison with the users measured PEFR. In order to capture a reading, the user aims the smartphone at the peak flow meter. The application automatically detects the augmented reality target and searches for the red indicator. The position of the red indicator is compared to the target and the lateral distance between them is used to determine the measurement value. The detected measurement is displayed on the screen. Underneath the peak flow meter indicator, the application displays green, yellow, and red bars that correspond to the measurement zones defined by the American Lung Association. Although our algorithm was customized to make the specific geometry and colors of the printed scale on this device, our software can be easily customized to capture readings from any device which uses a scale and moving indicator. Once the measurement has been detected, the user clicks continue and the reading is saved to the device. Image data is not saved, thus minimizing memory use. Each reading is saved with a timestamp and all data is saved in CSV format. The user is given the option to export all of their readings using any of the sharing options enabled on their device (text message, email, Dropbox, etc.) The user may also view a plot of their historical readings in the application. Screenshots are shown in Figure 2. Taking Our Time: Chronic Illness and Time-Based Objects in Families Andrea BarbarinTiffany C. VeinotPredrag Klasnja(p2)Moen and Brennan 46 identified four health information storage strategies in the home: 1) “just-in-time,” which tends to stay in a persons physical proximity because it is potentially needed at any moment; 2) “just-at-hand,” which is highly accessible, functioning as a cue for action; 3) “just-in-case,” stored for a future need; and 4) “just-because,” formerly important information that one holds onto while deciding whether to discard it. In work with breast cancer patients, Klasnja et al. 38 extended this concept of “just-in-time” storage to “unanchored settings” where patients do not have their health information with them. This work highlighted the fragmentation of patients information, and challenges in bringing the information to healthcare appointments 38. To address the challenges, they developed a personal health information management system that integrated information from a variety of sources into a smartphone calendar to facilitate easy access during medical appointments 39. (p3)treatment regimens (e.g., pills for HIV/AIDS versus pills/injections, diet and exercise for diabetes), we recruited families in which at least one member was living with either HIV/AIDS or type 2 diabetes. These two different diseases were also selected due to their differing social dynamics; previous research has shown that high levels of perceived stigma affect the information behavior of people with HIV/AIDS and their family members 58, whereas such a relationship was not established in a diabetes context. With deliberate sampling for such variance, we sought to understand time-based object use generally, rather than to identify differences between illnesses. Patient participants were recruited through three methods: 1) when attending appointments at a Veterans Affairs Medical Center in the Midwest; 2) via flyers distributed to clients of three disease-specific nonprofit organizations in the Midwest, or 3) via a university online research participant recruitment system. In addition to the interviews all families participated in, a subset of 24 families (59 individuals) consented to an optional home tour photography session 32, 46, during the fourth wave of interviews. Each home tour began with a participants identifying illness management activities (e.g., making medical appointments, reordering prescriptions, etc.) conducted at home. They led the researcher through the home and described the processes and artifacts/devices used to carry out each activity while a second researcher took photos based on a content/sequence protocol that structured the photography of referenced artifacts/devices, and the zone and room in which they were stored and used 15. Field notes were drafted to capture observations. (p7)Calendars excelled at providing an overview of upcoming time commitments, and calendar reminders effectively captured participants attention. From a safeguarding point of view, many participants noted that the accuracy of their calendars was improved by having multiple versions, and by the timely telephone or mail appointment reminders sent to them by their healthcare providers. The ability of calendar systems to effectively prompt appointment attendance was supported by the fact that only one participant, a non-calendar user, reported missing a medical appointment during the two years of the study. Calendar systems also provided incomplete support for prompting the multiple, interrelated activities involved in preparing to attend medical appointments. Weeks inadvance of an appointment, for example, people with HIV/AIDS were required to have a laboratory-based blood draw for their regular CD4 and viral load tests. Weeks or days in advance, many participants needed to arrange travel to an appointment, particularly if they used a transportation service or relied upon another person for a ride. Several participants regularly brought lists of questions for their providers, with varied time horizons accompanying their advance preparation. For instance, one couple always kept a notepad accessible at home, collaboratively building the list of questions to bring to provider appointments every few months. Provider-driven medication reconciliations also took place at many appointments, especially for participants with multiple healthcare providers. To prepare for this, several patients attempted to compile medication lists in advance. However, prompts could be helpful since this step was not always remembered; this diabetic female participant noted problems “when Im at the doctors office and they want to know what medications Im on, and I havent thought to print out a list.” Further support could be embedded in prompts for this preparation, as evidenced by the excess work performed by this participant with three physicians: “They always want a copy of it, so thats the pain, they copy it because its three pages longI just write it all out ahead of time and take it in and give it to them.”A further limitation of calendar systems concerns the extent to which they accommodate the shifting involvement of (geographically dispersed) family members in patient care. Over a two-year period, many participants noted changes in family relationships. From drifting apart to fractious conflicts, from a new job to the birth of a child, and from moving across the country to worsening health, the involvement of some network members reduced over time. Family members were also newly engaged in care as patients expanded the circle of people who knew about their health status. As this partner of a diabetic patient said, she could ask her husbands friends for transportation to medical appointments, “Now that they know how bad his sugar is.” Short-term health challenges such as broken bones also engaged previously-uninvolved family members in tasks like providing transportation. Participants also formed new relationships with partners and friends, some of whom quickly became involved in their care. Thus, static, place-bound calendars were ill-suited to engaging everyone involved in appointment preparation, and to maintaining accurate representations of the changing family care network. Of those participants using digital calendars, only one couple shared calendars digitally. Furthermore, no participants reported using currently-available online care coordination applications.
展开阅读全文