+- Free Academic Seminars And Projects Reports (https://easyreport.in)
+-- Forum: Seminars Topics And Discussions (https://easyreport.in/forumdisplay.php?fid=30)
+--- Forum: Engineering Seminars Topics (https://easyreport.in/forumdisplay.php?fid=7)
+---- Forum: Electronics Seminar Topics (https://easyreport.in/forumdisplay.php?fid=11)
+---- Thread: AUGMENTED REALITY (/showthread.php?tid=48779)
AUGMENTED REALITY - doomz - 10-04-2017
[attachment=767]
[attachment=768]
AUGMENTED REALITY
ABSTRACT
Video games have been entertaining us for nearly 30 years, ever since Pong was introduced to arcades in the early II 970 s.Computer graphics have become much more sophisticated since then, and soon, game graphics will seem all too real. In the next decade, researchers plan to pull graphics out of your television screen or computer display and integrate them into real- world environments. This new technology called augmented reality, will further blur the line between what is real and what is computer-generated by enhancing what we see, hear, feel and smell.
Augmented reality will truly change the way we view the world. Picture yourself walking or driving down the street. With augmented-reality displays, which will eventually look much like a normal pair of glasses, informative graphics will appear in your field of view, and audio will coincide with what ever you see. These enhancements will be refreshed continually to reflect the moments of your head.
Augmented reality is still in the early stage of research and development at various universities and high-tech companies. Eventually, possibly by the end of this decade we will see the first mass-marketed augmented-reality system, which can be described as the Walkman of the 21st Century.
1. INTRODUCTION
Augmented reality (AR) refers to computer displays that add virtual information to a user s sensory perception. Most AR research focuses on see-through devices, usually worn on the head that overlay graphics and text on the user s view of his or her surroundings. In general it superimposes graphics over a real world environment in real time.
Getting the right information at the right time and the right place is key in all these applications. Personal digital assistants such as the Palm and the Pocket PC can provide timely information using wireless networking and Global Positioning System (GPS) receivers that constantly track the handheld devices. But what make Augmented Reality different is how the information is presented: not on a separate display but integrated with the user s perceptions. This kind of interface minimizes the extra mental effort that a user has to expend when switching his or her attention back and forth between real-world tasks and a computer screen. In augmented reality, the user s view of the world and the computer interface literally become one.
Between the extremes of real life and Virtual Reality lies the spectrum of Mixed Reality, in which views of the real world are combined in some proportion with views of a virtual environment. Combining direct view, stereoscopic videos, and stereoscopic graphics, Augmented Reality describes that class of displays that consists primarily of a real world environment, with graphic enhancement or augmentations.
In Augmented Virtuality, real objects are added to a virtual environment. In Augmented Reality, virtual objects are added to real world. An AR system supplements the real world with virtual (computer generated) objects that appear to co-exist in the same space as the real world. Virtual Reality is a synthetic environment.
1.1 Comparison between AR and virtual environments
The overall requirements of AR can be summarized by comparing them against the requirements for Virtual Environments, for the three basic subsystems that they require.
1. Scene generator : Rendering is not currently one of the major problems in AR. VE systems have much higher requirements for realistic images because they completely replace the real world with the virtual environment . In AR, the virtual images only supplement the real world. Therefore, fewer virtual objects need to be drawn, and they do not necessarily have to be realistically rendered in order to serve the purposes of the application.
2. Display devices: The display devices used in AR may have less stringent requirements than VE systems demand, again because AR does not replace the real world. For example, monochrome displays may be adequate for some AR applications, while virtually all VE systems today use full color. Optical see-through HMD s with a small field-of-view may be satisfactory because the user can still see the real world with his peripheral vision; the see-through HMD does not shut off the user s normal field-of-view. Furthermore, the resolution of the monitor in an optical see-through HMD might be lower than what a user would tolerate in a VE application, since the optical see-through HMD does not reduce the resolution of the real environment.
3. Tracking and sending: While in the previous two cases AR had lower requirements than VE that is not the case for tracking and sensing. In this area, the requirements for AR are much stricter than those for VE systems. A major reason for this is the registration problem.
BASIC SUBSYTEMS VR AR
SCENE GENERATOR MORE ADVANCED LESS ADVANCED
DISPLAY DEVICE HIGH QUALITY LOW QUALITY
TRACKING AND SENSING LESS ADVANCED MORE ADVANCED
Table 1: Comparison of requirements of Augmented Reality and Virtual Reality
2. EVOLUTION
Although augmented reality may seem like the stuff of science fiction, researchers have been building prototype system for more than three decades. The first was developed in the 1960s by computer graphics pioneer Ivan Surtherland and his students at Harvard University.
In the 1970s and 1980s a small number of researchers studied augmented reality at institution such as the U.S. Air Force s Armstrong Laboratory, the NASA Ames Research Center and the university of North Carolina at Chapel Hill.
It wasn t until the early 1990s that the term Augmented Reality was coined by scientists at Boeing who were developing an experimental AR system to help workers assemble wiring harnesses.
In 1996 developers at Columbia University develop The Touring Machine
In 2001 MIT came up with a very compact AR system known as MIThrill.
Presently research is being done in developing BARS (Battlefield Augmented Reality Systems) by engineers at Naval Research Laboratory, Washington D.C.
3. WORKING
AR system tracks the position and orientation of the user s head so that the overlaid material can be aligned with the user s view of the world. Through this process, known as registration, graphics software can place a three dimensional image of a tea cup, for example on top of a real saucer and keep the virtual cup fixed in that position as the user moves about the room. AR systems employ some of the same hardware technologies used in virtual reality research, but there s a crucial differences: whereas virtual reality brashly aims to replace the real world, augmented reality respectfully supplement it.
Augmented Reality is still in an early stage of research and development at various universities and high-tech companies. Eventually, possible by the end of this decade, we will see first mass-marketed augmented reality system, which one researcher calls The Walkman of the 21st century. What augmented reality attempts to do is not only super impose graphics over a real environment in real-time, but also change those graphics to accommodate a user s head- and eye- movements, so that the graphics always fit and perspective.
Here are the three components needed to make an augmented-reality system work:
- Head-mounted display
- Tracking system
- Mobile computing power
3.1 Head-Mounted Display
Just as monitor allow us to see text and graphics generated by computers, head-mounted displays (HMD s) will enable us to view graphics and text created by augmented-reality systems.
There are two basic types of HMD s
- Optical see-through
- Video see-through
Optical Display Video Display
Fig 1: Optical and Video Display
3.1.1 Optical see-through display
Fig 2: Optical see-through HMD conceptual diagram.
A simple approach to optical see-through display employs a mirror beam splitter- a half silvered mirror that both reflects and transmits light. If properly oriented in front of the user s eye, the beam splitter can reflect the image of a computer display into the user s line of sight yet still allow light
from the surrounding world to pass through. Such beam splitters, which are called combiners, have long been used in head up displays for fighter-jet- pilots (and, more recently, for drivers of luxury cars). Lenses can be placed between the beam splitter and the computer display to focus the image so that it appears at a comfortable viewing distance. If a display and optics are provided for each eye, the view can be in stereo. Sony makes a see-through display that some researchers use, called the Glasstron.
3.1.2 Video see-through displays
Fig 3: Video see-through HMD conceptual diagram
In contrast, a video see through display uses video mixing technology, originally developed for television special effects, to combine the image from a head worn camera with synthesized graphics. The merged image is typically presented on an opaque head worn display. With careful design the camera can be positioned so that its optical path is closed to that of the user s eye; the video image thus approximates what the user
would normally see. As with optical see through displays, a separate system can be provided for each eye to support stereo vision. Video composition can be done in more than one way. A simple way is to use chroma-keying: a technique used in many video special effects. The background of the computer graphics images is set to a specific color, say green, which none of the virtual objects use. Then the combining step replaces all green areas with the corresponding parts from the video of the real world. This has the effect of superimposing the virtual objects over the real world. A more sophisticated composition would use depth information at each pixel for the real world images; it could combine the real and virtual images by a pixel-by-pixel depth comparison. This would allow real objects to cover virtual objects and vice-versa.
A different approach is the virtual retinal display, which forms images directly on the retina. These displays, which Micro Vision is developing commercially, literally draw on the retina with low power lasers modulated beams are scanned by microelectro-mechanical mirror assemblies that sweep the beam horizontally and vertically. Potential advantages include high brightness and contrast, low power consumption, and large depth of field.
Fig 4: Two views of a combined augmented and virtual environment
Fig 5: Two optical see-through HMD s, made by Hughes Electronics
3.1.3 Comparison of optical see through and video see through displays
Each of approaches to see through display design has its pluses and minuses. Optical see through systems allows allow the user to see the real world with resolution and field of view. But the overlaid graphics in current optical see through systems are not opaque and therefore cannot completely obscure the physical objects behind them. As result, the superimposed text may be hard to read against some backgrounds, and three-dimensional graphics may not produce a convincing illusion. Furthermore, although a focuses physical objects depending on their distance, virtual objects are all focused in the plane of the display. This means that a virtual object that is intended to be at the same position as a physical object may have a geometrically correct projection, yet the user may not be able to view both objects in focus at the same time.
In video see-through systems, virtual objects can fully obscure physical ones and can be combined with them using a rich variety of graphical effects. There is also discrepancy between how the eye focuses virtual and physical objects, because both are viewed on same plane. The limitations of current video technology, however, mean that the quality of the visual experience of the real world is significantly decreased, essentially to the level of the synthesized graphics, with everything focusing at the same apparent distance. At present, a video camera and display is no match for the human eye.
An optical approach has the following advantages over a video approach
1. Simplicity: Optical blending is simpler and cheaper than video blending. Optical approaches have only one stream of video to worry about: the graphic images. The real world is seen directly through the combiners, and that time delay is generally a few nanoseconds. Video blending, on the other hand, must deal with separate video streams for the real and virtual images. The two streams of real and virtual images must be properly synchronized or temporal distortion results. Also, optical see through HMD s with narrow field of view combiners offer views of the real world that have little distortion. Video cameras almost always have some amount of distortion that must be compensated for, along with any distortion from the optics in front of the display devices. Since video requires cameras and combiners that optical approaches do not need, video will probably be more expensive and complicated to build than optical based systems.
2. Resolution: Video blending limits the resolution of what the user sees, both real and virtual, to the resolution of the display devices. With current displays, this resolution is far less than the resolving power of the fovea. Optical see-through also shows the graphic images at the resolution of the display devices, but the user s view of the real world is not degraded. Thus, video reduces the resolution of the real world, while optical see-through does not.
3. Safety: Video see-through HMD s are essentially modified closed-view HMD s. If the power is cut off, the user is effectively blind. This is a safety concern in some applications. In contrast, when power is removed from an optical see-through HMD, the user still has a direct view of the real world. The HMD then becomes a pair of heavy sunglasses, but the user can still see.
4. No eye offset: With video see-through, the user s view of the real world is provided by the video cameras. In essence, this puts his eyes where the video cameras are not located exactly where the user s eyes are, creating an
offset between the cameras and the real eyes. The distance separating the cameras may also not be exactly the same as the user s interpupillary distance (IPD). This difference between camera locations and eye locations introduces displacements from what the user sees compared to what he expects to see. For example, if the cameras are above the user s eyes, he will see the world from a vantage point slightly taller than he is used to.
Video blending offers the following advantages over optical blending
1. Flexibility in composition strategies: A basic problem with optical see-through is that the virtual objects do not completely obscure the real world objects, because the optical combiners allow light from both virtual and real sources. Building an optical see-through HMD that can selectively shut out the light from the real world is difficult. Any filter that would selectively block out light must be placed in the optical path at a point where the image is in focus, which obviously cannot be the user s eye. Therefore, the optical system must have two places where the image is in focus: at the user s eye and the point of the hypothetical filter. This makes the optical design much more difficult and complex. No existing optical see-through HMD blocks incoming light in this fashion. Thus, the virtual objects appear Ghost-like and semi-transparent. This damages the illusion of reality because occlusion is one of the strongest depth cues. In contrast, video see-through is far more flexible about how it merges the real and virtual images. Since both the real and virtual are available in digital form, video see-through compositors can, on a pixel-by-pixel basis, take the real, or the virtual, or some blend between the two to simulate transparency.
2. Wide field-of-view: Distortions in optical systems are a function of the radial distance away from the optical axis. The further one looks away from the center of the view, the larger the distortions get. A digitized image taken through a distorted optical system can be undistorted by applying image processing techniques to unwrap the image, provided that the optical distortion is well characterized. This requires significant amount of computation, but this constraint will be less important in the future as computers become faster. It is harder to build wide field-of-view displays with optical see-through techniques. Any distortions of the user s view of the real world must be corrected optically, rather than digitally, because the system has no digitized image of the real world to manipulate. Complex optics is expensive and add weight to the HMD. Wide field-of-view systems are an exception to the general trend of optical approaches being simpler and cheaper than video approaches.
3. Real and virtual view delays can be matched: Video offers an approach for reducing or avoiding problems caused by temporal mismatches between the real and virtual images. Optical see-through HMD s offer an almost instantaneous view of the real world but a delayed view of the virtual. This temporal mismatch can cause problems. With video approaches, it is possible to delay the video of the real world to match the delay from the virtual image stream.
4. Additional registration strategies: In optical see-through, the only information the system has about the user s head location comes from the head tracker. Video blending provides another source of information: the digitized image of the real scene. This digitized image means that video approaches can employ additional registration strategies unavailable to optical approaches.
5. Easier to match the brightness of the real and virtual objects: Both optical and video technologies have their roles, and the choice of technology depends upon the application requirements. Many of the mismatch assembly and repair prototypes use optical approaches, possibly because of the cost and safety issues. If successful, the equipment would have to be replicated in large numbers to equip workers on a factory floor. In contrast, most of the prototypes for medical applications use video approaches, probably for the flexibility in blending real and virtual and for the additional registration strategies offered.
3.2 Tracking and Orientation
The biggest challenge facing developers of augmented reality the need to know where the user is located in reference to his or her surroundings. There s also the additional problem of tracking the movement of users eyes and heads. A tracking system has to recognize these movements and project the graphics related to the real-world environment the user is seeing at any given movement. Currently both video see-through and optical see-through displays optically have lag in the overlaid material due to the tracking technologies currently available.
3.2.1 Indoor Tracking
Tracking is easier in small spaces than in large spaces. Trackers typically have two parts: one worn by the tracked person or object and other built into the surrounding environment, usually within the same room. In optical trackers, the targets LED s or reflectors, for instance can be attached to the tracked person or to the object, and an array of optical sensors can be embedded in the room s ceiling. Alternatively the tracked users can wear the sensors, and targets can be fixed to the ceiling. By calculating the distance to reach visible target, the sensors can determine the user s position and orientation.
Researchers at the University of North Carolina-Chapel Hill have developed a very precise system that works within 500 sq feet. The HiBall Tracking System is an optoelectronic tracking system made of two parts:
Six user-mounted, optical sensors.
Infrared-light-emitting diodes (LED s) embedded in special ceiling panels.
The system uses the known location of LED s the known geometry of the user-mounted optical sensors and a special algorithm to compute and report the user s position and orientation. The system resolves linear motion of less than 0.2 millimeters, and angular motions less than 0.03 degrees. It has an update rate of more than 1500Hz, and latency is kept at about one millisecond. In everyday life, people rely on several senses-including what they see, cues from their inner ears and gravity s pull on their bodies- to maintain their bearings. In a similar fashion, Hybrid Trackers draw on several sources of sensory information. For example, the wearer of an AR display can be equipped with inertial sensors (gyroscope and accelerometers) to record changes in head orientation. Combining this information with data from optical, video or ultrasonic devices greatly improve the accuracy of tracking.
3.2.2Out door Tracking
Head orientation is determined with a commercially available hybrid tracker that combines gyroscopes and accelerometers with magnetometers that measure the earth s magnetic field. For position tracking we take advantage OF a high-precision version of the increasingly popular Global Positioning system receiver.
A GPS receiver can determine its position by monitoring radio signals from navigation satellites. GPS receivers have an accuracy of about 10 to 30 meters. An augmented reality system would be worthless if the graphics projected were of something 10 to 30 meters away from what you were actually looking at.
User can get better result with a technique known as differential GPS. In this method, the mobile GPS receiver also monitors signals from another GPS receiver and a radio transmitter at a fixed location on the earth. This transmitter broadcasts the correction based on the difference between the stationary GPS antenna s known and computed positions. By using these signals to correct the satellite signals, the differential GPS can reduce the margin of error to less than one meter.
The system is able to achieve the centimeter-level accuracy by employing the real-time kinematics GPS, a more sophisticated form of differential GPS that also compares the phases of the signals at the fixed and mobile receivers. Trimble Navigation reports that they have increased the precision of their global positioning system (GPS) by replacing local reference stations with what they term a Virtual Reference Station (VRS). This new VRS will enable users to obtain a centimeter-level positioning without local reference stations; it can achieve long-range, real-time kinematics (RTK) precision over greater distances via wireless communications wherever they are located. Real-time kinematics technique is a way to use GPS measurements to generate positioning within one to two centimeters (0.39 to 0.79 inches). RTK is often used as the key component in navigational system or automatic machine guidance.
Unfortunately, GPS is not the ultimate answer to position tracking. The satellite signals are relatively weak and easily blocked by buildings or even foliage. This rule out useful tracking indoors or in places likes midtown Manhattan, where rows of tall building block most of the sky. GPS tracking works well in wide open spaces and relatively low buildings.
GPS provide far too few updates per second and is too inaccurate to support the precise overlaying of graphics on nearby objects. Augmented Reality system places extra ordinary high demands on the accuracy, resolution, repeatability and speed of tracking technologies. Hardware and software delays introduce a lag between the user s movement and the update of the display. As a result, virtual objects will not remain in their proper position as the user moves about or turns his or her head. One technique for combating such errors is to equip AR system with software that makes short-term predictions about the user s future motion by extrapolating from previous movements. And in the long run, hybrid trackers that include computer vision technologies may be able trigger appropriate graphics overlays when the devices recognize certain objects in the user s view.
4. MOBILE COMPUTING POWER
For a wearable augmented realty system, there is still not enough computing power to create stereo 3-D graphics. So researchers are using whatever they can get out of laptops and personal computers, for now. Laptops are just now starting to be equipped with graphics processing unit (GPU s). Toshiba just now added a NVIDIA to their notebooks that is able to process more than 17-million triangles per second and 286-million pixels per second, which can enable CPU-intensive programs, such as 3D games. But still notebooks lag far behind- NVIDIA has developed a custom 300-MHz 3-D graphics processor for Microsoft s Xbox game console that can produce 150 million polygon per second and polygons are more complicated than triangles. So you can see how far mobiles graphics chips have to go before they can create smooth graphics like the ones you see on your home video-game system.
5. APPLICATIONS
Only recently have the capabilities of real-time video image processing, computer graphics systems and new display technologies converged to make possible the display of a virtual graphical image correctly registered with a view of the 3D environment surrounding the user. Researchers working with the AR system have proposed them as solutions in many domains. The areas have been discussed range from entertainment to military training. Many of the domains, such as medical are also proposed for traditional virtual reality systems. This section will highlight some of the proposed application for augmented reality.
5.1 Medical
Because imaging technology is so pervasive throughout the medical field, it is not surprising that this domain is viewed as one of the more important for augmented reality systems. Most of the medical application deal with image guided surgery. Pre-operative imaging studies such as CT or MRI scans, of the patient provide the surgeon with the necessary view of the internal anatomy. From these images the surgery is planned. Visualization of the path through the anatomy to the affected area where, for example, a tumor must be removed is done by first creating the 3D model from the multiple views and slices in the preoperative study. This is most often done mentally though some systems will create 3D volume visualization from the image study. AR can be applied so that the surgical team can see the CT or MRI data correctly registered on the patient in the operating theater while the procedure is progressing. Being able to accurately register the images at this point will enhance the performance of the surgical team.
Another application for AR in the medical domain is in ultra sound imaging. Using an optical see-through display the ultrasound technician
can view a volumetric rendered image of the fetus overlaid on the abdomen of the pregnant woman. The image appears as if it were inside of the abdomen and is correctly rendered as the user moves.
Fig 6: Virtual fetus inside womb of pregnant patient.
Fig 7: Mockup of breast tumor biopsy. 3-D graphics guide needle insertion.
5.2 Entertainment
A simple form of the augmented reality has been in use in the entertainment and news business for quite some time. Whenever you are watching the evening weather report the weather reporter is shown standing in the front of changing weather maps. In the studio the reporter is standing in front of a blue or a green screen. This real image is augmented with the computer generated maps using a technique called chroma-keying. It is also possible to create a virtual studio environment so that the actors can appear to be positioned in a studio with computer generated decorating.
Movie special effects make use of digital computing to create illusions. Strictly speaking with current technology this may not be considered augmented reality because it is not generated in the real-time. Most special effects are created off-line, frame by frame with a substantial amount of user interaction and computer graphics system rendering. But some work is progressing in
computer analysis of the live action images to determine the camera parameters and use this to drive the generation of the virtual graphics objects to be merged.
Princeton Electronics Billboard has developed an augmented reality system that allows broadcasters to insert advertisement into specific areas of the broadcast image. For example, while broadcasting a baseball game this system would be able to place an advertisement in the image so that it appears on the outfield wall of the stadium. By using pre-specified reference points in the stadium, the system automatically determines the camera angles being used and referring to the pre-defined stadium map inserts the advertisement into the current place. AR QUAKE, 76 designed using the same platform, blends users in the real world with those in a purely virtual environment. A mobile AR user plays as a combatant in the computer game Quake, where the game runs with a virtual model of the real environment.
Fig 8: AR in sports broadcasting. The annotations on the race cars and the yellow first down line are inserted into the broad cast in real time.
5.3 Military Training
The military has been using display in cockpits that present information to the pilot on the windshield of the cockpit or the visor of their flight helmet. This is a form of Augmented Reality display. SIMNET, a distributed war games simulating system, is also embracing augmented reality technology. By equipping military personnel with helmet mounted visor displays or a special purpose rangefinder the activities of other units participating in the exercise
can be imaged. While looking at the horizon, for example, the display equipped soldier could see a helicopter rising above the tree line. This helicopter could be being flown in simulation by another participant. In war time, the display of the real battlefield scene could be augmented with annotation information or highlighting to emphasize hidden enemy units.
5.4 Engineering Design
Imagine that a group of designers are working on the model of a complex device for their clients. The designers and clients want to do a joint design reviews even though they are physically separated. If each of them had a conference room that was equipped with an augmented re4ality display this could be accomplished. The physical prototype that the designers have mocked up is imaged and displayed in the client s conference room in 3D. The clients can walk around display looking at different aspects of it. To hold the discussion the client can point at the prototype to highlight sections and this will be reflected on the real model in the augmented display that the designers are using. Or perhaps in an earlier stage of the design, before a prototype is built, the view in each conference room is augmented with a computer generated image of the current design built from the CAD file describing it. This would allow real time interactions with elements of the design so that either side can make adjustments and change that are reflected in the view seen by both groups.
5.5 Robotics and Telerobotics
In the domain of robotics and Telerobotics an augmented display can assist the user of the system. A Telerobotics operator uses a visual image of the remote workspace to guide the robot. Annotation of the view would still be useful just as it is when the scene is in front of the operator. There is an added potential benefit. Since often the view of the remote scene is monoscopic, augmentation with wire frame drawings of structures in the view can facilitate visualization of the remote 3D geometry. If the operator is attempting a motion it could be practiced on a virtual robot that is visualized as an augmentation to the real scene. The operator can decide to proceed
with the motion after seeing the results. The robot motion could then be executed directly which in a telerobotics application would eliminate any oscillations caused by long delays to the remote site.
Fig 9: Virtual lines show a planned motion of a robot arm
5.6 Manufacturing, maintenance and repair
When the maintenance technician approaches a new or unfamiliar piece of equipment instead of opening several repair manuals they could put on an augmented reality display. In this display the image of the equipment would be augmented with annotations and information pertinent to the repair. For example, the location of fasteners and attachment hardware that must be removed would be highlighted. Then the inside view of the machine would highlight the boards that need to be replaced. The military has developed a wireless vest worn by personnel that is attached to an optical see-through display. The wireless connection allows the soldier to access repair manuals and images of the equipment. Future versions might register those images on the live scene and provide animation to show the procedures that must be performed.Boeing researchers are developing an augmented reality display to replace the large work frames used for making wiring harnesses for their aircraft. Using this experimental system, the technicians are guided by the augmented display that shows the routing of the cables on a generic frame used for all harnesses. The augmented display allows a single fixture to be used for making the multiple harnesses.
5.7 Consumer design
Virtual reality systems are already used for consumer design. Using perhaps more of a graphics system than virtual reality, when you go to the typical home store wanting to add a new deck to your house, they will show you a graphical picture of what the deck will look like. It is conceivable that a future system would allow you to bring a video tape of your house shot from various viewpoints in your backyard and in real time it would augment that view to show the new deck in its finished form attached to your house. Or bring in a tape of your current kitchen and the augmented reality processor would replace your current kitchen cabinetry with virtual images of the new kitchen that you are designing.
Applications in the fashion and beauty industry that would benefit from an augmented reality system can also be imaged. If the dress store does not have a particular style dress in your size an appropriate sized dress could be used to augment the image of you. As you looked in the three sided mirror you would see the image of the new dress on your body. Changes in hem length, shoulder styles or other particulars of the design could be viewed on you before you place the order. When you head into some high-tech beauty shops today you can see what a new hair style would look like on a digitized image of yourself. But with an advanced augmented reality system you would be able to see the view as you moved. If the dynamics of hair are included in the description of the virtual object you would also see the motion of hair as your head moved.
5.8 Instant information
Tourists and students could use these systems to learn more about a certain historical event. Imagine walking onto a Civil War battlefield and seeing a re-creation of historical events on a head-mounted, augmented reality display. It would immerse you in the event, and the view would be panoramic. The recently started Archeoguide project is developing a wearable AR system for providing tourists with information about a historical site in Olympia, Greece.
6. FUTURE DIRECTIONS
This section identifiers areas and approaches that require further researches to produce improved AR systems.
Hybrid approach
Further tracking systems may be hybrids, because combining approaches can cover weaknesses. The same may be true for other problems in AR. For example, current registration strategies generally focus on a single strategy. Further systems may be more robust if several techniques are combined. An example is combining vision-based techniques with prediction. If the fiducially are not available, the system switches to open-loop prediction to reduce the registration errors, rather than breaking down completely. The predicted viewpoints in turn produce a more accurate initial location estimate for the vision-based techniques.
Real time systems and time-critical computing
Many VE systems are not truly run in real time. Instead, it is common to build the system, often on UNIX, and then see how fast it runs. This may be sufficient for some VE applications. Since everything is virtual, all the objects are automatically synchronized with each other. AR is different story. Now the virtual and real must be synchronized, and the real world runs in real time. Therefore, effective AR systems must be built with real time performance in mind. Accurate timestamps must be available. Operating systems must not arbitrarily swap out the AR software process at any time, for arbitrary durations. Systems must be built ton guarantee completion within specified time budgets, rather than just running as quickly as possible. These are characteristics of flight simulators and a few VE systems. Constructing and debugging real-time systems is often painful and difficult, but the requirements for AR demand real-time performance.
Perceptual and psychophysical studies
Augmented reality is an area ripe for psychophysical studies. How much lag can a user detect? How much registration error is detectable when the head is moving? Besides questions on perception, psychological experiments that explore performance issues are also needed. How much does head-motion prediction improve user performance on a specific task? How much registration error is tolerable for a specific application before performance on that task degrades substantially? Is the allowable error larger while the user moves her head versus when she stands still? Furthermore, no much is known about potential optical illusion caused by errors or conflicts in the simultaneous display of real and virtual objects.
Portability
It is essential that potential AR applications give the user the ability to walk around large environments, even outdoors. This requires making the requirement self-continued and portable. Existing tracking technology is not capable of tracking a user outdoors at the required accuracy.
Multimodal displays
Almost all work in AR has focused on the visual sense: virtual graphic objects and overlays. But augmentation might apply to all other senses as well. In particular, adding and removing 3-D sound is a capability that could be useful in some AR applications.
7. CONCLUSION
Augmented reality is far behind Virtual Environments in maturity. Several commercial vendors sell complete, turnkey Virtual Environment systems. However, no commercial vendor currently sells an HMD-based Augmented Reality system. A few monitor-based virtual set systems are available, but today AR systems are primarily found in academic and industrial research laboratories.
The first deployed HMD-based AR systems will probably be in the application of aircraft manufacturing. Both Boeing and McDonnell Douglas are exploring this technology. The former uses optical approaches, while the letter is pursuing video approaches. Boeing has performed trial runs with workers using a prototype system but has not yet made any deployment decisions. Annotation and visualization applications in restricted, limited range environments are deployable today, although much more work needs to be done to make them cost effective and flexible.
Applications in medical visualization will take longer. Prototype visualization aids have been used on an experimental basis, but the stringent registration requirements and ramifications of mistakes will postpone common usage for many years. AR will probably be used for medical training before it is commonly used in surgery.
The next generation of combat aircraft will have Helmet Mounted Sights with graphics registered to targets in the environment. These displays, combined with short-range steer able missiles that can shoot at targets off-bore sight, give a tremendous combat advantage to pilots in dogfights. Instead of having to be directly behind his target in order to shoot at it, a pilot can now shoot at anything within a 60-90 degree cone of his aircraft s forward centerline. Russia and Israel currently have systems with this capability, and the U.S is expected to field the AIM-9X missile with its associated Helmet-mounted sight in 2002.
Augmented Reality is a relatively new field, where most of the research efforts have occurred in the past four years. Because of the numerous challenges and unexplored avenues in this area, AR will remain a vibrant area of research for at least the next several years.
After the basic problems with AR are solved, the ultimate goal will be to generate virtual objects that are so realistic that they are virtually indistinguishable from the real environment. Photorealism has been demonstrated in feature films, but accomplishing this in an interactive application will be much harder. Lighting conditions, surface reflections, and other properties must be measured automatically, in real time. More sophisticated lighting, texturing, and shading capabilities must run at interactive rates in future scene generators. Registration must be nearly perfect, without manual intervention or adjustments.
While these are difficult problems, they are probably not insurmountable. It took about 25 years to progress from drawing stick figures on a screen to the photorealistic dinosaurs in Jurassic Park. Within another 25 years, we should be able to wear a pair of AR glasses outdoors to see and interact with photorealistic dinosaurs eating a tree in our backyard.
8. REFERENCES
A survey of Augmented Reality by Ronald T. Azuma
Recent Advances in Augmented Reality by Ronald T.Azuma, Yohan Baillot, Reinhold Beringer, Simon Julier and Blair MacIntyre
Augmented Reality: A new way of seeing. Steven K Feiner
Augmented Reality and computer Augmented Environment, available at
http://csl.sony.co.jp/project/ar/ref.html
1. INTRODUCTION
Augmented reality (AR) refers to computer displays that add virtual information to a user s sensory perception. Most AR research focuses on see-through devices, usually worn on the head that overlay graphics and text on the user s view of his or her surroundings. In general it superimposes graphics over a real world environment in real time.
Getting the right information at the right time and the right place is key in all these applications. Personal digital assistants such as the Palm and the Pocket PC can provide timely information using wireless networking and Global Positioning System (GPS) receivers that constantly track the handheld devices. But what make Augmented Reality different is how the information is presented: not on a separate display but integrated with the user s perceptions. This kind of interface minimizes the extra mental effort that a user has to expend when switching his or her attention back and forth between real-world tasks and a computer screen. In augmented reality, the user s view of the world and the computer interface literally become one.
Between the extremes of real life and Virtual Reality lies the spectrum of Mixed Reality, in which views of the real world are combined in some proportion with views of a virtual environment. Combining direct view, stereoscopic videos, and stereoscopic graphics, Augmented Reality describes that class of displays that consists primarily of a real world environment, with graphic enhancement or augmentations.
In Augmented Virtuality, real objects are added to a virtual environment. In Augmented Reality, virtual objects are added to real world. An AR system supplements the real world with virtual (computer generated) objects that appear to co-exist in the same space as the real world. Virtual Reality is a synthetic environment.
1.1 Comparison between AR and virtual environments
The overall requirements of AR can be summarized by comparing them against the requirements for Virtual Environments, for the three basic subsystems that they require.
1. Scene generator : Rendering is not currently one of the major problems in AR. VE systems have much higher requirements for realistic images because they completely replace the real world with the virtual environment . In AR, the virtual images only supplement the real world. Therefore, fewer virtual objects need to be drawn, and they do not necessarily have to be realistically rendered in order to serve the purposes of the application.
2. Display devices: The display devices used in AR may have less stringent requirements than VE systems demand, again because AR does not replace the real world. For example, monochrome displays may be adequate for some AR applications, while virtually all VE systems today use full color. Optical see-through HMD s with a small field-of-view may be satisfactory because the user can still see the real world with his peripheral vision; the see-through HMD does not shut off the user s normal field-of-view. Furthermore, the resolution of the monitor in an optical see-through HMD might be lower than what a user would tolerate in a VE application, since the optical see-through HMD does not reduce the resolution of the real environment.
3. Tracking and sending: While in the previous two cases AR had lower requirements than VE that is not the case for tracking and sensing. In this area, the requirements for AR are much stricter than those for VE systems. A major reason for this is the registration problem.
BASIC SUBSYTEMS VR AR
SCENE GENERATOR MORE ADVANCED LESS ADVANCED
DISPLAY DEVICE HIGH QUALITY LOW QUALITY
TRACKING AND SENSING LESS ADVANCED MORE ADVANCED
Table 1: Comparison of requirements of Augmented Reality and Virtual Reality
2. EVOLUTION
Although augmented reality may seem like the stuff of science fiction, researchers have been building prototype system for more than three decades. The first was developed in the 1960s by computer graphics pioneer Ivan Surtherland and his students at Harvard University.
In the 1970s and 1980s a small number of researchers studied augmented reality at institution such as the U.S. Air Force s Armstrong Laboratory, the NASA Ames Research Center and the university of North Carolina at Chapel Hill.
It wasn t until the early 1990s that the term Augmented Reality was coined by scientists at Boeing who were developing an experimental AR system to help workers assemble wiring harnesses.
In 1996 developers at Columbia University develop The Touring Machine
In 2001 MIT came up with a very compact AR system known as MIThrill.
Presently research is being done in developing BARS (Battlefield Augmented Reality Systems) by engineers at Naval Research Laboratory, Washington D.C.
3. WORKING
AR system tracks the position and orientation of the user s head so that the overlaid material can be aligned with the user s view of the world. Through this process, known as registration, graphics software can place a three dimensional image of a tea cup, for example on top of a real saucer and keep the virtual cup fixed in that position as the user moves about the room. AR systems employ some of the same hardware technologies used in virtual reality research, but there s a crucial differences: whereas virtual reality brashly aims to replace the real world, augmented reality respectfully supplement it.
Augmented Reality is still in an early stage of research and development at various universities and high-tech companies. Eventually, possible by the end of this decade, we will see first mass-marketed augmented reality system, which one researcher calls The Walkman of the 21st century. What augmented reality attempts to do is not only super impose graphics over a real environment in real-time, but also change those graphics to accommodate a user s head- and eye- movements, so that the graphics always fit and perspective.
Here are the three components needed to make an augmented-reality system work:
- Head-mounted display
- Tracking system
- Mobile computing power
3.1 Head-Mounted Display
Just as monitor allow us to see text and graphics generated by computers, head-mounted displays (HMD s) will enable us to view graphics and text created by augmented-reality systems.
There are two basic types of HMD s
- Optical see-through
- Video see-through
Optical Display Video Display
Fig 1: Optical and Video Display
3.1.1 Optical see-through display
Fig 2: Optical see-through HMD conceptual diagram.
A simple approach to optical see-through display employs a mirror beam splitter- a half silvered mirror that both reflects and transmits light. If properly oriented in front of the user s eye, the beam splitter can reflect the image of a computer display into the user s line of sight yet still allow light
from the surrounding world to pass through. Such beam splitters, which are called combiners, have long been used in head up displays for fighter-jet- pilots (and, more recently, for drivers of luxury cars). Lenses can be placed between the beam splitter and the computer display to focus the image so that it appears at a comfortable viewing distance. If a display and optics are provided for each eye, the view can be in stereo. Sony makes a see-through display that some researchers use, called the Glasstron.
3.1.2 Video see-through displays
Fig 3: Video see-through HMD conceptual diagram
In contrast, a video see through display uses video mixing technology, originally developed for television special effects, to combine the image from a head worn camera with synthesized graphics. The merged image is typically presented on an opaque head worn display. With careful design the camera can be positioned so that its optical path is closed to that of the user s eye; the video image thus approximates what the user
would normally see. As with optical see through displays, a separate system can be provided for each eye to support stereo vision. Video composition can be done in more than one way. A simple way is to use chroma-keying: a technique used in many video special effects. The background of the computer graphics images is set to a specific color, say green, which none of the virtual objects use. Then the combining step replaces all green areas with the corresponding parts from the video of the real world. This has the effect of superimposing the virtual objects over the real world. A more sophisticated composition would use depth information at each pixel for the real world images; it could combine the real and virtual images by a pixel-by-pixel depth comparison. This would allow real objects to cover virtual objects and vice-versa.
A different approach is the virtual retinal display, which forms images directly on the retina. These displays, which Micro Vision is developing commercially, literally draw on the retina with low power lasers modulated beams are scanned by microelectro-mechanical mirror assemblies that sweep the beam horizontally and vertically. Potential advantages include high brightness and contrast, low power consumption, and large depth of field.
Fig 4: Two views of a combined augmented and virtual environment
Fig 5: Two optical see-through HMD s, made by Hughes Electronics
3.1.3 Comparison of optical see through and video see through displays
Each of approaches to see through display design has its pluses and minuses. Optical see through systems allows allow the user to see the real world with resolution and field of view. But the overlaid graphics in current optical see through systems are not opaque and therefore cannot completely obscure the physical objects behind them. As result, the superimposed text may be hard to read against some backgrounds, and three-dimensional graphics may not produce a convincing illusion. Furthermore, although a focuses physical objects depending on their distance, virtual objects are all focused in the plane of the display. This means that a virtual object that is intended to be at the same position as a physical object may have a geometrically correct projection, yet the user may not be able to view both objects in focus at the same time.
In video see-through systems, virtual objects can fully obscure physical ones and can be combined with them using a rich variety of graphical effects. There is also discrepancy between how the eye focuses virtual and physical objects, because both are viewed on same plane. The limitations of current video technology, however, mean that the quality of the visual experience of the real world is significantly decreased, essentially to the level of the synthesized graphics, with everything focusing at the same apparent distance. At present, a video camera and display is no match for the human eye.
An optical approach has the following advantages over a video approach
1. Simplicity: Optical blending is simpler and cheaper than video blending. Optical approaches have only one stream of video to worry about: the graphic images. The real world is seen directly through the combiners, and that time delay is generally a few nanoseconds. Video blending, on the other hand, must deal with separate video streams for the real and virtual images. The two streams of real and virtual images must be properly synchronized or temporal distortion results. Also, optical see through HMD s with narrow field of view combiners offer views of the real world that have little distortion. Video cameras almost always have some amount of distortion that must be compensated for, along with any distortion from the optics in front of the display devices. Since video requires cameras and combiners that optical approaches do not need, video will probably be more expensive and complicated to build than optical based systems.
2. Resolution: Video blending limits the resolution of what the user sees, both real and virtual, to the resolution of the display devices. With current displays, this resolution is far less than the resolving power of the fovea. Optical see-through also shows the graphic images at the resolution of the display devices, but the user s view of the real world is not degraded. Thus, video reduces the resolution of the real world, while optical see-through does not.
3. Safety: Video see-through HMD s are essentially modified closed-view HMD s. If the power is cut off, the user is effectively blind. This is a safety concern in some applications. In contrast, when power is removed from an optical see-through HMD, the user still has a direct view of the real world. The HMD then becomes a pair of heavy sunglasses, but the user can still see.
4. No eye offset: With video see-through, the user s view of the real world is provided by the video cameras. In essence, this puts his eyes where the video cameras are not located exactly where the user s eyes are, creating an
offset between the cameras and the real eyes. The distance separating the cameras may also not be exactly the same as the user s interpupillary distance (IPD). This difference between camera locations and eye locations introduces displacements from what the user sees compared to what he expects to see. For example, if the cameras are above the user s eyes, he will see the world from a vantage point slightly taller than he is used to.
Video blending offers the following advantages over optical blending
1. Flexibility in composition strategies: A basic problem with optical see-through is that the virtual objects do not completely obscure the real world objects, because the optical combiners allow light from both virtual and real sources. Building an optical see-through HMD that can selectively shut out the light from the real world is difficult. Any filter that would selectively block out light must be placed in the optical path at a point where the image is in focus, which obviously cannot be the user s eye. Therefore, the optical system must have two places where the image is in focus: at the user s eye and the point of the hypothetical filter. This makes the optical design much more difficult and complex. No existing optical see-through HMD blocks incoming light in this fashion. Thus, the virtual objects appear Ghost-like and semi-transparent. This damages the illusion of reality because occlusion is one of the strongest depth cues. In contrast, video see-through is far more flexible about how it merges the real and virtual images. Since both the real and virtual are available in digital form, video see-through compositors can, on a pixel-by-pixel basis, take the real, or the virtual, or some blend between the two to simulate transparency.
2. Wide field-of-view: Distortions in optical systems are a function of the radial distance away from the optical axis. The further one looks away from the center of the view, the larger the distortions get. A digitized image taken through a distorted optical system can be undistorted by applying image processing techniques to unwrap the image, provided that the optical distortion is well characterized. This requires significant amount of computation, but this constraint will be less important in the future as computers become faster. It is harder to build wide field-of-view displays with optical see-through techniques. Any distortions of the user s view of the real world must be corrected optically, rather than digitally, because the system has no digitized image of the real world to manipulate. Complex optics is expensive and add weight to the HMD. Wide field-of-view systems are an exception to the general trend of optical approaches being simpler and cheaper than video approaches.
3. Real and virtual view delays can be matched: Video offers an approach for reducing or avoiding problems caused by temporal mismatches between the real and virtual images. Optical see-through HMD s offer an almost instantaneous view of the real world but a delayed view of the virtual. This temporal mismatch can cause problems. With video approaches, it is possible to delay the video of the real world to match the delay from the virtual image stream.
4. Additional registration strategies: In optical see-through, the only information the system has about the user s head location comes from the head tracker. Video blending provides another source of information: the digitized image of the real scene. This digitized image means that video approaches can employ additional registration strategies unavailable to optical approaches.
5. Easier to match the brightness of the real and virtual objects: Both optical and video technologies have their roles, and the choice of technology depends upon the application requirements. Many of the mismatch assembly and repair prototypes use optical approaches, possibly because of the cost and safety issues. If successful, the equipment would have to be replicated in large numbers to equip workers on a factory floor. In contrast, most of the prototypes for medical applications use video approaches, probably for the flexibility in blending real and virtual and for the additional registration strategies offered.
3.2 Tracking and Orientation
The biggest challenge facing developers of augmented reality the need to know where the user is located in reference to his or her surroundings. There s also the additional problem of tracking the movement of users eyes and heads. A tracking system has to recognize these movements and project the graphics related to the real-world environment the user is seeing at any given movement. Currently both video see-through and optical see-through displays optically have lag in the overlaid material due to the tracking technologies currently available.
3.2.1 Indoor Tracking
Tracking is easier in small spaces than in large spaces. Trackers typically have two parts: one worn by the tracked person or object and other built into the surrounding environment, usually within the same room. In optical trackers, the targets LED s or reflectors, for instance can be attached to the tracked person or to the object, and an array of optical sensors can be embedded in the room s ceiling. Alternatively the tracked users can wear the sensors, and targets can be fixed to the ceiling. By calculating the distance to reach visible target, the sensors can determine the user s position and orientation.
Researchers at the University of North Carolina-Chapel Hill have developed a very precise system that works within 500 sq feet. The HiBall Tracking System is an optoelectronic tracking system made of two parts:
Six user-mounted, optical sensors.
Infrared-light-emitting diodes (LED s) embedded in special ceiling panels.
The system uses the known location of LED s the known geometry of the user-mounted optical sensors and a special algorithm to compute and report the user s position and orientation. The system resolves linear motion of less than 0.2 millimeters, and angular motions less than 0.03 degrees. It has an update rate of more than 1500Hz, and latency is kept at about one millisecond. In everyday life, people rely on several senses-including what they see, cues from their inner ears and gravity s pull on their bodies- to maintain their bearings. In a similar fashion, Hybrid Trackers draw on several sources of sensory information. For example, the wearer of an AR display can be equipped with inertial sensors (gyroscope and accelerometers) to record changes in head orientation. Combining this information with data from optical, video or ultrasonic devices greatly improve the accuracy of tracking.
3.2.2Out door Tracking
Head orientation is determined with a commercially available hybrid tracker that combines gyroscopes and accelerometers with magnetometers that measure the earth s magnetic field. For position tracking we take advantage OF a high-precision version of the increasingly popular Global Positioning system receiver.
A GPS receiver can determine its position by monitoring radio signals from navigation satellites. GPS receivers have an accuracy of about 10 to 30 meters. An augmented reality system would be worthless if the graphics projected were of something 10 to 30 meters away from what you were actually looking at.
User can get better result with a technique known as differential GPS. In this method, the mobile GPS receiver also monitors signals from another GPS receiver and a radio transmitter at a fixed location on the earth. This transmitter broadcasts the correction based on the difference between the stationary GPS antenna s known and computed positions. By using these signals to correct the satellite signals, the differential GPS can reduce the margin of error to less than one meter.
The system is able to achieve the centimeter-level accuracy by employing the real-time kinematics GPS, a more sophisticated form of differential GPS that also compares the phases of the signals at the fixed and mobile receivers. Trimble Navigation reports that they have increased the precision of their global positioning system (GPS) by replacing local reference stations with what they term a Virtual Reference Station (VRS). This new VRS will enable users to obtain a centimeter-level positioning without local reference stations; it can achieve long-range,
AUGMENTED REALITY - sribs2007 - 10-04-2017
[attachment=1538]
AUGMENTED REALITY
A B S T R A C T
Augmented reality adds information and meaning to a real object or place. Unlike virtual reality, augmented reality does not create a simulated reality. Instead, it takes a real object or space and uses technologies to add contextual data to deepen understanding of it. This paper surveys the field of Augmented Reality, in which 3-D virtual objects are integrated into a real environment in real time. It describes the medical, manufacturing, visualization, entertainment and military applications that have been explored. This paper describes the characteristics of Augmented Reality systems, including a brief discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective Augmented Reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. On the spectrum between virtual reality, which creates immersible, computer-generated environments, and the real world, augmented reality is closer to the real world. Augmented reality adds graphics, sounds, haptics and smell to the natural world as it exists. You can expect video games to drive the development of augmented reality, but this technology will have countless applications. Everyone from tourists to military troops will benefit from the ability to place computer-generated graphics in their field of vision.
Name : Suryendu sangam Samal
Regd. : 0601289094
A Seminar Report for the Final Review On AUGMENTED REALITY
No: 0601289086
7th Semester
I N T R O D U C T I O N
This paper describes the current state-of-the-art in Augmented Reality. It describes work performed at many different sites and explains the issues and problems encountered when building Augmented Reality systems. It summarizes the tradeoffs and approaches taken so far to overcome these problems and speculates on future directions that deserve exploration.Section 1 describes what Augmented Reality is and the motivations for developing this technology. Section 2 discusses the issues involved in building an Augmented Reality system. Currently, two of the biggest problems are in registration and sensing: the subjects of Sections 3 and 4. Section 5 describes the advantage of augmented reality over virtual environment systems . Five classes of potential applications that have been explored are described in Section 6. Finally, Section 7 describes some areas that require further work and research. Augmented reality will truly change the way we view the world. Picture yourself walking or driving down the street. With augmented-reality displays, which will eventually look much like a normal pair of glasses, informative graphics will appear in your field of view, and audio will coincide with whatever you see. These enhancements will be refreshed continually to reflect the movements of your head. In this article, we will take a look at this future technology, its components and how it will be used. Tourists that visit historical sites, such as a Civil War battlefield do not see these locations as they were in the past, due to changes over time. It is often difficult for a modern visitor to imagine what these sites really looked like in the past. A tourist equipped with an outdoors AR system could see a computer-generated version of Living History. Tourists and students walking around the grounds with such AR displays would gain a much better understanding of these historical sites and the important events that took place there. After the basic problems with AR are solved, the ultimate goal will be photorealism has been demonstrated in feature films, but accomplishing this in an interactive application will be much harder.
Augmented Reality :
Augmented Reality (AR) is a variation of Virtual Environments (VE), or Virtual Reality as it is more commonly called. VE technologies completely immerse a user inside a synthetic environment. While immersed, the user cannot see the real world around him. In contrast, AR allows the user to see the real world, with virtual objects superimposed upon or composited with the real world. Therefore, AR supplements reality, rather than completely replacing it. Ideally, it would appear to the user that the virtual and real objects coexisted in the same space, Figure shows an example of what this might look like. All these things are optional also ,i.e. they can be ignored if the user want some specific details rather than details for everything that comes in it s way . AR can be thought of as the "middle ground" between VE (completely synthetic) and telepresence (completely real).This survey defines AR as systems that have the following three characteristics:
1) Combines real and virtual
2) Interactive in real time
3) Registered in 3-D
REALITY AUGMENTED REALITY VIRTUAL REALITY
2 DESIGN :
A see-through HMD is one device used to combine real and virtual. Standard closed-view HMDs do not allow any direct view of the real world. In contrast, a seethrough HMD lets the user see the real world, with virtual objects superimposed by optical or video technologies.
2.1 Optical see-through HMD :
Optical see-through HMDs work by placing optical combiners in front of the user's eyes. These combiners are partially transmissive, so that the user can look directly through them to see the real world. The combiners are also partially reflective, so that the user sees virtual images bounced off the combiners from head mounted monitors. This approach is similar in nature to Head-Up Displays (HUDs) commonly used in military aircraft, except that the combiners are attached to the head. The optical combiners usually reduce the amount of light that the user sees
from the real world. Since the combiners act like half-silvered mirrors, they only let in some of the light from the real world, so that they can reflect some of the light from the monitors into the user's eyes. Choosing the level of blending is a design problem. More sophisticated combiners might vary the level of contributions based upon the wavelength of light. For example, such a combiner might be set to reflect all light of a certain wavelength and none at any other wavelengths. This would be ideal with a monochrome monitor. Virtually all the light from the monitor would be reflected into the user's eyes, while almost all the light from the real world (except at the particular wavelength) would reach the user's eyes. However, most existing optical see-through HMDs do reduce the amount of light from the real world, so they act like a pair of sunglasses when the power is cut off.
2.2 Video see-through HMD :
Video see-through HMDs work by combining a closed-view HMD with one or two head-mounted video cameras. The video cameras provide the user's view of the real world. Video from these cameras is combined with the graphic images created by the scene generator, blending the real and virtual. The result is sent to the monitors in front of the user's eyes in the closed-view HMD. Figure shows a conceptual diagram of a video see-through HMD. Video composition can be done in more than one way. A simple way is to use chroma-keying, a technique used in many video special effects. The background of the computer graphic images is set to a specific color, say green, which none of the virtual objects use. Then the combining step replaces all green areas with the corresponding parts from the video of the real world. This has the effect of superimposing the virtual objects over the real world. A more sophisticated composition would use depth information. If the system had depth information at each pixel for the real world images, it could combine the real and virtual images by a 12 pixel-by-pixel depth comparison. This would allow real objects to cover virtual objects and vice-versa.
VIDEO SEE-THROUGH HMD CONCEPTUAL DIAGRAM
2.3 Monitor based AR :
AR systems can also be built using monitor-based configurations, instead of see-through HMDs. Figure shows how a monitor-based system might be built. In this case, one or two video cameras view the environment. The cameras may be static or mobile. In the mobile case, the cameras might move around by being attached to a robot, with their locations tracked. The video of the real world and the graphic images generated by a scene generator are combined, just as in the video see-through HMD case, and displayed in a monitor in front of the user. The user does not wear the display device.
MONITOR BASED AR CONCEPTUAL DIAGRAM
2.4 Trade offs between the two approaches :
The rest of this section compares the relative advantages and disadvantages of optical and video approaches, starting with optical. An optical approach has the following advantages over a video approach:
1) Simplicity:
Optical blending is simpler and cheaper than video blending. Optical approaches have only one "stream" of video to worry about: the graphic images. The real world is seen directly through the combiners, and that time delay is generally a few nanoseconds. Video blending, on the other hand, must deal with separate video streams for the real and virtual images. Both streams have inherent delays in the tens of milliseconds. Digitizing video images usually adds at least one frame time of delay to the video stream, where a frame time is how long it takes to completely update an image. A monitor that completely refreshes the screen at 60 Hz has a frame time of 16.67 ms. The two streams of real and virtual images must be properly synchronized or temporal distortion results. Also, optical see-through HMDs with narrow field-of-view combiners offer views of the real world that have little distortion. Video cameras almost always have some amount of distortion that must be compensated for, along with any distortion from the optics in front of the display devices. Since video requires cameras and combiners that optical approaches do not need, video will probably be more expensive and complicated to build than optical-based systems.
2) Resolution:
Video blending limits the resolution of what the user sees, both real and virtual, to the resolution of the display devices. With current displays, this resolution is far less than the resolving power of the fovea. Optical see-through also shows the graphic images at the resolution of the display device, but the user's view of the real world is not degraded. Thus, video reduces the resolution of the real world, while optical see-through does not.
3) Safety:
Video see-through HMDs are essentially modified closed-view HMDs. If the power is cut off, the user is effectively blind. This is a safety concern in some applications. In contrast, when power is removed from an optical seethrough HMD, the user still has a direct view of the real world. The HMD then becomes a pair of heavy sunglasses, but the user can still see.
4) No eye offset:
With video see-through, the user's view of the real world is provided by the video cameras. In essence, this puts his "eyes" where the video cameras are. In most configurations, the cameras are not located exactly where the user's eyes are, creating an offset between the cameras and the real eyes. The distance separating the cameras may also not be exactly the same as the user's interpupillary distance (IPD). This difference between camera locations and eye locations introduces displacements from what the user sees compared to what he expects to see. For example, if the cameras are above the user's eyes, he will see the world from a vantage point slightly taller than he is used to. Video see-through can avoid the eye offset problem through the use of mirrors to create another set of optical paths that mimic the paths directly into the user's eyes. Using those paths, the cameras will see what the user's eyes would normally see without the HMD. However, this adds complexity to the HMD design. Offset is generally not a difficult design problem for optical see-through displays. While the user's eye can rotate with respect to the position of the HMD, the resulting errors are tiny. Using the eye's center of rotation as the viewpoint in the computer graphics model should eliminate any need for eye tracking in an optical see-through HMD.
Video blending offers the following advantages over optical blending:
1) Flexibility in composition strategies:
A basic problem with optical seethrough is that the virtual objects do not completely obscure the real world objects, because the optical combiners allow light from both virtual and real sources. Building an optical see-through HMD that can selectively shut out the light from the real world is difficult. In a normal optical system, the objects are designed to be in focus at only one point in the optical path: the user's eye. Any filter that would selectively block out light must be placed in the optical path at a point where the image is in focus, which obviously cannot be the user's eye. Therefore, the optical system must have two places where the image is in focus: at the user's eye and the point of the hypothetical filter. This makes the optical design much more difficult and complex. No existing optical see-through HMD blocks incoming light in thisfashion. Thus, the virtual objects appear ghost-like and semi-transparent. This damages the illusion of reality because occlusion is one of the strongest depth cues. In contrast, video see-through is far more flexible about how it merges the real and virtual images. Since both the real and virtual are available in digital form, video seethrough compositors can, on a pixel-by-pixel basis, take the real, or the virtual, or some blend between the two to simulate transparency. Because of this flexibility, video see-through may ultimately produce more compelling environments than optical see-through approaches.
2) Wide field-of-view:
Distortions in optical systems are a function of the radial distance away from the optical axis. The further one looks away from the center of the view, the larger the distortions get. A digitized image taken through a distorted optical system can be undistorted by applying image processing techniques to unwrap the image, provided that the optical distortion is well characterized. This requires significant amounts of computation, but this constraint will be less important in the future as computers become faster. It is harder to build wide field-of-view displays with optical see-through techniques. Any distortions of the user's view of the real world must be corrected optically, rather than digitally, because the system has no digitized image of the real world to manipulate. Complex optics are expensive and add weight to the HMD. Wide field-of-view systems are an exception to the general trend of optical approaches being simpler and cheaper than video approaches.
3) Real and virtual view delays can be matched:
Video offers an approach for reducing or avoiding problems caused by temporal mismatches between the real and virtual images. Optical see-through HMDs offer an almost instantaneous view of the real world but a delayed view of the virtual. This temporal mismatch can cause problems. With video approaches, it is possible to delay the video of the real world to match the delay from the virtual image stream.
4) Additional registration strategies:
In optical see-through, the only information the system has about the user's head location comes from the head tracker. Video blending provides another source of information: the digitized image of the real scene. This digitized image means that video approaches can employ additional registration strategies unavailable to optical approaches.
Both optical and video technologies have their roles, and the choice of technology depends on the application requirements. Many of the mechanical assembly and repair prototypes use optical approaches, possibly because of the cost and safety issues. If successful, the equipment would have to be replicated in large numbers to equip workers on a factory floor. In contrast, most of the prototypes for medical applications use video approaches, probably for the flexibility in blending real and virtual and for the additional registration strategies offered.
3 Registration :
3.1 The registration problem
One of the most basic problems currently limiting Augmented Reality applications is the registration problem. The objects in the real and virtual worlds must be properly aligned with respect to each other, or the illusion that the two worlds coexist will be compromised. More seriously, many applications demand accurate registration. For example, recall the needle biopsy application. If the virtual object is not where the real tumor is, the surgeon will miss the tumor and the biopsy will fail. Without accurate registration, Augmented Reality will not be accepted in many applications. For example, a user wearing a closed-view HMD might hold up her real hand and see a virtual hand. This virtual hand should be displayed exactly where she would see her real hand, if she were not wearing an HMD. But if the virtual hand is wrong by five millimeters, she may not detect that unless actively looking for such errors. The same error is much more obvious in a see-through HMD, where the conflict is visual-visual. Furthermore, a phenomenon known as visual capture makes it even more difficult to detect such registration errors. Visual capture is the tendency of the brain to believe what it sees rather than what it feels, hears, etc. That is, visual information tends to override all other senses. When watching a television program, a viewer believes the sounds come from the mouths of the actors on the screen, even though they actually come from a speaker in the TV. Ventriloquism works because of visual capture. Similarly, a user might believe that her hand is where the virtual hand is drawn, rather than where her real hand actually is, because of visual capture. This effect increases the amount of registration error users can tolerate in Virtual Environment systems. If the errors are systematic, users might even be able to adapt to the new environment, given a long exposure time of several hours or days. Augmented Reality demands much more accurate registration than Virtual Environments. Imagine the same scenario of a user holding up her hand, but this time wearing a see-through HMD. Registration errors now result in visual-visual conflicts between the images of the virtual and real hands. Such conflicts are easy to detect because of the resolution of the human eye and the sensitivity of the human visual system to differences. Registration of real and virtual objects is not limited to AR. Special-effects artists seamlessly integrate computer-generated 3-D objects with live actors in film and video. The difference lies in the amount of control available. With film, a director can carefully plan each shot, and artists can spend hours per frame, adjusting each by hand if necessary, to achieve perfect registration. As an interactive medium, AR is far more difficult to work with. The AR system cannot control the motions of the HMD wearer. The user looks where she wants, and the system must respond within tens of milliseconds. Registration errors are difficult to adequately control because of the high accuracy requirements and the numerous sources of error. These sources of error can be divided into two types: static and dynamic. Static errors are the ones that cause registration errors even when the user's viewpoint and the objects in the environment remain completely still. Dynamic errors are the ones that have no effect until either the viewpoint or the objects begin moving. For current HMD-based systems, dynamic errors are by far the largest contributors to registration errors, but static errors cannot be ignored either. The next two sections discuss static and dynamic errors and what has been done to reduce them.
3.1.1 Static errors
The three main sources of static errors are:
3.1.1.1 Distortion in the optics:
Optical distortions exist in most camera and lens systems, both in the cameras that record the real environment and in the optics used for the display. Because distortions are usually a function of the radial distance away from the optical axis, wide field-of-view displays can be especially vulnerable to this error. Near the center of the field-of-view, images are relatively undistorted, but far away from the center, image distortion can be large. For example, straight lines may appear curved. In a see-through HMD with narrow field-of-view displays, the optical combiners add virtually no distortion, so the user's view of the real world is not warped. However, the optics used to focus and magnify the graphic images from the display monitors can introduce distortion. This mapping of distorted virtual images on top of an undistorted view of the real world causes static registration errors. The cameras and displays may also have nonlinear distortions that cause errors. Optical distortions are usually systematic errors, so they can be mapped and compensated. This mapping may not be trivial, but it is often possible. For example,
describes the distortion of one commonly-used set of HMD optics. The distortions might be compensated by additional optics. An alternate approach is to do the compensation digitally. This can be done by image warping techniques, both on the digitized video and the graphic images. Typically, this involves predistorting the images so that they will appear undistorted after being displayed. Digital compensation methods can be computationally expensive, often requiring special hardware to accomplish in real time.
3.1.1.2 Errors in the tracking system:
Errors in the reported outputs from the tracking and sensing systems are often the most serious type of static registration errors. These distortions are not easy to measure and eliminate, because that requires another "3-D ruler" that is more accurate than the tracker being tested. These errors are often non-systematic and difficult to fully characterize. Almost all commercially available tracking systems are not accurate enough to satisfy the requirements of AR systems.
3.1.1.3 Mechanical misalignments:
Mechanical misalignments are discrepancies between the model or specification of the hardware and the actual physical properties of the real system. For example, the combiners, optics, and monitors in an optical see-through HMD may not be at the expected distances or orientations with respect to each other. If the frame is not sufficiently rigid, the various component parts may
change their relative positions as the user moves around, causing errors. Mechanical misalignments can cause subtle changes in the position and orientation of the projected virtual images that are difficult to compensate. While some alignment errors can be calibrated, for many others it may be more effective to "build it right" initially.
3.1.2 Dynamic errors :
Dynamic errors occur because of system delays, or lags. The end-to-end system delay is defined as the time difference between the moment that the tracking system measures the position and orientation of the viewpoint to the moment when the generated images corresponding to that position and orientation appear in the displays. These delays exist because each component in an Augmented Reality system requires some time to do its job. The delays in the tracking subsystem, the communication delays, the time it takes the scene generator to draw the appropriate images in the frame buffers, and the scanout time from the frame buffer to the
displays all contribute to end-to-end lag. End-to-end delays of 100 ms are fairly typical on existing systems. Simpler systems can have less delay, but other systems have more. Delays of 250 ms or more can exist on slow, heavily loaded, or networked systems. End-to-end system delays cause registration errors only when motion occurs. Assume that the viewpoint and all objects remain still. Then the lag does not cause registration errors. No matter how long the delay is, the images generated are appropriate, since nothing has moved since the time the tracker measurement was taken. Compare this to the case with motion. For example, assume a user wears a see-through HMD and moves her head. The tracker measures the head at an initial time t. The images corresponding to time t will not appear until some future time t2, because of the end-to-end system delays. During this delay, the user's head remains in motion, so when the images computed at time t finally appear, the user sees them at a different location than the one they were computed for. Thus, the images are incorrect for the time they are actually viewed. To the user, the virtual objects appear to "swim around" and "lag behind" the real objects. This was graphically System delays seriously hurt the illusion that the real and virtual worlds coexist because they cause large registration errors. With a typical end-to-end lag of 100 ms and a moderate head rotation rate of 50 degrees per second, the angular dynamic error is 5 degrees. At a 68 cm arm length, this results in registration errors of almost 60 mm. System delay is the largest single source of registration error in existing AR systems, outweighing all others combined .
3.1.2.1 Reduce system lag:
The most direct approach is simply to reduce, or ideally eliminate, the system delays. If there are no delays, there are no dynamic errors. Unfortunately, modern scene generators are usually built for throughput, not minimal latency. It is sometimes possible to reconfigure the software to sacrifice throughput to minimize latency. For example, the SLATS system completes rendering a pair of interlaced NTSC images in one field time (16.67 ms) on Pixel-Planes. Being careful about synchronizing pipeline tasks can also reduce the end-to-end lag. System delays are not likely to completely disappear anytime soon. Some believe that the current course of technological development will automatically solve this problem. Unfortunately, it is difficult to reduce system delays to the point where they are no longer an issue. Recall that registration errors must be kept to a small fraction of a degree. At the moderate head rotation rate of 50 degrees per second, system lag must be 10 ms or less to keep angular errors below 0.5 degrees. Just scanning out a frame buffer to a display at 60 Hz requires 16.67 ms. It might be possible to build an HMD system with less than 10 ms of lag, but the drastic cut in throughput and the expense required to construct the system would make alternate solutions attractive. Minimizing system delay is important, but reducing delay to the point where it is no longer a source of registration error is not currently practical.
3.1.2.2 Match temporal streams:
In video-based AR systems, the video camera and digitization hardware impose inherent delays on the user's view of the real world. This is potentially a blessing when reducing dynamic errors, because it allows the temporal streams of the real and virtual images to be matched. Additional delay is added to the video from the real world to match the scene generator delays in generating the virtual images. This additional delay to the video stream will probably not remain constant, since the scene generator delay will vary with the complexity of the rendered scene. Therefore, the system must dynamically synchronize the two streams. Note that while this reduces conflicts between the real and virtual, now both the real and virtual objects are delayed in time.
3.1.2.3 Predict:
The last method is to predict the future viewpoint and object locations. If the future locations are known, the scene can be rendered with these future locations, rather than the measured locations. Then when the scene finally appears, the viewpoints and objects have moved to the predicted locations, and the graphic images are correct at the time they are viewed. For short system delays
(under 80 ms), prediction has been shown to reduce dynamic errors by up to an order of magnitude. Accurate predictions require a system built for realtime measurements and computation. Using inertial sensors makes predictions more accurate by a factor of 2-3. Predictors have been developed for a few AR systems, but the majority were implemented and evaluated with VE systems. More work needs to be done on ways of comparing the theoretical performance of various predictors and in developing prediction models that better match actual head motion .
3.2 Current status :
The registration problem is far from solved. Many systems assume a static viewpoint, static objects, or even both. Even if the viewpoint or objects are allowed to move, they are often restricted in how far they can travel. Registration is shown under controlled circumstances, often with only a small number of real-world objects, or where the objects are already well-known to the system. For example, registration may only work on one object marked with fiducials, and not on any other objects in the scene. Much more work needs to be done to increase the domains in which registration is robust. Duplicating registration methods remains a nontrivial task, due to both the complexity of the methods and the additional hardware required. If simple yet effective solutions could be developed, that would speed the acceptance of AR systems.
4 Sensing :
Accurate registration and positioning of virtual objects in the real environment requires accurate tracking of the user's head and sensing the locations of other objects in the environment. The biggest single obstacle to building effective Augmented Reality systems is the requirement of accurate, long-range sensors and trackers that report the locations of the user and the surrounding objects in the environment. Commercial trackers are aimed at the needs of Virtual Environments and motion capture applications. Compared to those two applications, Augmented Reality has much stricter accuracy requirements and demands larger working volumes. No tracker currently provides high accuracy at long ranges in real time. More work needs to be done to develop sensors and trackers that can meet these stringent requirements. Specifically, AR demands more from trackers and sensors in three areas :
Greater input variety and bandwidth
Higher accuracy
Longer range
4.1 Input variety and bandwidth :
VE systems are primarily built to handle output bandwidth: the images displayed, sounds generated, etc. The input bandwidth is tiny: the locations of the user's head and hands, the outputs from the buttons and other control devices, etc. AR systems, however, will need a greater variety of input sensors and much more input bandwidth. There are a greater variety of possible input sensors than output displays. Outputs are limited to the five human senses. Inputs can come
from anything a sensor can detect. It is speculated that Augmented Reality may be useful in any application that requires displaying information not directly available or detectable by human senses by making that information visible (or audible, touchable, etc.). Other future applications
might use sensors to extend the user's visual range into infrared or ultraviolet frequencies, and remote sensors would let users view objects hidden by walls or hills. Conceptually, anything not detectable by human senses but detectable by machines might be transduced into something that a user can sense in an AR system. Range data is a particular input that is vital for many AR applications. The AR system knows the distance to the virtual objects, because that model is built into the system. But the AR system may not know where all the real objects are in the
environment. The system might assume that the entire environment is measured at the beginning and remains static thereafter. However, some useful applications will require a dynamic environment, in which real objects move, so the objects must be tracked in real time. Thus, a significant modeling effort may be required and should be taken into consideration when building an AR application.
4.2 High accuracy :
The accuracy requirements for the trackers and sensors are driven by the accuracies needed for visual registration, as described in Section 3. For many approaches, the registration is only as accurate as the tracker. Therefore, the AR system needs trackers that are accurate to around a millimeter and a tiny fraction of a degree, across the entire working range of the tracker. Few trackers can meet this specification, and every technology has weaknesses. Some mechanical trackers are accurate enough, although they tether the user to a limited working volume. Magnetic trackers are vulnerable to distortion by metal in the environment, which exists in many desired AR application environments. Ultrasonic trackers suffer from noise and are difficult to make accurate at long ranges because of variations in the ambient temperature. Optical technologies have distortion and calibration problems. Inertial trackers drift with time. Of the
individual technologies, optical technologies show the most promise due to trends toward high-resolution digital cameras, real-time photogrammetric techniques, and structured light sources that result in more signal strength at long distances. Future tracking systems that can meet the stringent requirements of AR will probably be hybrid systems, such as a combination of inertial and optical technologies. Using multiple technologies opens the possibility of covering for each technology's weaknesses by combining their strengths. Attempts have been made to calibrate the distortions in commonly-used magnetic tracking systems. These have succeeded at removing much of the gross error from the tracker at long ranges, but not to the level required by AR systems. For example, mean errors at long ranges can be reduced from several inches to around one inch. The requirements for registering other sensor modes are not nearly as stringent. For example, the human auditory system is not very good at localizing deep bass sounds, which is why subwoofer placement is not critical in a home theater system.
4.3 Long range :
Few trackers are built for accuracy at long ranges, since most VE applications do not require long ranges. Motion capture applications track an actor's body parts to control a computer-animated character or for the analysis of an actor's movements. This is fine for position recovery, but not for orientation. Orientation recovery is based upon the computed positions. Even tiny errors in those positions can cause orientation errors of a few degrees, which is too large for AR systems. A scalable system is one that can be expanded to cover any desired range, simply by adding more modular components to the system. This is done by building a cellular tracking system, where only nearby sources and sensors are used to track a user. As the user walks around, the set of sources and sensors changes, thus achieving large working volumes while avoiding long distances between the current working set of sources and sensors. While scalable trackers can be effective, they are complex and by their very nature have many components,
making them relatively expensive to construct. The Global Positioning System (GPS) is used to track the locations of vehicles almost anywhere on the planet. It might be useful as one part of a long range tracker for AR systems. However, by itself it will not be sufficient. The best reported
accuracy is approximately one centimeter, assuming that many measurements are integrated (so that accuracy is not generated in real time), when GPS is run in differential mode. That is not sufficiently accurate to recover orientation from a set of positions on a user. Tracking an AR system outdoors in real time with the required accuracy has not been demonstrated and remains an open problem.
5 Comparison against virtual environments :
The overall requirements of AR can be summarized by comparing them against the requirements for Virtual Environments, for the three basic subsystems that they require.
5.1 Scene generator:
Rendering is not currently one of the major problems in AR. VE systems have much higher requirements for realistic images because they completely replace the real world with the virtual environment. In AR, the virtual images only supplement the real world. Therefore, fewer virtual objects need to be drawn, and they do not necessarily have to be realistically rendered in order to serve the purposes of the application. For example, in the annotation applications, text and 3-D wireframe drawings might suffice. Ideally, photorealistic graphic objects would be seamlessly merged with the real environment, but more basic problems have to be solved first.
5.2 Display device:
The display devices used in AR may have less stringent requirements than VE systems demand, again because AR does not replace the real world. For example, monochrome displays may be adequate for some AR applications, while virtually all VE systems today use full color. Optical see-through HMDs with a small field-of-view may be satisfactory because the user can still see the real world with his peripheral vision; the see-through HMD does not shut off the
user's normal field-of-view. Furthermore, the resolution of the monitor in an optical see-through HMD might be lower than what a user would tolerate in a VE application, since the optical see-through HMD does not reduce the resolution of the real environment.
5.3 Tracking and sensing:
While in the previous two cases AR had lower requirements than VE, that is not the case for tracking and sensing. In this area, the requirements for AR are much stricter than those for VE systems since it is done in real time.
6 MOTIVATION AND APPLICATIONS :
Why is combining real andvirtual objects in 3-D useful? Augmented Reality enhances a user's perception of and interaction with the real world. The virtual objects display information that the user cannot directly detect with his own senses. It can be otherwise termed as Intelligence Amplification. At least five classes of potential AR applications have been explored: medical visualization, maintenance and repair, annotation, entertainment and military aircraft navigation and targeting. The next section describes work that has been done in each area. While these do not cover every potential application area of this technology, they do cover the areas explored so far.
6.1 Medical
Doctors could use Augmented Reality as a visualization and training aid for surgery. It may be possible to collect 3-D datasets of a patient in real time, using noninvasive sensors like Magnetic Resonance Imaging (MRI), Computed Tomography scans (CT), or ultrasound imaging. These datasets could then be rendered and combined in real time with a view of the real patient. AR technology could provide an internal view without the need for larger incisions. AR might also be helpful for general medical visualization tasks in the surgical room. The information from the non-invasive sensors would be directly displayed on the patient, showing exactly where to perform the operation. AR might also be useful for training purposes. Virtual instructions could remind a novice surgeon of the required steps, without the need to look away from a patient to consult a manual.
6.2 Manufacturing and repair
Another category of Augmented Reality applications is the assembly, maintenance, and repair of complex machinery. Instructions might be easier to understand if they were available, not as manuals with text and pictures, but rather as 3-D drawings superimposed upon the actual equipment, showing step-by-step the tasks that need to be done and how to do them. These superimposed 3-D drawings can be animated, making the directions even more explicit.
6.3 Annotation and visualization
AR could be used to annotate objects and environments with public or private information. Applications using public information assume the availability of public databases to draw upon. For example, a hand-held display could provide information about the contents of library shelves as the user walks around the library A user can point at parts of an engine model and the AR system displays the name of the part that is being pointed at .AR might give architects "X-ray vision" inside a building, showing where the pipes, electric lines, and structural supports are inside the walls. Similarly, virtual lines and objects could aid navigation and scene understanding during poor visibility conditions, such as underwater or in fog.
6.4 Entertainment
In the entertainment field AR has still bigger achievements . The actors stand in front of a large blue screen, while a computer-controlled motion camera records the scene. Since the camera's location is tracked, and the actor's motions are scripted, it is possible to digitally composite the actor into a 3-D virtual background. The entertainment industry sees this as a way to reduce production costs: creating and storing sets virtually is potentially cheaper than constantly building new physical sets from scratch. It can be further enhanced by populating the environment with intelligent virtual creatures that respond to user actions .
6.5 Military aircraft
For many years, military aircraft and helicopters have used Head-Up Displays (HUDs) and Helmet-Mounted Sights (HMS) to superimpose vector graphics upon the pilot's view of the real world. Besides providing basic navigation and flight information, these graphics are sometimes registered with targets in the environment, providing a way to aim the aircraft's weapons. Future generations of combat aircraft will be developed with an HMD built into the pilot's helmet.
7. Future directions
This section identifies areas and approaches that require further research to produce improved AR systems.
7.1 Hybrid approaches:
Future tracking systems may be hybrids, because combining approaches can cover weaknesses. The same may be true for other problems in AR. For example, current registration strategies generally focus on a single strategy. Future systems may be more robust if several techniques are combined. An example is combining vision-based techniques with prediction. If the fiducials are not available, the system switches to open-loop prediction to reduce the registration errors, rather than breaking down completely. The predicted viewpoints in turn produce a more accurate initial location estimate for the vision-based techniques.
7.2 Real-time systems and time-critical computing:
Many VE systems are not truly run in real time. Instead, it is common to build the system, often on UNIX, and then see how fast it runs. This may be sufficient for some VE applications. Since everything is virtual, all the objects are automatically synchronized with each other. AR is a different story. Now the virtual and real must be synchronized, and the real world "runs" in real time. Therefore, effective AR systems must be built with real time performance in mind. Accurate timestamps must be available. Operating systems must not arbitrarily swap out the AR software process at any time, for arbitrary durations. Systems must be built to guarantee completion within specified time budgets, rather than just "running as quickly as possible." These are characteristics of flight simulators and a few VE systems. Constructing and debugging real-time systems is often painful and difficult, but the requirements for AR demand real-time performance.
7.3 Perceptual and psychophysical studies:
Augmented Reality is an area ripe for psychophysical studies. How much lag can a user detect? How much registration error is detectable when the head is moving? Besides questions on perception, psychological experiments that explore performance issues are also needed. How much does head-motion prediction improve user performance on a specific task? How much registration error is tolerable for a specific application before performance on that task degrades substantially? Is the allowable error larger while the user moves her head versus when she stands still? Furthermore, not much is known about potential optical illusions caused by errors or conflicts in the simultaneous display of real and virtual objects. Few experiments in this area have been performed. Jannick Rolland, Frank Biocca and their students conducted a study of the effect caused by eye displacements in video see-through HMDs. They found that users partially adapted to the eye displacement, but they also had negative aftereffects after removing the HMD.
7.4 Portability:
AR requires making the equipment self-contained and portable. Existing tracking technology is not capable of tracking a user outdoors at the required accuracy.
7.5 Multimodal displays:
Almost all work in AR has focused on the visual sense: virtual graphic objects and overlays. But augmentation might apply to all other senses as well. In particular, adding and removing 3-D sound is a capability that could be useful in some AR applications.
7.6 Social and political issues:
Technological issues are not the only ones that need to be considered when building a real application. There are also social and political dimensions when getting new technologies into the hands of real users. Sometimes, perception is what counts, even if the technological reality is different. For example, if workers perceive lasers to be a health risk, they may refuse to use a system with lasers in the display or in the trackers, even if those lasers are eye safe.
Ergonomics and ease of use are paramount considerations. Whether AR is truly a cost-effective solution in its proposed applications has yet to be determined. Another important factor is whether or not the technology is perceived as a threat to jobs, as a replacement for workers, especially with many corporations undergoing recent layoffs. AR may do well in this regard, because it is intended as a tool to make the user's job easier, rather than something that completely replaces the human worker. Although technology transfer is not normally a subject of academic papers, it is a real problem. Social and political concerns should not be ignored during attempts to move AR out of the research lab and into the hands of real users.
Conclusion
Augmented Reality is far behind Virtual Environments in maturity. No commercial vendor currently sells an HMD-based Augmented Reality system. Today AR systems are primarily found in academic and industrial research laboratories. The first deployed HMD-based AR systems will probably be in the application of aircraft manufacturing. Both Boeing and McDonnell Douglas are exploring this technology. The former uses optical approaches, while the latter is pursuing video approaches. Annotation and visualization applications in restricted, limited-range environments are deployable today. Applications in medical visualization will take longer. Prototype visualization aids have been used on an experimental basis, but the stringent registration requirements and ramifications of mistakes will postpone common usage for many years. AR will probably be used for medical training before it is commonly used in surgery. The next generation of combat aircraft will have Helmet-Mounted Sights with graphics registered to targets in the environment. Augmented Reality is a relatively new field, where most of the research efforts have occurred in the past ten years. One area where a breakthrough is required is tracking an HMD outdoors at the accuracy required by AR. If this is accomplished, several interesting applications will become possible. Two examples are: navigation maps and visualization of past and future environments. The first application is a navigation aid to people walking outdoors. An AR system makes navigation easier by performing the association step automatically. If the user's position and orientation are known, and the AR system has access to a digital map of the area, then the AR system can draw the map in 3-D directly upon the user's view. The second application is visualization of locations and events as they were in the past or as they will be after future changes are performed.
References
[1] Teleoperators and Virtual Environments , 355-385 A Survey of Augmented Reality Ronald T. Azuma Hughes Research Laboratories 3011 Malibu Canyon Road, MS RL96 Malibu, CA 90265 [email protected]
[2] Ronald Azuma HRL Laboratories, Yohan Baillot NRL Virtual Reality Lab/ITT Advanced Engineering, Reinhold Behringer Rockwell Scienti.c Steven Feiner Columbia University
[3] Simon Julier NRL Virtual Reality Lab/ITT Advanced Engineering, Blair MacIntyre Georgia Institute of Technology, Recent Advances in Augmented Reality
[4] James R Vallino Interactive Augmented Reality Submitted in partial Fulfillment of the Requirements for the Degree Doctor of philosophy. http://se.rit.edu/ jrv/research/ar/introduction.html
CONTENTS
1. Introduction ..1
2. Design ..2
Optical see-through HMD
Video see-through HMD
Monitor based AR
3. Registration problem .8
Static problem
Dynamic problem
4. Sensing problem .. 14
Greater input variety and bandwidth
Higher accuracy
Long range
5. Advantage of Augmented Reality over Virtual Environment.. .17
6. Applications . .. ..18
Medical
Maintenance and repair
Visualization
Entertainment
Military
7. Future areas of work and research .19
AUGMENTED REALITY - riyarose - 10-04-2017
[attachment=2584]
1. ABSTRACT
This paper surveys the field of Augmented Reality, in which 3-D virtual objects are integrated into a 3-D real environment in real time. It describes the medical, manufacturing, visualization, path planning, entertainment and military applications that have been explored. This paper describes the characteristics of Augmented Reality systems, including a detailed discussion of the tradeoffs between optical and video blending approaches. Registration and sensing errors are two of the biggest problems in building effective Augmented Reality systems, so this paper summarizes current efforts to overcome these problems. Future directions and areas requiring further research are discussed. This survey provides a starting point for anyone interested in researching or using Augmented Reality.
2. INTRODUCTION
Augmented reality (AR) refers to computer displays that add virtual information to a user's sensory perceptions. Most AR research focuses on see-through devices, usually worn on the head that overlay graphics and text on the user's view of his or her surroundings. In general it superimposes graphics over a real world environment in real time.
Getting the right information at the right time and the right place is key in all these applications. Personal digital assistants such as the Palm and the Pocket PC can provide timely information using wireless networking and Global Positioning System (GPS) receivers that constantly track the handheld devices. But what makes augmented reality different is how the information is presented: not on a separate display but integrated with the user's perceptions. This kind of interface minimizes the extra mental effort that a user has to expend when switching his or her attention back and forth between real-world tasks and a computer screen. In augmented reality, the user's view of the world and the computer interface literally become one.
Mixed Reality
Real Environment
Augmented Reality
Augmented Virtuality
Virtual Environment
Milligrams Reality-Virtuality Continuum
Between the extremes of real life and Virtual Reality lies the spectrum of Mixed Reality, in which views of the real world are combined in some proportion with views of a virtual environment. Combining direct view, stereoscopic video, and stereoscopic graphics, Augmented Reality describes that class of displays that consists primarily of a real environment, with graphic enhancements or augmentations.
3
In Augmented Virtuality, real objects are added to a virtual environment. In Augmented reality, virtual objects are added to real world.
An AR system supplements the real world with virtual (computer generated) objects that appear to co-exist in the same space as the real world. Virtual Reality is a synthetic environment
2.1 Comparison between AR and virtual environments
The overall requirements of AR can be summarized by comparing them against the requirements for Virtual Environments, for the three basic subsystems that they require.
1) Scene generator: Rendering is not currently one of the major problems in AR. VE
systems have much higher requirements for realistic images because they completely
replace the real world with the virtual environment. In AR, the virtual images only
supplement the real world. Therefore, fewer virtual objects need to be drawn, and
they do not necessarily have to be realistically rendered in order to serve the purposes
of the application.
2) Display device: The display devices used in AR may have less stringent requirements than VE systems demand, again because AR docs not replace the real world. For example, monochrome displays may be adequate for some AR applications, while virtually all VE systems today use full color. Optical see-through HMDs with a small field-of-view may be satisfactory because the user can still see the real world with his peripheral vision; the see-through HMD does not shut off the user's normal field-of-view. Furthermore, the resolution of the monitor in an optical see-through HMD might be lower than what a user would tolerate in a VE application, since the optical see-through HMD does not reduce the resolution of the real environment.
3) Tracking and sensing: While in the previous two cases AR had lower requirements than VE, which is not the case for tracking and sensing. In this area, the requirements for AR are much stricter than those for VE systems. A major reason for this is the registration problem.
3. DEVELOPMENTS
Although augmented reality may seem like the stuff of science fiction, researchers have been building prototype systems for more than three decades. The first was developed in the 1960s by computer graphics pioneer Ivan Sutherland and his students at Harvard University and the University of Utah.
In the 1970s and 1980s a small number of researchers studied augmented reality at institutions such as the U.S. Air Force's Armstrong Laboratory, the NASA Ames Research Center and the University of North Carolina at Chapel Hill.
It wasn't until the early 1990s that the term "augmented reality" was coined by scientists at Boeing who were developing an experimental AR system to help workers assemble wiring harnesses.
In 1996 developers at Columbia University developed 'The Touring Machine'
In 2001 MIT came up with a very compact AR system known as 'MIThrill'
Presently research is being done in developing BARS (Battlefield Augmented Reality Systems) by engineers at Naval Research Laboratory, Washington D.C.
4. WORKING
AR systems track the position and orientation of the user's head so that the overlaid material can be aligned with the user's view of the world. Through this process, known as registration, graphics software can place a three-dimensional image of a teacup, for example, on top of a real saucer and keep the virtual cup fixed in that position as the user moves about the room. AR systems employ some of the same hardware technologies used in virtual-reality research, but there's a crucial difference: whereas virtual reality brashly aims to replace the real world, augmented reality respectfully supplements it.
Augmented reality is still in an early stage of research and development at various universities and high-tech companies. Eventually, possibly by the end of this decade, we will see the first mass-marketed augmented-reality system, which one researcher calls "the Walkman of the 21st century." What augmented reality attempts to do is not only superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective. Here are the three components needed to make an augmented-reality system work:
Head Mounted Display
Tracking System
Mobile Computing System
4.1 Head Mounted Displays
Just as monitors allow us to see text and graphics generated by computers, head-mounted displays (HMDs) will enable us to view graphics and text created by augmented-reality systems. There are two basic types of HMDS:
optical see-through
video see-through
4.1.1 Optical see-through displays:
A simple approach to optical see-through display employs a mirror beam splitter a half-silvered mirror that both reflects and transmits light. If properly oriented in front of the user's eye, the beam splitter can reflect the image of a computer display into the user's line of sight yet still allow light from the surrounding world to pass through. Such beam splitters, which are called combiners, have long been used in "head-up" displays for fighter-jet pilots (and, more recently, for drivers of luxury cars). Lenses can be placed between the beam splitter and the computer display to focus the image so that it appears at a comfortable viewing distance. If a display and optics are provided for each eye, the view can be in stereo. Sony makes a see-through display that some researchers use, called the Glasstron.
OPTICAL SEE-THROUGH DtSPUY
ILIUUID CKYSIAL L'ISI'l/Vi
CDMPTNSATING PRISM
1 TRACKING TARGETS
VIRTU41
spinrR :
:CHPUTER
4.1.2 Video see-through displays:
In contrast, a video see-through display uses video mixing technology, originally developed for television special effects, to combine the image from a head-worn camera with synthesized graphics. The merged image is typically presented on an opaque head-worn display. With careful design, the camera can be positioned so that its optical path is close to that of the user's eye; the video image thus approximates what the user would normally see. As with optical see-through displays, a separate system can be provided for each eye to support stereo vision.
Video composition can be done in more than one way. A simple way is to use chroma-keying: a technique used in many video special effects. The background of the computer graphic images is set to a specific color, say green, which none of the virtual objects use. Then the combining step replaces all green areas with the corresponding parts from the video of the real world. This has the effect of superimposing the virtual objects over the real world. A more sophisticated composition would use depth information. If the system had depth information at each pixel for the real world images, it could combine the real and virtual images by a
A different approach is the virtual retinal display, which forms images directly on the retina. These displays, which Micro Vision is developing commercially, literally draw on the retina with low-power lasers whose modulated beams are scanned by micro electro-mechanical mirror assemblies that sweep the beam horizontally and vertically. Potential advantages include high brightness and contrast, low power consumption, and large depth of field.
Each of the approaches to see-through display design has its pluses and minuses. Optical see-through systems allow the user to see the real world with full resolution and field of view. But the overlaid graphics in current optical see-through systems are not opaque and therefore cannot completely obscure the physical objects behind them. As a result, the superimposed text may be hard to read against some backgrounds, and the three-dimensional graphics may not produce a convincing illusion. Furthermore, although a user focuses physical objects depending on their distance, virtual objects are all focused in the plane of the display. This means that a virtual object that is intended to be at the same position as a physical object may have a geometrically correct projection, yet the user may not be able to view both objects in focus at the same time.
In video see-through systems, virtual objects can fully obscure physical ones and can be combined with them using a rich variety of graphical effects. There is also no discrepancy between how the eye focuses virtual and physical objects, because both are viewed on the same plane. The limitations of current video technology, however, mean that the quality of the visual experience of the real world is significantly decreased, essentially to the level of the synthesized graphics, with everything focusing at the same apparent distance. At present, a video camera and display are no match for the human eye.
VIDEO SEE-THROUGH DISPLAY
COMPUTER (WORN ON H!P)
j LIQUID CKYS ML DISPLAY
1 DISPLAY AND
TRACKHie [TRACKING) i SENSORS TARGETS
i \
COMPUTER
VIRTUAL
SPinrR .
4.2 An optical approach has the following advantages over a video approach:
I) Simplicity: Optical blending is simpler and cheaper than video blending. Optical approaches have only one "stream" of video to worry about: the graphic images. The real world is seen directly through the combiners, and that time delay is generally a few nanoseconds. Video blending, on the other hand, must deal with separate video streams for the real and virtual images. The two streams of real and virtual images must be properly synchronized or temporal distortion results. Also, optical see-through HMDs with narrow field-of-view combiners offer views of the real world that have little distortion. Video cameras almost always have some amount of distortion that must be compensated for, along with any distortion from the optics in front of the display devices. Since video requires cameras and combiners that optical approaches do not need, video will probably be more expensive and complicated to build than optical-based systems.
2) Resolution: Video blending limits the resolution of what the user sees, both real and virtual, to the resolution of the display devices. With current displays, this resolution is far less than the resolving power of the fovea. Optical see-through also shows the graphic images at the resolution of the display device, but the user's view of the real world is not degraded. Thus, video reduces the resolution of the real world, while optical see-through does not.
3) Safety: Video see-through HMDs are essentially modified closed-view HMDs. If the power is cut off, the user is effectively blind. This is a safety concern in some applications. In contrast, when power is removed from an optical see-through HMD, the user still has a direct view of the real world. The HMD then becomes a pair of heavy sunglasses, but the user can still see.
4) No eye offset: With video see-through, the user's view of the real world is provided by the video cameras. In essence, this puts his "eyes" where the video cameras are. In most configurations, the cameras are not located exactly where the user's eyes are, creating an offset between the cameras and the real eyes. The distance separating the cameras may also not be exactly the same as the user's interpupillary distance (IPD). This difference between camera locations and eye locations introduces displacements from what the user sees compared to what he expects to see. For example, if the cameras are above the user's eyes, he will see the world from a vantage point slightly taller than he is used to.
4.3 Video blending offers the following advantages over optical blending:
1) Flexibility in composition strategies: A basic problem with optical see-through is
that the virtual objects do not completely obscure the real world objects, because the
optical combiners allow light from both virtual and real sources. Building an optical
see-through HMD that can selectively shut out the light from the real world is
difficult. Any filter that would selectively block out light must be placed in the optical
path at a point where the image is in focus, which obviously cannot be the user's eye.
Therefore, the optical system must have two places where the image is in focus: at the
user's eye and the point of the hypothetical filter. This makes the optical design much
more difficult and complex. No existing optical see-through HMD blocks incoming
light in this fashion. Thus, the virtual objects appear ghost-like and semi-transparent.
This damages the illusion of reality because occlusion is one of the strongest depth
cues. In contrast, video see-through is far more flexible about how it merges the real
and virtual images. Since both the real and virtual are available in digital form, video
see-through compositors can, on a pixel-by-pixel basis, take the real, or the virtual, or
some blend between the two to simulate transparency. Because of this flexibility,
video see-through may ultimately produce more compelling environments than
optical see-through approaches.
2) Wide field-of-view: Distortions in optical systems are a function of the radial
distance away from the optical axis. The further one looks away from the center of
the view, the larger the distortions get. A digitized image taken through a distorted
optical system can be undistorted by applying image processing techniques to unwrap
the image, provided that the optical distortion is well characterized. This requires
significant amounts of computation, but this constraint will be less important in the
future as computers become faster. It is harder to build wide field-of-view displays
with optical see-through techniques. Any distortions of the user's view of the real
world must be corrected optically, rather than digitally, because the system has no
digitized image of the real world to manipulate. Complex optics are expensive and
add weight to the HMD. Wide field-of-view systems are an exception to the general trend of optical approaches being simpler and cheaper than video approaches.
3) Real and virtual view delays can be matched: Video offers an approach for reducing or avoiding problems caused by temporal mismatches between the real and virtual images. Optical see-through HMDs offer an almost instantaneous view of the real world but a delayed view of the virtual. This temporal mismatch can cause problems. With video approaches, it is possible to delay the video of the real world to match the delay from the virtual image stream.
4) Additional registration strategies: In optical see-through, the only information the system has about the user's head location comes from the head tracker. Video blending provides another source of information: the digitized image of the real scene. This digitized image means that video approaches can employ additional registration strategies unavailable to optical approaches.
5) Easier to match the brightness of real and virtual objects: Both optical and video technologies have their roles, and the choice of technology depends on the application requirements. Many of the mechanical assembly and repair prototypes use optical approaches, possibly because of the cost and safety issues. If successful, the equipment would have to be replicated in large numbers to equip workers on a factory floor. In contrast, most of the prototypes for medical applications use video approaches, probably for the flexibility in blending real and virtual and for the additional registration strategies offered.
5. TRACKING AND ORIENTATION
The biggest challenge facing developers of augmented reality is the need to know where the user is located in reference to his or her surroundings. There's also the additional problem of tracking the movement of users' eyes and heads. A tracking system has to recognize these movements and project the graphics related to the real-world environment the user is seeing at any given moment. Currently, both video sec-through and optical see-through displays typically have lag in the overlaid material due to the tracking technologies currently available.
5.1 INDOOR TRACKING:
Tracking is easier in small spaces than in large spaces. Trackers typically have two parts: one worn by the tracked person or object and the other built into the surrounding environment, usually within the same room. In optical trackers, the targets LEDs or reflectors, for instance can be attached to the tracked person or object, and an array of optical sensors can be embedded in the room's ceiling. Alternatively, the tracked users can wear the sensors, and the targets can be fixed to the ceiling. By calculating the distance to each visible target, the sensors can determine the user's position and orientation.
Researchers at the University of North Carolina-Chapel Hill have developed a very precise system that works within 500 square feet. The HiBall Tracking System is an optoelectronic tracking system made of two parts:
six user-mounted, optical sensors
infrared-light-emitting diodes (LEDs) embedded in special ceiling panels
The system uses the known location of the LEDs, the known geometry of the user-mounted optical sensors and a special algorithm to compute and report the user's position and orientation. The system resolves linear motion of less than .2 millimeters, and angular motions less than .03 degrees. It has an update rate of more than 1500 Hz, and latency is kept at about one millisecond.
In everyday life, people rely on several senses including what they see, cues from their inner ears and gravity's pull on their bodies to maintain their bearings. In a similar fashion, "hybrid trackers" draw on several sources of sensory information. For example, the wearer of an AR display can be equipped with inertial sensors (gyroscopes and accelerometers) to record changes in head orientation. Combining this information with data from the optical, video or ultrasonic devices greatly improves the accuracy of the tracking.
5.2 OUTDOOR TRACKING:
Head orientation is determined with a commercially available hybrid tracker that combines gyroscopes and accelerometers with a magnetometer that measures the earth's magnetic field. For position tracking, we take advantage of a high-precision version of the increasingly popular Global Positioning System receiver.
A GPS receiver determines its position by monitoring radio signals from navigation satellites. GPS receivers have an accuracy of about 10 to 30 meters. An augmented-reality system would be worthless if the graphics projected were of something 10 to 30 meters away from what you were actually looking at.
Users can get better results with a technique known as differential GPS. In this method, the mobile GPS receiver also monitors signals from another GPS receiver and a radio transmitter at a fixed location on the earth. This transmitter broadcasts corrections based on the difference between the stationary GPS antenna's known and computed positions. By using these signals to correct the satellite signals, differential GPS can reduce the margin of error to less than one meter. Our system is able to achieve centimeter-level accuracy by employing real-time kinematic GPS, a more sophisticated form of differential GPS that also compares the phases of the signals at the fixed and mobile receivers.
Augmented-reality systems place extraordinarily high demands on the accuracy, resolution, repeatability and speed of tracking technologies. Hardware and software delays introduce a lag between the user's movement and the update of the display. As a result, virtual objects will not remain in their proper positions as the user moves about or turns his or her head. One technique for combating such errors is to equip AR systems with software that makes short-term predictions about the user's future motions by extrapolating from previous movements. And in the long run, hybrid trackers that include computer vision technologies may be able to trigger appropriate graphics overlays when the devices recognize certain objects in the user's view.
6. MOBILE COMPUTING POWER
For a wearable augmented reality system, there is still not enough computing power to create stereo 3-D graphics. So researchers are using whatever they can get out of laptops and personal computers, for now. Laptops are just now starting to be equipped with graphics processing units (GPUs).
Toshiba just added an NVidia GPU to their notebooks that is able to process more than 17-million triangles per second and 286-million pixels per second, which can enable CPU-intensive programs, such as 3-D games. But still, notebooks lag far behind NVidia has developed a custom 300-MHz 3-D graphics processor for Microsoft's Xbox game console that can produce 150 million polygons per second and polygons are more complicated than triangles. So you can see how far mobile graphics chips have to go before they can create smooth graphics like the ones you see on your home video-game system.
7. APPLICATION DOMAINS
Only recently have the capabilities of real-time video image processing, computer graphic systems and new display technologies converged to make possible the display of a virtual graphical image correctly registered with a view of the 3D environment surrounding the user. Researchers working with augmented reality systems have proposed them as solutions in many domains. The areas that have been discussed range from entertainment to military training. Many of the domains, such as medical are also proposed for traditional virtual reality systems. This section will highlight some of the proposed applications for augmented reality.
7.1 MEDICAL
Because imaging technology is so pervasive throughout the medical field, it is not surprising that this domain is viewed as one of the more important for augmented reality systems. Most of the medical applications deal with image guided surgery. Pre-operative imaging studies, such as CT or MRI scans, of the patient provide the surgeon with the necessary view of the internal anatomy. From these images the surgery is planned. Visualization of the path through the anatomy to the affected area where, for example, a tumor must be removed is done by first creating a 3D model from the multiple views and slices in the preoperative study. This is most often done mentally though some systems will create 3D volume visualizations from the image study. Augmented reality can be applied so that the surgical team can see the CT or MRI data correctly registered on the patient in the operating theater while the procedure is progressing. Being able to accurately register the images at this point will enhance the performance of the surgical team.
Another application for augmented reality in the medical domain is in ultrasound imaging. Using an optical see-through display the ultrasound technician can view a volumetric rendered image of the fetus overlaid on the abdomen of the pregnant woman. The image appears as if it was inside of the abdomen and is correctly rendered as the user moves.
7.2 ENTERTAINMENT
A simple form of augmented reality has been in use in the entertainment and news business for quite some time. Whenever you are watching the evening weather report the weather reporter is shown standing in front of changing weather maps. In the studio the reporter is actually standing in front of a blue or green screen. This real image is augmented with computer generated maps using a technique called chroma-keying. It is also possible to create a virtual studio environment so that the actors can appear to be positioned in a studio with computer generated decorating.
Movie special effects make use of digital compositing to create illusions. Strictly speaking with current technology this may not be considered augmented reality because it is not generated in real-time. Most special effects are created off line, frame by frame with a substantial amount of user interaction and computer graphics system rendering. But some work is progressing in computer analysis of the live action images to determine the camera parameters and use this to drive the generation of the virtual graphics objects to be merged.
Princeton Electronic Billboard has developed an augmented reality system that allows broadcasters to insert advertisements into specific areas of the broadcast image. For example, while broadcasting a baseball game this system would be able to place an advertisement in the image so that it appears on the outfield wall of the stadium. The electronic billboard requires calibration to the stadium by taking images from typical camera angles and zoom settings in order to build a map of the stadium including the locations in the images where advertisements will be inserted. By using pre-specified reference points in the stadium, the system automatically determines the camera angle being used and referring to the pre-defined stadium map inserts the advertisement into the correct place.
ARQuake, 76 designed using the same platform, blends users in the real world with those in a purely virtual environment. A mobile AR user plays as a combatant in the computer game 'Quake', where the game runs with a virtual model of the real environment
7.3 MILITARY TRAINING
The military has been using displays in cockpits that present information to the pilot on the windshield of the cockpit or the visor of their flight helmet. This is a form of augmented reality display. SIMNET, a distributed war games simulation system, is also embracing augmented reality technology. By equipping military personnel with helmet mounted visor displays or a special purpose rangefinder the activities of other units participating in the exercise can be imaged. While looking at the horizon, for example, the display equipped soldier could see a helicopter rising above the tree line. This helicopter could be being flown in simulation by another participant. In wartime, the display of the real battlefield scene could be augmented with annotation information or highlighting to emphasize hidden enemy units.
7.4 ENGINEERING DESIGN
Imagine that a group of designers are working on the model of a complex device for their clients. The designers and clients want to do a joint design review even though they are physically separated. If each of them had a conference room that was equipped with an augmented reality display this could be accomplished. The physical prototype that the designers have mocked up is imaged and displayed in the client's conference room in 3D. The clients can walk around the display looking at different aspects of it. To hold discussions the client can point at the prototype to highlight sections and this will be reflected on the real model in the augmented display that the designers are using. Or perhaps in an earlier stage of the design, before a prototype is built, the view in each conference room is augmented with a computer generated image of the current design built from the CAD files describing it. This would allow real time interactions with elements of the design so that either side can make adjustments and changes that are reflected in the view seen by both groups.
7.5 ROBOT PATH PLANNING
Virtual lines show a planned motion of a robot arm
Teleoperation of a robot is often a difficult problem, especially when the robot is far away, with long delays in the communication link. Under this circumstance, instead of controlling the robot directly, it may be preferable to instead control a virtual version of the robot. The user plans and specifies the robot's actions by manipulating the local virtual version, in real time. The results are directly displayed on the real world. Once the plan is tested and determined, then user tells the real robot to execute the specified plan. This avoids pilot-induced oscillations caused by the lengthy delays. The virtual versions can also predict the effects of manipulating the environment, thus serving as a planning and previewing tool to aid the user in performing the desired task. The ARGOS system has demonstrated that stereoscopic AR is an easier and more accurate way of doing robot path planning than traditional monoscopic interfaces. Others have also used registered overlays with telepresence systems.
7.6 Manufacturing, Maintenance and Repair
When the maintenance technician approaches a new or unfamiliar piece of equipment instead of opening several repair manuals they could put 011 an augmented reality display. In this display the image of the equipment would be < ugmented with annotations and information pertinent to the repair. For example, :he location of fasteners and attachment hardware that must be removed would be hi flighted. Then the inside view of the machine would highlight the boards that need to be replaced. The military has developed a wireless vest worn by personnel that is attached to an optical see-through display. The wireless connection allows the scldier to access repair manuals and images of the equipment. Future versions might register those images on the live scene and provide animation to show the procedui es that must be performed.
External view of Columbia printer maintenance application. Note that
all objects must be tracked.
Boeing researchers are developing an augmented reality display to -eplace the large work frames used for making wiring harnesses for their aircraft. Using this experimental system, the technicians are guided by the augmented display that shows the routing of the cables on a generic frame used for all harnesses The augmented display allows a single fixture to be used for making the multiple hanr
ses.7.7 Consumer Design
Virtual reality systems are already used for consumer design. Using perhaps more of a graphics system than virtual reality, when you go to the typical home store wanting to add a new deck to your house, they will show you a graphical picture of what the deck will look like. It is conceivable that a future system would allow you to bring a video tape of your house shot from various viewpoints in your backyard and in real time it would augment that view to show the new deck in its finished form attached to your house. Or bring in a tape of your current kitchen and the augmented reality processor would replace your current kitchen cabinetry with virtual images of the new kitchen that you are designing.
Applications in the fashion and beauty industry that would benefit from an augmented reality system can also be imagined. If the dress store does not have a particular style dress in your size an appropriate sized dress could be used to augment the image of you. As you looked in the three sided mirror you would see the image of the new dress on your body. Changes in hem length, shoulder styles or other particulars of the design could be viewed on you before you place the order. When you head into some high-tech beauty shops today you can see what a new hair style would look like on a digitized image of yourself. But with an advanced augmented reality system you would be able to see the view as you moved. If the dynamics of hair are included in the description of the virtual object you would also see the motion of your hair as your head moved.
7.8 Instant Information
Tourists and students could use these systems to learn more about a certain historical event. Imagine walking onto a Civil War battlefield and seeing a re-creation of historical events on a head-mounted, augmented-reality display. It would immerse you in the event, and the view would be panoramic. The recently started Archeoguide project is developing a wearable AR system for providing tourists with information about a historical site in Olympia, Greece.
7.9 Portability
In almost all Virtual Environment systems, the user is not encouraged to walk around much. Instead, the user navigates by "flying" through the environment, walking on a treadmill, or driving some mockup of a vehicle. Whatever the technology, the result is that the user stays in one place in the real world. Some AR applications, however, will need to support a user who will walk around a large environment. AR requires that the user actually be at the place where the task is to take place. "Flying," as performed in a VE system, is no longer an option. If a mechanic needs to go to the other side of a jet engine, she must physically move herself and the display devices she wears. Therefore, AR systems will place a premium on portability, especially the ability to walk around outdoors, away from controlled environments. The scene generator, the HMD, and the tracking system must all be self-contained and capable of surviving exposure to the environment. If this capability is achieved, many more applications that have not been tried will become available. For example, the ability to annotate the surrounding environment could be useful to soldiers, hikers, or tourists in an unfamiliar new location.
8. LIMITATIONS
8.1 Technological Limitations
Although we've seen much progress in the basic enabling technologies, they still primarily prevent the deployment of many AR applications. Displays, trackers, and AR systems in general need to become more accurate, lighter, cheaper, and less power consuming. By describing problems from our common experiences in building outdoor AR systems, we hope to impart a sense of the many areas that still need improvement. Displays such as the Sony Glasstron are intended for indoor consumer use and aren't ideal for outdoor use. The display isn't very bright and completely washes out in bright sunlight. The image has axed focus to appear several feet away from the user, which is often closer than the outdoor landmarks. The equipment isn't nearly as portable as desired. Since the user must wear the PC, sensors, display, batteries, and everything else required, the end result is a cumbersome and heavy backpack. Laptops today have only one CPU, limiting the amount of visual and hybrid tracking that we can do. Operating systems aimed at the consumer market aren't built to support real-time computing, but specialized real-time operating systems don't have the drivers to support the sensors and graphics in modern hardware. Tracking in unprepared environments remains an enormous challenge. Outdoor demonstrations today have shown good tracking only with significant restrictions in operating range, often with sensor suites that are too bulky and expensive for practical use. Today's systems generally require extensive calibration procedures that an end user would be unacceptably complicated. Many connectors such as universal serial bus (USB) connectors aren't rugged enough for outdoor operation and are prone to breaking. While we expect some improvements to naturally occur from other fields such as wearable computing, research in AR can reduce these difficulties through improved tracking in unprepared environments and calibration- free or auto calibration approaches to minimize set-up requirements.
8.2 User interface limitations
We need a better understanding of how to display data to a user and how the user should interact with the data. Most existing research concentrates on low-level perceptual issues, such as properly perceiving depth or how latency affects manipulation tasks. However, AR also introduces many high-level tasks, such as the need to identify what information should be provided, what's the appropriate representation for that data, and how the user should make queries and reports. For example, a user might want to walk down a street, look in a shop window, and query the inventory of that shop. To date, few have studied such issues. However, we expect significant growth in this area because research AR systems with sufficient capabilities are now more commonly available. For example, recent work suggests that the creation and presentation of narrative performances and structures may lead to more realistic and richer AR experiences.
9. Future Directions:
This section identifies areas and approaches that require further research to produce improved AR systems.
Hybrid approaches: Future tracking systems may be hybrids, because combining approaches can cover weaknesses. The same may be true for other problems in AR. For example, current registration strategies generally focus on a single strategy. Future systems may be more robust if several techniques are combined. An example is combining vision-based techniques with prediction. If the fiducials are not available, the system switches to open-loop prediction to reduce the registration errors, rather than breaking down completely. The predicted viewpoints in turn produce a more accurate initial location estimate for the vision-based techniques.
Real-time systems and time-critical computing: Many VE systems are not truly run in real time. Instead, it is common to build the system, often on UNIX, and then see how fast it runs. This may be sufficient for some VE applications. Since everything is virtual, all the objects are automatically synchronized with each other. AR is a different story. Now the virtual and real must be synchronized, and the real world "runs" in real time. Therefore, effective AR systems must be built with real time performance in mind. Accurate timestamps must be available. Operating systems must not arbitrarily swap out the AR software process at any time, for arbitrary durations. Systems must be built to guarantee completion within specified time budgets, rather than just "running as quickly as possible." These are characteristics of flight simulators and a few VE systems. Constructing and debugging real-time systems is often painful and difficult, but the requirements for AR demand real-time performance.
Perceptual and psychophysical studies: Augmented Reality is an area ripe for psychophysical studies. How much lag can a user detect How much registration error is detectable when the head is moving Besides questions on perception, psychological experiments that explore performance issues are also needed. How much does head-motion prediction improve user performance on a specific task How much registration error is tolerable for a specific application before performance on that task degrades substantially Is the allowable error larger while the user moves
her head versus when she stands still Furthermore, not much is known about potential optical illusions caused by errors or conflicts in the simultaneous display of real and virtual objects.
Portability: It is essential that potential AR applications give the user the ability to walk around large environments, even outdoors. This requires making the equipment self-contained and portable. Existing tracking technology is not capable of tracking a user outdoors at the required accuracy.
Multimodal displays: Almost all work in AR has focused on the visual sense: virtual graphic objects and overlays. But augmentation might apply to all other senses as well. In particular, adding and removing 3-D sound is a capability that could be useful in some AR applications.
10. CONCLUSION
Augmented Reality is far behind Virtual Environments in maturity. Several commercial vendors sell complete, turnkey Virtual Environment systems. However, no commercial vendor currently sells an HMD-based Augmented Reality system. A few monitor-based "virtual set" systems are available, but today AR systems are primarily found in academic and industrial research laboratories. The first deployed HMD-based AR systems will probably be in the application of aircraft manufacturing. The former uses optical approaches, while the latter is pursuing video approaches. Boeing has performed trial runs with workers using a prototype system but has not yet made any deployment decisions. Annotation and visualization applications in restricted, limited-range environments are deployable today, although much more work needs to be done to make them cost effective and flexible. Applications in medical visualization will take longer. Prototype visualization aids have been used on an experimental basis, but the stringent registration requirements and ramifications of mistakes will postpone common usage for many years. AR will probably be used for medical training before it is commonly used in surgery. The next generation of combat aircraft will have Helmet-Mounted Sights with graphics registered to targets in the environment. These displays, combined with short-range steerable missiles that can shoot at targets off-bore sight, give a tremendous combat advantage to pilots in dogfights. Instead of having to be directly behind his target in order to shoot at it, a pilot can now shoot at anything within a 60-90 degree cone of his aircraft's forward centerline.
One area where a breakthrough is required is tracking an HMD outdoors at the accuracy required by AR. If this is accomplished, several interesting applications will become possible. Two examples are described here: navigation maps and visualization of past and future environments. The first application is a navigation aid to people walking outdoors. These individuals could be soldiers advancing upon their objective, hikers lost in the woods, or tourists seeking directions to their intended destination. Today, these individuals must pull out a physical map and associate what they see in the real environment around them with the markings on the 2-D map. If landmarks are not easily identifiable, this association can be difficult to perform, as
anyone lost in the woods can attest. An AR system makes navigation easier by performing the association step automatically. If the user's position and orientation are known, and the AR system has access to a digital map of the area, then the AR system can draw the map in 3-D directly upon the user's view. Tourists and students walking around the grounds with such AR displays would gain a much better understanding of these historical sites and the important events that took place there. Similarly, AR displays could show what proposed architectural changes would look like before they are carried out. An urban designer could show clients and politicians what a now stadium would look like as they walked around the adjoining neighborhood, to better understand how the stadium project will affect nearby residents.
After the basic problems with AR are solved, the ultimate goal will be to generate virtual objects that are so realistic that they are virtually indistinguishable from the real environment. Photorealism has been demonstrated in feature films, but accomplishing this in an interactive application will be much harder. Lighting conditions, surface reflections, and other properties must be measured automatically, in real time. More sophisticated lighting, texturing, and shading capabilities must run at interactive rates in future scene generators. Registration must be nearly perfect, without manual intervention or adjustments. While these are difficult problems, they are probably not insurmountable. It took about. 25 years tQ progress from drawing stick figures on a screen to the photorealistic dinosaurs in "Jurassic Park." Within another 25 years, we should be able to wear a pair of AR glasses outdoors to see and interact with photorealistic dinosaurs eating a tree in our backyard.
11. REFERENCES:
Ronald Azuma, Steven Feiner ; Recent Advances in Augmented Reality; IEE Computer Graphics and Applications, November-December 2001 [page 34]
Columbia University's Computer Graphics and User Interfaces Lab is at cs.columbia.edu/graphics/ [28/09/02, 4:30 pm]
A list of relevant publications can be found at
cs.columbia.edu/graphics/publications/publications.html [22/09/02, 10:30 pm]
AR research sites and conferences are listed at augmented-reality.org
Information on medical applications of augmented reality is at cs.unc.edu/ us/ [30/09/02, 12:30 pm]
please read http://seminarsprojects.net/Thread-Augmented-Reality--7901 for presentation of augmented reality
AUGMENTED REALITY - Mathew - 10-04-2017
[attachment=1939]
ABSTRACT
Video games have been entertaining us for nearly 30 years, ever since Pong was introduced to arcades in the early II 970's.Computer graphics have become much more sophisticated since then, and soon, game graphics will seem all too real. In the next decade, researchers plan to pull graphics out of your television screen or computer display and integrate them into real- world environments. This new technology called augmented reality, will further blur the line between what is real and what is computer-generated by enhancing what we see, hear, feel and smell.
Augmented reality will truly change the way we view the world. Picture yourself walking or driving down the street. With augmented-reality displays, which will eventually look much like a normal pair of glasses, informative graphics will appear in your field of view, and audio will coincide with what ever you see. These enhancements will be refreshed continually to reflect the moments of your head.
Augmented reality is still in the early stage of research and development at various universities and high-tech companies. Eventually, possibly by the end of this decade we will see the first mass-marketed augmented-reality system, which can be described as "the Walkman of the 21st Century".
ACKNOWLEDGEMENT
Firstly I would like to express my sincere gratitude to the Almighty for His solemn presence throughout the seminar study .1 would also like to express my special thanks to the Principal Prof. K. Rajendran for providing an opportunity to undertake this seminar .1 am deeply indebted to our seminar coordinator Mr. Saini Jacob, Assistant Professor in the Department of Computer Science and Engineering for providing me with valuable advice and guidance during the course of the study.
I would like to extend my heartfelt gratitude to the Faculty of the Department of Computer Science and Engineering for their constructive support and cooperation at each and every juncture of the seminar study.
Finally I would like to express my gratitude to Sree Narayana Gurukulam College of Engineering for providing me with all the required facilities without which the seminar study would not have been possible.
CONTENTS
1. INTRODUCTION
Augmented reality (AR) refers to computer displays that add virtual information to a user's sensory perception. Most AR research focuses on see-through devices, usually worn on the head that overlay graphics and text on the user's view of his or her surroundings. In general it superimposes graphics over a real world environment in real time.
Getting the right information at the right time and the right place is key in all these applications. Personal digital assistants such as the Palm and the Pocket PC can provide timely information using wireless networking and Global Positioning System (GPS) receivers that constantly track the handheld devices. But what make Augmented Reality different is how the information is presented: not on a separate display but integrated with the user's perceptions. This kind of interface minimizes the extra mental effort that a user has to expend when switching his or her attention back and forth between real-world tasks and a computer screen. In augmented reality, the user's view of the world and the computer interface literally become one.
Real Augmented Augmented Virtual
Environment Reality virtuality Environment
Between the extremes of real life and Virtual Reality lies the spectrum of Mixed Reality, in which views of the real world are combined in some proportion with views of a virtual environment. Combining direct view, stereoscopic videos, and stereoscopic graphics, Augmented Reality describes that class of displays that consists primarily of a real world environment, with graphic enhancement or augmentations.
In Augmented Virtuality, real objects are added to a virtual environment. In Augmented Reality, virtual objects are added to real world. An AR system supplements the real world with virtual (computer generated) objects that appear to co-exist in the same space as the real world. Virtual Reality is a synthetic environment.
1.1 Comparison between AR and virtual environments
The overall requirements of AR can be summarized by comparing them against the requirements for Virtual Environments, for the three basic subsystems that they require.
1. Scene generator : Rendering is not currently one of the major problems in AR. VE systems have much higher requirements for realistic images because they completely replace the real world with the virtual environment . In AR, the virtual images only supplement the real world. Therefore, fewer virtual objects need to be drawn, and they do not necessarily have to be realistically rendered in order to serve the purposes of the application.
2. Display devices: The display devices used in AR may have less stringent requirements than VE systems demand, again because AR does not replace the real world. For example, monochrome displays may be adequate for some AR applications, while virtually all VE systems today use full color. Optical see-through HMD's with a small field-of-view may be satisfactory because the user can still see the real world with his peripheral vision; the see-through HMD does not shut off the user's normal field-of-view. Furthermore, the resolution of the monitor in an optical see-through HMD might be lower than what a user would tolerate in a VE application, since the optical see-through HMD does not reduce the resolution of the real environment.
3. Tracking and sending: While in the previous two cases AR had lower requirements than VE that is not the case for tracking and sensing. In this area, the requirements for AR are much stricter than those for VE systems. A major reason for this is the registration problem.
2. EVOLUTION
Although augmented reality may seem like the stuff of science fiction, researchers have been building prototype system for more than three decades. The first was developed in the 1960s by computer graphics pioneer Ivan Surtherland and his students at Harvard University.
In the 1970s and 1980s a small number of researchers studied augmented reality at institution such as the U.S. Air Force's Armstrong Laboratory, the NASA Ames Research Center and the university of North Carolina at Chapel Hill.
It wasn't until the early 1990s that the term "Augmented Reality "was coined by scientists at Boeing who were developing an experimental AR system to help workers assemble wiring harnesses.
In 1996 developers at Columbia University develop 'The Touring Machine'
In 2001 MIT came up with a very compact AR system known as "MIThrill".
Presently research is being done in developing BARS (Battlefield Augmented Reality Systems) by engineers at Naval Research Laboratory, Washington D.C.
3. WORKING
AR system tracks the position and orientation of the user's head so that the overlaid material can be aligned with the user's view of the world. Through this process, known as registration, graphics software can place a three dimensional image of a tea cup, for example on top of a real saucer and keep the virtual cup fixed in that position as the user moves about the room. AR systems employ some of the same hardware technologies used in virtual reality research, but there's a crucial differences: whereas virtual reality brashly aims to replace the real world, augmented reality respectfully supplement it.
Augmented Reality is still in an early stage of research and development at various universities and high-tech companies. Eventually, possible by the end of this decade, we will see first mass-marketed augmented reality system, which one researcher calls "The Walkman of the 21st century". What augmented reality attempts to do is not only super impose graphics over a real environment in real time, but also change those graphics to accommodate a user's head- and eye-movements, so that the graphics always fit and perspective.
Here are the three components needed to make an augmented-reality system work:
Head-mounted display Tracking system Mobile computing power
3.1 Head-Mounted Display
Just as monitor allow us to see text and graphics generated by computers, head-mounted displays (HMD's) will enable us to view graphics and text created by augmented-reality systems. There are two basic types of HMD's
- Optical see-through
- Video see-through
Optical Display Video Display
Fig 1: Optical and Video Display
3.1.1 Optical see-through display
Monitors
Real World
Head Head Tracker
Optical
V v y Combiners
Fig 2: Optical see-through HMD conceptual diagram.
A simple approach to optical see-through display employs a mirror beam splitter- a half silvered mirror that both reflects and transmits light. If properly oriented in front of the user's eye, the beam splitter can reflect the image of a computer display into the user's line of sight yet still allow light from the surrounding world to pass through. Such beam splitters, which are called combiners, have long been used in head up displays for fighter-jet- pilots (and, more recently, for drivers of luxury cars). Lenses can be placed between the beam splitter and the computer display to focus the image so that it appears at a comfortable viewing distance. If a display and optics are provided for each eye, the view can be in stereo. Sony makes a see-through display that some researchers use, called the "Glasstron".
3.1.2 Video see-through displays
Real World
Combined Video Fig 3: Video see-through HMD conceptual diagram
In contrast, a video see through display uses video mixing technology, originally developed for television special effects, to combine the image from a head worn camera with synthesized graphics. The merged image is typically presented on an opaque head worn display. With careful design the camera can be positioned so that its optical path is closed to that of the user's eye; the video image thus approximates what the user would normally see. As with optical see through displays, a separate system can be provided for each eye to support stereo vision. Video composition can be done in more than one way. A simple way is to use chroma-keying: a technique used in many video special effects. The background of the computer graphics images is set to a specific color, say green, which none of the virtual objects use. Then the combining step replaces all green areas with the corresponding parts from the video of the real world. This has the effect of superimposing the virtual objects over the real world. A more sophisticated composition would use depth information at each pixel for the real world images; it could combine the real and virtual images by a pixel-by-pixel depth comparison. This would allow real objects to cover virtual objects and vice-versa.
A different approach is the virtual retinal display, which forms images directly on the retina. These displays, which Micro Vision is developing commercially, literally draw on the retina with low power lasers modulated beams are scanned by microelectro-mechanical mirror assemblies that sweep the beam horizontally and vertically. Potential advantages include high brightness and contrast, low power consumption, and large depth of field.
it*
Fig 5: Two optical see-through HMD's, made by Hughes Electronics
3.1.3 Comparison of optical see through and video see through displays
Each of approaches to see through display design has its pluses and minuses. Optical see through systems allows the user to see the real world with resolution and field of view. But the overlaid graphics in current optical see through systems are not opaque and therefore cannot completely obscure the physical objects behind them. As result, the superimposed text may be hard to read against some backgrounds, and three-dimensional graphics may not produce a convincing illusion. Furthermore, although a focuses physical objects depending on their distance, virtual objects are all focused in the plane of the display. This means that a virtual object that is intended to be at the same position as a physical object may have a geometrically correct projection, yet the user may not be able to view both objects in focus at the same time.
In video see-through systems, virtual objects can fully obscure physical ones and can be combined with them using a rich variety of graphical effects. There is also discrepancy between how the eye focuses virtual and physical objects, because both are viewed on same plane. The limitations of current video technology, however, mean that the quality of the visual experience of the real world is significantly decreased, essentially to the level of the synthesized graphics, with everything focusing at the same apparent distance. At present, a video camera and display is no match for the human eye.
An optical approach has the following advantages over a video approach
1. Simplicity: Optical blending is simpler and cheaper than video blending. Optical approaches have only one "stream" of video to worry about: the graphic images. The real world is seen directly through the combiners, and that time delay is generally a few nanoseconds. Video blending, on the other hand, must deal with separate video streams for the real and virtual images. The two streams of real and virtual images must be properly synchronized or temporal distortion results. Also, optical see through HMD's with narrow field of view combiners offer views of the real world that have little distortion. Video cameras almost always have some amount of distortion that must be compensated for, along with any distortion from the optics in front of the display devices. Since video requires cameras and combiners that optical approaches do not need, video will probably be more expensive and complicated to build than optical based systems.
2. Resolution: Video blending limits the resolution of what the user sees, both real and virtual, to the resolution of the display devices. With current displays, this resolution is far less than the resolving power of the fovea. Optical see-through also shows the graphic images at the resolution of the display devices, but the user's view of the real world is not degraded. Thus, video reduces the resolution of the real world, while optical see-through does not.
3. Safety: Video see-through HMD's are essentially modified closed-view HMD's. If the power is cut off, the user is effectively blind. This is a safety concern in some applications. In contrast, when power is removed from an optical see-through HMD, the user still has a direct view of the real world. The HMD then becomes a pair of heavy sunglasses, but the user can still see.
4. No eye offset: With video see-through, the user's view of the real world is provided by the video cameras. In essence, this puts his "eyes" where the video cameras are not located exactly where the user's eyes are, creating an offset between the cameras and the real eyes. The distance separating the cameras may also not be exactly the same as the user's interpupillary distance (IPD). This difference between camera locations and eye locations introduces displacements from what the user sees compared to what he expects to see. For example, if the cameras are above the user's eyes, he will see the world from a vantage point slightly taller than he is used to.
Video blending offers the following advantages over optical blending
1. Flexibility in composition strategies: A basic problem with optical see-through is that the virtual objects do not completely obscure the real world objects, because the optical combiners allow light from both virtual and real sources. Building an optical see-through HMD that can selectively shut out the light from the real world is difficult. Any filter that would selectively block out light must be placed in the optical path at a point where the image is in focus, which obviously cannot be the user's eye. Therefore, the optical system must have two places where the image is in focus: at the user's eye and the point of the hypothetical filter. This makes the optical design much more difficult and complex. No existing optical see-through HMD blocks incoming light in this fashion. Thus, the virtual objects appear Ghost-like and semi-transparent. This damages the illusion of reality because occlusion is one of the strongest depth cues. In contrast, video see-through is far more flexible about how it merges the real and virtual images. Since both the real and virtual are available in digital form, video see-through compositors can, on a pixel-by-pixel basis, take the real, or the virtual, or some blend between the two to simulate transparency.
2. Wide field-of-view: Distortions in optical systems are a function of the radial distance away from the optical axis. The further one looks away from the center of the view, the larger the distortions get. A digitized image taken through a distorted optical system can be undistorted by applying image processing techniques to unwrap the image, provided that the optical distortion is well characterized. This requires significant amount of computation, but this constraint will be less important in the future as computers become faster. It is harder to build wide field-of-view displays with optical see-through techniques. Any distortions of the user's view of the real world must be corrected optically, rather than digitally, because the system has no digitized image of the real world to manipulate. Complex optics is expensive and add weight to the HMD. Wide field-of-view systems are an exception to the general trend of optical approaches being simpler and cheaper than video approaches.
3. Real and virtual view delays can be matched: Video offers an approach for reducing or avoiding problems caused by temporal mismatches between the real and virtual images. Optical see-through HMD's offer an almost instantaneous view of the real world but a delayed view of the virtual. This temporal mismatch can cause problems. With video approaches, it is possible to delay the video of the real world to match the delay from the virtual image stream.
4. Additional registration strategies: In optical see-through, the only information the system has about the user's head location comes from the head tracker. Video blending provides another source of information: the digitized image of the real scene. This digitized image means that video approaches can employ additional registration strategies unavailable to optical approaches.
5. Easier to match the brightness of the real and virtual objects: Both optical and video technologies have their roles, and the choice of technology depends upon the application requirements. Many of the mismatch assembly and repair prototypes use optical approaches, possibly because of the cost and safety issues. If successful, the equipment would have to be replicated in large numbers to equip workers on a factory floor. In contrast, most of the prototypes for medical applications use video approaches, probably for the flexibility in blending real and virtual and for the additional registration strategies offered.
3.2 Tracking and Orientation
The biggest challenge facing developers of augmented reality the need to know where the user is located in reference to his or her surroundings. There's also the additional problem of tracking the movement of users eyes and heads. A tracking system has to recognize these movements and project the graphics related to the real-world environment the user is seeing at any given movement. Currently both video see-through and optical see-through displays optically have lag in the overlaid material due to the tracking technologies currently available.
3.2.1 Indoor Tracking
Tracking is easier in small spaces than in large spaces. Trackers typically have two parts: one worn by the tracked person or object and other built into the surrounding environment, usually within the same room. In optical trackers, the targets - LED's or reflectors, for instance - can be attached to the tracked person or to the object, and an array of optical sensors can be embedded in the room's ceiling. Alternatively the tracked users can wear the sensors, and targets can be fixed to the ceiling. By calculating the distance to reach visible target, the sensors can determine the user's position and orientation.
Researchers at the University of North Carolina-Chapel Hill have developed a very precise system that works within 500 sq feet. The HiBall Tracking System is an optoelectronic tracking system made of two parts: Six user-mounted, optical sensors.
Infrared-light-emitting diodes (LED's) embedded in special ceiling panels.
The system uses the known location of LED's the known geometry of the user-mounted optical sensors and a special algorithm to compute and report the user's position and orientation. The system resolves linear motion of less than 0.2 millimeters, and angular motions less than 0.03 degrees. It has an update rate of more than 1500Hz, and latency is kept at about one millisecond. In everyday life, people rely on several senses-including what they see, cues from their inner ears and gravity's pull on their bodies- to maintain their bearings. In a similar fashion, "Hybrid Trackers" draw on several sources of sensory information. For example, the wearer of an AR display can be equipped with inertial sensors (gyroscope and accelerometers) to record changes in head orientation. Combining this information with data from optical, video or ultrasonic devices greatly improve the accuracy of tracking.
3.2.20ut door Tracking
Head orientation is determined with a commercially available hybrid tracker that combines gyroscopes and accelerometers with magnetometers that measure the earth's magnetic field. For position tracking we take advantage OF a high-precision version of the increasingly popular Global Positioning system receiver.
A GPS receiver can determine its position by monitoring radio signals from navigation satellites. GPS receivers have an accuracy of about 10 to 30 meters. An augmented reality, system would be worthless if the graphics projected were of something 10 to 30 meters away from what you were actually looking at.
User can get better result with a technique known as differential GPS. In this method, the mobile GPS receiver also monitors signals from another GPS receiver and a radio transmitter at a fixed location on the earth. This transmitter broadcasts the correction based on the difference between the stationary GPS antenna's known and computed positions. By using these signals to correct the satellite signals, the differential GPS can reduce the margin of error to less than one meter.
The system is able to achieve the centimeter-level accuracy by employing the real-time kinematics GPS, a more sophisticated form of differential GPS that also compares the phases of the signals at the fixed and mobile receivers. Trimble Navigation reports that they have increased the precision of their global
positioning system (GPS) by replacing local reference stations with what they term a Virtual Reference Station (VRS). This new VRS will enable users to obtain a centimeter-level positioning without local reference stations; it can achieve long-range, real-time kinematics (RTK) precision over greater distances via wireless communications wherever they are located. Real-time kinematics technique is a way to use GPS measurements to generate positioning within one to two centimeters (0.39 to 0.79 inches). RTK is often used as the key component in navigational system or automatic machine guidance.
Unfortunately, GPS is not the ultimate answer to position tracking. The satellite signals are relatively weak and easily blocked by buildings or even foliage. This rule out useful tracking indoors or in places likes midtown Manhattan, where rows of tall building block most of the sky. GPS tracking works well in wade open spaces and relatively low buildings.
GPS provide far too few updates per second and is too inaccurate to support the precise overlaying of graphics on nearby objects. Augmented Reality system places extra ordinary high demands on the accuracy, resolution, repeatability and speed of tracking technologies. Hardware and software delays introduce a lag between the user's movement and the update of the display. As a result, virtual objects will not remain in their proper position as the user moves about or turns his or her head. One technique for combating such errors is to equip AR system with software that makes short-term predictions about the user's future motion by extrapolating from previous movements. And in the long run, hybrid trackers that include computer vision technologies may be able trigger appropriate graphics overlays when the devices recognize certain objects in the user's view.
4. MOBILE COMPUTING POWER
For a wearable augmented realty system, there is still not enough computing power to create stereo 3-D graphics. So researchers are using whatever they can get out of laptops and personal computers, for now. Laptops are just now starting to be equipped with graphics processing unit (GPU's). Toshiba just now added a NVIDIA to their notebooks that is able to process more than 17-million triangles per second and 286-million pixels per second, which can enable CPU-intensive programs, such as 3D games. But still notebooks lag far behind-NVIDIA has developed a custom 300-MHz 3-D graphics processor for Microsoft's Xbox game console that can produce 150 million polygon per second and polygons are more complicated than triangles. So you can see how far mobiles graphics chips have to go before they can create smooth graphics like the ones you see on your home video-game system.
5.APPLICATIONS
Only recently have the capabilities of real-time video image processing, computer graphics systems and new display technologies converged to make possible the display of a virtual graphical image correctly registered with a view of the 3D environment surrounding the user. Researchers working with the AR system have proposed them as solutions in many domains. The areas have been discussed range from entertainment to military training. Many of the domains, such as medical are also proposed for traditional virtual reality systems. This section will highlight some of the proposed application for augmented reality.
5.1 Medical
Because imaging technology is so pervasive throughout the medical field, it is not surprising that this domain is viewed as one of the more important for augmented reality systems. Most of the medical application deal with image guided surgery. Pre-operative imaging studies such as CT or MRI scans, of the patient provide the surgeon with the necessary view of the internal anatomy. From these images the surgery is planned. Visualization of the path through the anatomy to the affected area where, for example, a tumor must be removed is done by first creating the 3D model from the multiple views and slices in the preoperative study. This is most often done mentally though some systems will create 3D volume visualization from the image study. AR can be applied so that the surgical team can see the CT or MRI data correctly registered on the patient in the operating theater while the procedure is progressing. Being able to accurately register the images at this point will enhance the performance of the surgical team.
Another application for AR in the medical domain is in ultra sound imaging. Using an optical see-through display the ultrasound technician
can view a volumetric rendered image of the fetus overlaid on the abdomen of the pregnant woman. The image appears as if it were inside of the abdomen and is correctly rendered as the user moves.
5.2 Entertainment
A simple form of the augmented reality has been in use in the entertainment and news business for quite some time. Whenever you are watching the evening weather report the weather reporter is shown standing in the front of changing weather maps. In the studio the reporter is standing in front of a blue or a green screen. This real image is augmented with the computer generated maps using a technique called chroma-keying. It is also possible to create a virtual studio environment so that the actors can appear to be positioned in a studio with computer generated decorating.
Movie special effects make use of digital computing to create illusions. Strictly speaking with current technology this may not be considered augmented reality because it is not generated in the real-time. Most special effects are created off-line, frame by frame with a substantial amount of user interaction and computer graphics system rendering. But some work is progressing in computer analysis of the live action images to determine the camera parameters and use this to drive the generation of the virtual graphics objects to be merged.
Princeton Electronics Billboard has developed an augmented reality system that allows broadcasters to insert advertisement into specific areas of the broadcast image. For example, while broadcasting a baseball game this system would be able to place an advertisement in the image so that it appears on the outfield wall of the stadium. By using pre-specified reference points in the stadium, the system automatically determines the camera angles being used and referring to the pre-defined stadium map inserts the advertisement into the current place. AR QUAKE, 76 designed using the same platform, blends users in the real world with those in a purely virtual environment. A mobile AR user plays as a combatant in the computer game Quake, where the game runs with a virtual model of the real environment.
Fig 8: AR in sports broadcasting. The annotations on the race cars and the yellow first down line are inserted into the broad cast in real time.
5.3 Military Training
The military has been using display in cockpits that present information to the pilot on the windshield of the cockpit or the visor of their flight helmet. This is a form of Augmented Reality display. SIMNET, a distributed war games simulating system, is also embracing augmented reality technology. By equipping military personnel with helmet mounted visor displays or a special purpose rangefinder the activities of other units participating in the exercise can be imaged. While looking at the horizon, for example, the display equipped soldier could see a helicopter rising above the tree line. This helicopter could be being flown in simulation by another participant. In war time, the display of the real battlefield scene could be augmented with annotation information or highlighting to emphasize hidden enemy units.
5.4 Engineering Design
Imagine that a group of designers are working on the model of a complex device for their clients. The designers and clients want to do a joint design reviews even though they are physically separated. If each of them had a conference room that was equipped with an augmented re4ality display this could be accomplished. The physical prototype that the designers have mocked up is imaged and displayed in the client's conference room in 3D. The clients can walk around display looking at different aspects of it. To hold the discussion the client can point at the prototype to highlight sections and this will be reflected on the real model in the augmented display that the designers are using. Or perhaps in an earlier stage of the design, before a prototype is built, the view in each conference room is augmented with a computer generated image of the current design built from the CAD file describing it. This would allow real time interactions with elements of the design so that either side can make adjustments and change that are reflected in the view seen by both groups.
5.5 Robotics and Telerobotics
In the domain of robotics and Telerobotics an augmented display can assist the user of the system. A Telerobotics operator uses a visual image of the remote workspace to guide the robot. Annotation of the view would still be useful just as it is when the scene is in front of the operator. There is an added potential benefit. Since often the view of the remote scene is monoscopic, augmentation with wire frame drawings of structures in the view can facilitate visualization of the remote 3D geometry. If the operator is attempting a motion it could be practiced on a virtual robot that is visualized as an augmentation to the real scene. The operator can decide to proceed with the motion after seeing the results. The robot motion could then be executed directly which in a telerobotics application would eliminate any oscillations caused by long delays to the remote site.
Fig 9: Virtual lines show a planned motion of a robot arm
I
5.6 Manufacturing, maintenance and repair
When the maintenance technician approaches a new or unfamiliar piece of equipment instead of opening several repair manuals they could put on an
1 augmented reality display. In this display the image of the equipment would be
i
I augmented with annotations and information pertinent to the repair. For example,
the location of fasteners and attachment hardware that must be removed would be
! highlighted. Then the inside view of the machine would highlight the boards that
need to be replaced. The military has developed a wireless vest worn by personnel that is attached to an optical see-through display. The wireless connection allows the soldier to access repair manuals and images of the equipment. Future versions might register those images on the live scene and provide animation to show the procedures that must be performed.Boeing researchers are developing an augmented reality display to replace the large work frames used for making wiring harnesses for their aircraft. Using this experimental system, the technicians are guided by the augmented display that shows the routing of the cables on a generic frame used for all harnesses. The augmented display allows a single fixture to be used for making the multiple harnesses.
5.7 Consumer design
Virtual reality systems are already used for consumer design. Using perhaps more of a graphics system than virtual reality, when you go to the typical home store wanting to add a new deck to your house, they will show you a graphical picture of what the deck will look like. It is conceivable that a future system would allow you to bring a video tape of your house shot from various viewpoints in your backyard and in real time it would augment that view to show the new deck in its finished form attached to your house. Or bring in a tape of your current kitchen and the augmented reality processor would replace your current kitchen cabinetry with virtual images of the new kitchen that you are designing.
Applications in the fashion and beauty industry that would benefit from an augmented reality system can also be imaged. If the dress store does not have a particular style dress in your size an appropriate sized dress could be used to augment the image of you. As you looked in the three sided mirror you would see the image of the new dress on your body. Changes in hem length, shoulder styles or other particulars of the design could be viewed on you before you place the order. When you head into some high-tech beauty shops today you can see what a new hair style would look like on a digitized image of yourself. But with an advanced augmented reality system you would be able to see the view as you moved. If the dynamics of hair are included in the description of the virtual object you would also see the motion of hair as your head moved.
5.8 Instant information
Tourists and students could use these systems to learn more about a certain historical event. Imagine walking onto a Civil War battlefield and seeing a re-creation of historical events on a head-mounted, augmented reality display. It would immerse you in the event, and the view would be panoramic. The recently started Archeoguide project is developing a wearable AR system for providing tourists with information about a historical site in Olympia, Greece.
6. CONCLUSION
Augmented reality is far behind Virtual Environments in maturity. Several commercial vendors sell complete, turnkey Virtual Environment systems. However, no commercial vendor currently sells an HMD-based Augmented Reality system. A few monitor-based "virtual set" systems are available, but today AR systems are primarily found in academic and industrial research laboratories.
The first deployed HMD-based AR systems will probably be in the application of aircraft manufacturing. Both Boeing and McDonnell Douglas are exploring this technology. The former uses optical approaches, while the letter is pursuing video approaches. Boeing has performed trial runs with workers using a prototype system but has not yet made any deployment decisions. Annotation and visualization applications in restricted, limited range environments are deployable today, although much more work needs to be done to make them cost effective and flexible.
Applications in medical visualization will take longer. Prototype visualization aids have been used on an experimental basis, but the stringent registration requirements and ramifications of mistakes will postpone common usage for many years. AR will probably be used for medical training before it is commonly used in surgery.
The next generation of combat aircraft will have Helmet Mounted Sights with graphics registered to targets in the environment. These displays, combined with short-range steer able missiles that can shoot at targets off-bore sight, give a tremendous combat advantage to pilots in dogfights. Instead of having to be directly behind his target in order to shoot at it, a pilot can now shoot at anything within a 60-90 degree cone of his aircraft's forward centerline. Russia and Israel currently have systems with this capability, and the U.S is expected to field the AIM-9X missile with its associated Helmet-mounted sight in 2002.
Augmented Reality is a relatively new field, where most of the research efforts have occurred in the past four years. Because of the numerous challenges and unexplored avenues in this area, AR will remain a vibrant area of research for at least the next several years.
After the basic problems with AR are solved, the ultimate goal will be to generate virtual objects that are so realistic that they are virtually indistinguishable from the real environment. Photorealism has been demonstrated in feature films, but accomplishing this in an interactive application will be much harder. Lighting conditions, surface reflections, and other properties must be measured automatically, in real time. More sophisticated lighting, texturing, and shading capabilities must run at interactive rates in future scene generators. Registration must be nearly perfect, without manual intervention or adjustments.
While these are difficult problems, they are probably not insurmountable. It took about 25 years to progress from drawing stick figures on a screen to the photorealistic dinosaurs in "Jurassic Park." Within another 25 years, we should be able to wear a pair of AR glasses outdoors to see and interact with photorealistic dinosaurs eating a tree in our backyard.
n. FUTURE DIRECTIONS
This section identifiers areas and approaches that require further researches to produce improved AR systems.
Hybrid approach
Further tracking systems may be hybrids, because combining approaches can cover weaknesses. The same may be true for other problems in AR. For example, current registration strategies generally focus on a single strategy. Further systems may be more robust if several techniques are combined. An example is combining vision-based techniques with prediction. If the fiducially are not available, the system switches to open-loop prediction to reduce the registration errors, rather than breaking down completely. The predicted viewpoints in turn produce a more accurate initial location estimate for the vision-based techniques.
Real time systems and time-critical computing
Many VE systems are not truly run in real time. Instead, it is common to build the system, often on UNIX, and then see how fast it runs. This may be sufficient for some VE applications. Since everything is virtual, all the objects are automatically synchronized with each other. AR is different story. Now the virtual and real must be synchronized, and the real world "runs" in real time. Therefore, effective AR systems must be built with real time performance in mind. Accurate timestamps must be available. Operating systems must not arbitrarily swap out the AR software process at any time, for arbitrary durations. Systems must be built ton guarantee completion within specified time budgets, rather than just "running as quickly as possible". These are characteristics of flight simulators and a few VE systems. Constructing and debugging real-time systems is often painful and difficult, but the requirements for AR demand real-time performance.
Perceptual and psychophysical studies
Augmented reality is an area ripe for psychophysical studies. How much lag can a user detect? How much registration error is detectable when the head is
moving? Besides questions on perception, psychological experiments that explore performance issues are also needed. How much does head-motion prediction improve user performance on a specific task? How much registration error is tolerable for a specific application before performance on that task degrades substantially? Is the allowable error larger while the user moves her head versus when she stands still? Furthermore, no much is known about potential optical illusion caused by errors or conflicts in the simultaneous display of real and virtual objects.
Portability
It is essential that potential AR applications give the user the ability to walk around large environments, even outdoors. This requires making the requirement self-continued and portable. Existing tracking technology is not capable of tracking a user outdoors at the required accuracy.
Multimodal displays
Almost all work in AR has focused on the visual sense: virtual graphic objects and overlays. But augmentation might apply to all other senses as well. In particular, adding and removing 3-D sound is a capability that could be useful in some AR applications.
8. BIBLIOGRAPHY
> A survey of Augmented Reality by Ronald T. Azuma
> Recent Advances in Augmented Reality by Ronald TAzuma, Yohan Baillot, Reinhold Beringer, Simon Julier and Blair Maclntyre
> Augmented Reality: A new way of seeing. Steven K Feiner
> Augmented Reality and computer Augmented Environment, available at http://csl.sony.co.jp/project/ar/ref.html
1. INTRODUCTION 1
2. EVOLUTION 4
3. WORKING 5
3.1 HEAD MOUNTED DIPLAY 6
3.1.1 OPTICAL SEE-THROUGH DP LAYS 6
3.1.2 VIDEO SEE-THROUGH DISPLAYS 7
3.1.3 COMPARISON OF OPTICAL AND VIDEO SEE THROUGH DISPLAY 9
3.2 TRACKING AND ORIENTATION 13
3.2.1 INDOOR TRACKING 13
3.2.2 OUTDOOR TRACKING 14
4. MOBILE COMPUTING POWER 16
5. APPLICATION 17
5.1 MEDICAL 17
5.2 ENTERTAINMENT 18
5.3 MILITARY TRAIN8ING 19
5.4 ENGINEERING DESIGN 20
5.5 ROBOTICS AND TELEROBOTICS 20
5.6 MANUFACTURING, MAINTENANCE AND REPAIR 21
5.7 CONSUMER DESIGN 22
5.8 INSTANT INFORMATION 22
6. CONCLUSION 23
7. FUTURE DIRECTIONS 25
! 8. BIBLIOGRAPHY 27
AUGMENTED REALITY - gracy - 10-04-2017
ABSTRACT
Augmented reality (AR) refers to computer displays that add virtual information to a user's sensory perceptions. Most AR research focuses on "see-through" devices, usually worn on the head that overlay graphics and text on the user's view of his or her surroundings. AR systems track the position and orientation of the user's head so that the overlaid material can be aligned with the user's view of the world.
Consider what AR could make routinely possible. A repairperson viewing a broken piece of equipment could see instructions highlighting the parts that need to be inspected. A surgeon could get the equivalent of x-ray vision by observing live ultrasound scans of internal organs that are overlaid on the patient's body. Soldiers could see the positions of enemy snipers who had been spotted by unmanned reconnaissance planes.
Getting the right information at the right time and the right place is key in all these applications. Personal digital assistants such as the Palm and the Pocket PC can provide timely information using wireless networking and Global Positioning System (GPS) receivers that constantly track the handheld devices. But what makes augmented reality different is how the information is presented: not on a separate display but integrated with the user's perceptions. In augmented reality, the user's view of the world and the computer interface literally become one.
1.INTRODUCTION
INTRODUCTION
Video games have been entertaining us for nearly 30 years. Computer graphics have become much more sophisticated since then, and soon, game graphics will seem all too real. In the next decade, researchers plan to pull graphics out of your television screen or computer display and integrate them into real-world environments. This new technology, called augmented reality, will further blur the line between what's real and what's computer-generated by enhancing what we see, hear, feel and smell. The basic idea of augmented reality is to superimpose graphics, audio and other sense enhancements over a real-world environment in real-time. An augmented reality system generates a composite view for the user. It is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information.
Walk down the street, look at the world. This is reality. Now repeat, but wearing an odd-looking, bulky pair of glasses that place into your line of vision selective, relevant bits of data about the world or informative graphics and produce sound which will coincide with whatever you see.. This is augmented reality. An AR system, can superimpose computer generated text, graghics,3-D animation, sound, or any other digitised data on the real world. The augmented reality systems employ a see-through head-worn display that overlays graphics and sound on a person's naturally occurring sight and hearing. By tracking users and objects in space, these systems provide users with visual information that is tied to the physical environment. It not only superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective.
Augmented-reality displays will overlay
computer-generated graphics onto the real world.
On the spectrum between virtual reality, which creates immersible, computer-generated environments, and the real world, augmented reality is closer to the real world. Augmented reality adds graphics, sounds, hap tics and smell to the natural world as it exists. You can expect video games to drive the development of augmented reality, but this technology will have countless applications. Everyone from tourists to military troops will benefit from the ability to place computer-generated graphics in their field of vision.
Augmented reality will truly change the way we view the world. Picture yourself walking or driving down the street. With augmented-reality displays, which will eventually look much like a normal pair of glasses, informative graphics will appear in your field of view, and audio will coincide with whatever you see. These enhancements will be refreshed continually to reflect the movements of your head. In this article, we will take a look at this future technology, its components and how it will be used.
2. COMPARISON WITH VIRTUAL REALITY
COMPARISON WITH VIRTUAL REALITY
Augmented reality is very much close to virtual reality. Virtual reality is a technology that encompasses a broad spectrum of ideas. The term was defined as "a computer generated, interactive, three-dimensional environment in which a person is immersed." Virtual reality creates immersible, computer generated environments which replaces real world .Here the head mounted displays block out all the external world from the viewer and present a view that is under the complete control of the computer.
A very visible difference between the two types of systems is the immersiveness of the system. Virtual reality strives for a totally immersive environment. The visual, and in some systems aural and proprioceptive, senses are under control of the system. In contrast, an augmented reality system is augmenting the real world scene necessitating that the user maintains a sense of presence in that world. The virtual images are merged with the real view to create the augmented display. There must be a mechanism to combine the real and virtual that is not present in other virtual reality work. Developing the technology for merging the real and virtual image streams is an active research topic
Augmented reality is closer to the real world. Augmented reality adds graphics, sounds, hap tics and smell to the natural world, as it exists. Thus it augments the real world scene in such a way that the user can maintain a sense of presence in that world. That is, the user can interact with the real world , and at the same time can see, both the real and virtual world co-existing. For the same reason it has a large number of applications in the day to day life as compared to virtual reality.
The computer generated virtual objects must be accurately registered with the real world in all dimensions. Errors in this registration will prevent the user from seeing the real and virtual images as fused. The correct registration must also be maintained while the user moves about within the real environment. Discrepancies or changes in the apparent registration will range from distracting which makes working with the augmented view more difficult, to physically disturbing for the user making the system completely unusable. An immersive virtual reality system must maintain registration so that changes in the rendered scene match with the perceptions of the user. Any errors here are conflicts between the visual system and the kinesthetic or proprioceptive systems.
Milgram (Milgram and Kishino 1994; Milgram, Takemura et al. 1994) describes a taxonomy that identifies how augmented reality and virtual reality work are related. He defines the Reality-Virtuality continuum shown as Figure 2.
Figure 2 - Milgram's Reality-Virtuality Continuum
The real world and a totally virtual environment are at the two ends of this continuum with the middle region called Mixed Reality. Augmented reality lies near the real world end of the line with the predominate perception being the real world augmented by computer generated data. Augmented virtuality is a term created by Milgram to identify systems which are mostly synthetic with some real world imagery added such as texture mapping video onto virtual objects. This is a distinction that will fade as the technology improves and the virtual elements in the scene become less distinguishable from the real ones.
3. EARLY DEVELOPMENT
EARLY DEVELOPMENT
The first AR system was developed in the 1960s by computer graphics pioneer Ivan Sutherland and his students at Harvard University and the University of Utah. In the 1970s and 1980s a small number of researchers studied augmented reality at institutions such as the U.S. Air Force's Armstrong Laboratory, the NASA Ames Research Center and the University of North Carolina at Chapel Hill. It wasn't until the early 1990s that the term "augmented reality" was coined by scientists at Boeing who were developing an experimental AR system to help workers assemble wiring harnesses.
The past decade has seen a flowering of AR research as hardware costs have fallen enough to make the necessary lab equipment affordable. Scientists have gathered at yearly AR conferences since 1998. Eventually, possibly by the end of this decade, we will see the first mass-marketed augmented-reality system, which one researcher calls "the Walkman of the 21st century."
4.COMPONENTS OF AUGMENTED
REALITY SYSTEM
COMPONENTS OF AUGMENTED REALITY SYSTEM
What augmented reality attempts to do is not only superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective. Here are the three components needed to make an augmented-reality system work :
Head mounted displays
Tracking and orientation system
Mobile computing power
Photo courtesy Columbia University Computer Graphics and User Interfaces Lab
Early prototype of a mobile augmented-reality system
The goal of augmented-reality developers is to incorporate these three components into one unit, housed in a belt-worn device that wirelessly relays information to a display that resembles an ordinary pair of eyeglasses. Let's take a look at each of the components of this system.
4.1 HEAD MOUNTED DISPLAYS
Just as monitors allow us to see text and graphics generated by computers, head-mounted displays (HMDs) will enable us to view graphics and text created by augmented-reality systems. So far, there haven't been many HMDs created specifically with augmented reality in mind. These forms one of the main components of an augmented reality system. They are used to merge the virtual world and real world in front of the user in such a way that he feels he is looking at a single real scene . They resemble some type of skiing goggles.
There are two basic types of HMDS:
video see-through
optical see-through
VIDEO SEE- THROUGH
Video see-through displays block out the wearer's surrounding environment, using small video cameras attached to the outside of the goggles to capture images. On the inside of the display, the video image is played in real-time and the graphics are superimposed on the video. One problem with the use of video cameras is that there is more lag, meaning that there is a delay in image-adjustment when the viewer moves his or her head.
OPTICAL SEE-THROUGH
Optical see-through displays is not fully realized yet. It is supposed to consist of a ordinary-looking pair of glasses that will have a light source on the side to project images on to the retina.
COMPARISON
There are advantages and disadvantages to each of these types of displays. With both of the displays that use a video camera to view the real world there is a forced delay of up to one frame time to perform the video merging operation. At standard frame rates that will be potentially a 33.33 millisecond delay in the view seen by the user. Since everything the user sees is under system control compensation for this delay could be made by correctly timing the other paths in the system. Or, alternatively, if other paths are slower then the video of the real scene could be delayed. With an optical see-through display the view of the real world is instantaneous so it is not possible to compensate for system delays in other areas. On the other hand, with monitor based and video see-through displays a video camera is viewing the real scene. An advantage of this is that the image generated by the camera is available to the system to provide tracking information.
The optical see-through display does not have this additional information. The only position information available with that display is what can be provided by position sensors on the head mounted display itself.
4.2 TRACKING AND ORIENTATION SYSTEMS
The biggest challenge facing developers of augmented reality is the need to know where the user is located in reference to his or her surroundings.
In order to combine real and virtual worlds seamlessly so that the virtual objects align well with the real ones, an AR system needs to know two things precisely:
1) where the user is located, and
2) where he is looking.
A tracking system has to recognize these movements and project the graphics related to the real-world environment the user is seeing at any given moment. Currently, both video see-through and optical see-through displays typically have lag in the overlaid material due to the tracking technologies currently available. For augmented reality to reach its full potential, it must be usable both outdoors and indoors.
There are ways to increase tracking accuracy
Small area tracking and orientation systems
Tracking is easier in small spaces than in large spaces. Researchers at the University of North Carolina-Chapel Hill have developed a very precise system that works within 500 square feet. The HiBall Tracking System is an optoelectronic tracking system made of two parts:
six user-mounted, optical sensors
infrared-light-emitting diodes (LEDs) embedded in special ceiling panels
Photo courtesy Tracking Project at UNC-Chapel Hill
The Hiball Tracking System uses an optical sensing device and LED-embedded ceiling tiles to track movements over a short range.
The system uses the known location of the LEDs, the known geometry of the user-mounted optical sensors and a special algorithm to compute and report the user's position and orientation. The system resolves linear motion of less than .2 millimetres, and angular motions less than .03 degrees. It has an update rate of more than 1500 Hz, and latency is kept at about one millisecond
Large area tracking and orientation systems
For instance, the military uses multiple GPS (Global Positioning System) signals. There is also differential GPS, which involves using an area that has already been surveyed. Then the system would use a GPS receiver with an antenna that's location is known very precisely to track your location within that area. This will allow users to know exactly how inaccurate their GPS receivers are, and can adjust an augmented-reality system accordingly. Differential GPS allows for sub meter accuracy. A more accurate system being developed, known as real-time kinematic GPS, can achieve centimetre-level accuracy.
In case of out door application where the movement of user will be comparatively larger, his location with respect to his environments is tracked with the help of GPS RECEIVERS which works in coordination with the GPS satellites and the direction of vision of the user is calculated down to few degrees by INERTIAL/MAGNETIC TRACKER.
Tracking using GPS
The Global Positioning System (GPS) is actually a constellation of 27 Earth-orbiting satellites (24 in operation and three extras in case one fails). The U.S. military developed and implemented this satellite network as a military navigation system, but soon opened it up to everybody else.
Each of these 3,000- to 4,000-pound solar-powered satellites circles the globe at about 12,000 miles (19,300 km), making two complete rotations every day. The orbits are arranged so that at any time, anywhere on Earth, there are at least four satellites "visible" in the sky.
A GPS receiver's job is to locate four or more of these satellites, figure out the distance to each, and use this information to deduce its own location. This operation is based on a simple mathematical principle called trilateration
Positioning by 3-D trilateration
If we know we are 10 miles from satellite A in the sky, we could be anywhere on the surface of a huge, imaginary sphere with a 10-mile radius. If we also know we are 15 miles from satellite B, we can overlap the first sphere with another, larger sphere. The spheres intersect in a perfect circle. If we know the distance to a third satellite, we get a third sphere, which intersects with this circle at two points.
The Earth itself can act as a fourth sphere -- only one of the two possible points will actually be on the surface of the planet, so you can eliminate the one in space. Receivers generally look to four or more satellites, however, to improve accuracy and provide precise altitude information.
Measuring Distance
A GPS receiver calculates the distance to GPS satellites by timing a signal's journey from satellite to receiver. As it turns out, this is a fairly elaborate process. At a particular time (let's say midnight), the satellite begins transmitting a long, digital pattern called a pseudo-random code. The receiver begins running the same digital pattern also exactly at midnight. When the satellite's signal reaches the receiver, its transmission of the pattern will lag a bit behind the receiver's playing of the pattern. The length of the delay is equal to the signal's travel time. The receiver multiplies this time by the speed of light to determine how far the signal travelled. Assuming the signal travelled in a straight line, this is the distance from receiver to satellite. In order to make this measurement, the receiver and satellite both need clocks that can be synchronized down to the nanosecond. Every satellite contains an expensive atomic clock, but the receiver itself uses an ordinary quartz clock, which it constantly resets. In a nutshell, the receiver looks at incoming signals from four or more satellites and gauges its own inaccuracy.
When we measure the distance to four located satellites, we can draw four spheres that all intersect at one point. Three spheres will intersect even if our numbers are way off, but four spheres will not intersect at one point if we have measured incorrectly. Since the receiver makes all its distance measurements using its own built-in clock, the distances will all be proportionally incorrect. The receiver can easily calculate the necessary adjustment that will cause the four spheres to intersect at one point. Based on this, it resets its clock to be in sync with the satellite's atomic clock. The receiver does this constantly whenever it's on, which means it is nearly as accurate as the expensive atomic clocks in the satellites. In order for the distance information to be of any use, the receiver also has to know where the satellites actually are. This isn't particularly difficult because the satellites travel in very high and predictable orbits. The GPS receiver simply stores an almanac that tells it where every satellite should be at any given time. Things like the pull of the moon and the sun do change the satellites' orbits very slightly, but the Department of Defence constantly monitors their exact positions and transmits any adjustments to all GPS receivers as part of the satellites' signals
This system works pretty well, but inaccuracies do pop up. For one thing, this method assumes the radio signals will make their way through the atmosphere at a consistent speed (the speed of light). In fact, the Earth's atmosphere slows the electromagnetic energy down somewhat, particularly as it goes through the ionosphere and troposphere. The delay varies depending on where you are on Earth, which means it's difficult to accurately factor this into the distance calculations. Problems can also occur when radio signals bounce off large objects, such as skyscrapers, giving a receiver the impression that a satellite is farther away than it actually is. On top of all that, satellites sometimes just send out bad almanac data, misreporting their own position.
Differential GPS (DGPS) helps correct these errors. The basic idea is to gauge GPS inaccuracy at a stationary receiver station with a known location. Since the DGPS hardware at the station already knows its own position, it can easily calculate its receiver's inaccuracy. The station then broadcasts a radio signal to all DGPS-equipped receivers in the area, providing signal correction information for that area. In general, access to this correction information makes DGPS receivers much more accurate than ordinary receivers.
Thus the most essential function of a GPS receiver is to pick up the transmissions of at least four satellites and combine the information in those transmissions with information in an electronic almanac, all in order to figure out the receiver's position on Earth. Once the receiver makes this calculation, it can tell us the latitude, longitude and altitude (or some similar measurement) of its current position. To make the navigation more user-friendly, most receivers plug this raw data into map files stored in memory. We can use maps stored in the receiver's memory, connect the receiver to a computer that can hold more detailed maps in its memory. A standard GPS receiver will not only place us on a map at any particular location, but will also trace our path across a map as you move. If we leave our receiver on, it can stay in constant communication with GPS satellites to see how our location is changing.
ORIENTATION
For orientation, an inertial/magnetic tracker rides on a headband above the AR glasses. This device is a combination of miniature gyroscopes and accelerometers that detect head movements along with an electronic compass that establishes the direction of the viewer's gaze in relation to Earth's magnetic field.
4.3 MOBILE COMPUTING POWER
Mobile computing can be accomplished with the help of a wearable computer. A wearable computer is a battery-powered computer system worn on the user's body (on a belt, backpack or vest). It is designed for mobile and predominantly hands-free operations, often incorporating head-mounted displays and speech input.
The wearable computer is more than just a wristwatch or regular eyeglasses: it has the full functionality of a computer system but in addition to being a fully featured computer, it is also inextricably intertwined with the wearer. This is what sets the wearable computer apart from other wearable devices such as wristwatches, regular eyeglasses, wearable radios, etc
Three important features of wearable computers are
1.Constancy
The computer runs continuously, and is always ready'' to interact with the user. Unlike a hand-held device, laptop computer, or PDA, it does not need to be opened up and turned on prior to use. The signal flow from human to computer, and computer to human runs continuously to provide a constant user--interface.
2. Augmentation
Traditional computing paradigms are based on the notion that computing is the primary task. Wearable computing, however, is based on the notion that computing is NOT the primary task. The assumption of wearable computing is that the user will be doing something else at the same time as doing the computing. Thus the computer should serve to augment the intellect, or augment the senses. The signal flow between human and computer is depicted in the figure below
3.Mediation:
Unlike hand held devices, laptop computers, and PDAs, the wearable computer can encapsulate us. It doesn't necessarily need to completely enclose us, but the concept allows for a greater degree of encapsulation than traditional portable computers
5. APPLICATIONS
APPLICATIONS OF AR SYSTEMS.
1. Maintenance and construction
This is one of the first uses for augmented reality. Markers can be attached to a particular object that a person is working on, and the augmented-reality system can draw graphics on top of it. This is a more simple form of augmented reality, since the system only has to know where the user is in reference to the object that he or she is looking at. It's not necessary to track the person's exact physical location.
Eg: Usage if AR system in space frame construction
Space frames are typically made from a large number of components of similar size and shape (typically cylindrical struts and spherical nodes). Although the exterior dimensions of all the members may be identical, the forces they carry, and therefore their inner diameters, vary with their position in the structure. Consequently it is relatively easy to assemble pieces in the wrong position-which if undetected could lead to structural failure. Our augmented reality construction system is designed to guide workers through the assembly of a spaceframe structure, to ensure that each member is properly placed and fastened.
The spaceframe is assembled one component (strut or node) at a time. For each step of construction, the augmented reality system :
Directs the worker to a pile of parts and tells her which part to pick up. This is currently done by displaying textual instructions and playing a sound file containing verbal instructions.
Confirms that she has the correct piece. This is done by having her scan a barcode on the component.
Directs her to install the component. A 3D virtual image of the component indicates where to install the component and verbal instructions played from a sound file explain how to install it.
Verifies that the component is installed by asking her to scan the component with the tracked barcode scanner. This checks both the identity and position of the part.
Similarly, a repairperson viewing a broken piece of equipment could see instructions highlighting the parts that need to be inspected.
2. Military
The military has been devising uses for augmented reality for decades. The idea here is that an augmented-reality system could provide troops with vital information about their surroundings, such as showing where entrances are on the opposite end of a building, somewhat like X-ray vision. Augmented reality displays could also highlight troop movements, and give soldiers the ability to move to where the enemy can't see them.
In the AR future, a small team of soldiers airlifted into a remote combat area will encounter terrain that has been mapped in advance. Soldiers won't see just rocks, trees, and buildings, they'll see annotated warnings: "buried mines" or "enemy stores arms in this building." As surveillance reports flow into the command centre, new graphics will be broadcast to the AR gear.
3. Medical
Most of the medical applications deal with IMAGE GUIDED SURGERY. Pre-operative imaging studies, such as CT or MRI scans, of the patient provide the surgeon with the necessary view of the internal anatomy. From these images the surgery is planned. Visualization of the path through the anatomy to the affected area where, for example, a tumour must be removed is done by first creating a 3D model from the multiple views and slices in the preoperative study. This is most often done mentally though some systems will create 3D volume visualizations from the image study. Augmented reality can be applied so that the surgical team can see the CT or
MRI data correctly registered on the patient in the operating theatre while the procedure is progressing. Being able to accurately register the images at this point will enhance the performance of the surgical team and eliminate the need for the painful and cumbersome stereo tactic frames. Augmented reality systems can also be helpful in surgery to sense and MARK the vital parts so that the surgeon can be very careful at these regions.
Another application for augmented reality in the medical domain is in ULTRASOUND IMAGING. Using an optical see-through display the ultrasound technician can view a volumetric rendered image of the fetus overlaid on the abdomen of the pregnant woman. The image appears as if it was inside of the abdomen and is correctly rendered as the user moves.
Similarly, Patients admitted for routine breast biopsies and possible lumpectomies are randomly assigned to the AR test. Instead of the radiologist's usual practice of looking up at a sonogram screen and then back again at the patient, ultrasound images are seen through the physician's headgear as projected directly onto the patient's body. This provides a sort of virtual X-ray vision throughout the procedure. Breast lumps and other possibly cancerous anomalies show up as ghostly white outlines against an uneven grey background. And the position- and orientation-sensing technology in the head-mounted display lets the radiologist "see" where to
guide a biopsy needle with unprecedented precision. The hoped-for outcome of this AR application includes fewer complications and shorter recovery times for existing procedures, as well as the development of new surgical techniques. For brief procedures such as biopsies and laparoscopic (minimally invasive) surgery, a head-mounted AR display offers an ideal solution for combining actual and computer worlds.
4. Media and entertainment
A simple form of augmented reality has been in use in the entertainment and news business for quite some time. Whenever you are watching the evening weather report the weather reporter is shown standing in front of changing weather maps. In the studio the reporter is actually standing in front of a blue or green screen. This real image is augmented with computer generated maps using a technique called chroma-keying. It is also possible to create a virtual studio environment so that the actors can appear to be positioned in a studio with computer generated decorating. Augmented reality system allows broadcasters to insert advertisements into specific areas of the broadcast image . For example, while broadcasting a baseball game this system would be able to place an advertisement in the image so that it appears on the outfield wall of the stadium.
5. Gaming
The game could be projected onto the real world around you, and you could, literally, be in it as one of the characters. How cool would it be to take video games outside? The game could be projected onto the real world around you, and you could, literally, be in it as one of the characters. One Australian researcher has created a prototype game that combines Quake, a popular video game, with augmented reality. He put a model of a university campus into the game's software. Now, when he uses this system, the game surrounds him as he walks across campus.
6. IMPACT ON FUTURE LIFE
In Future-Everyday Life
There is no shortage of wish list applications for personal AR, whether handheld or head-mounted. Consider the home garage of the future, for instance. While fixing a car, there will no longer be the need to pull our head in and out from under the open hood to consult a bulky, greasy manual. With AR, we will simply slip on a tiny visor and guided repair instructions will appear next to each under-the-hood part that we gaze at: "Now that you've disconnected the radiator hose, move it to one side and unscrew the carburettor cap." Or we can retrieve the same data and navigate through parts information and replacement sales sites on the Web by merely holding a PDA-size position-sensing screen in front of any section of the engine.
And when AR headgear does shrink down to the size of common glasses, it could be a must for up-and-coming managers, to avoid career or social gaffes at business meetings and cocktail parties. Everyone will be packing extra data in their spectacles. Each time we look at someone across a conference table or a crowded room, information about who they are and what their background is could appear before your eyes. Learning how not to make it obvious that we are "scanning" a person's data will be a new business skill, like trying to look natural in front of a teleprompter.
7. CURRENT LIMITATIONS
Current Limitations
1. Accurate tracking and orientation is a problem in outdoors today because the tracking system currently used is sensitive to sudden variations in magnetic fields, the alignment of graphics and a street scene can be easily thrown off by even a stray remnant of 19th century technology like old iron trolley car tracks beneath asphalt. Moreover, a tracking system which can work accurately for a long time has not been developed yet.
2. The size of AR system is yet another problem. Augmented-reality displays are still pretty bulky; the weight and size of a wearable computer also needs to be brought down. Researchers believe that they will succeed in this within 2 years.
3. For a wearable augmented reality system, there is still not enough computing power to create stereo 3-D graphics. So researchers are using whatever they can get out of laptops and personal computers, for now. Laptops are just now starting to be equipped with graphics processing units (GPUs). Toshiba just added an NVidia GPU to their notebooks that is able to process more than 17-million triangles per second and 286-million pixels per second, which can enable CPU-intensive programs, such as 3-D games.
8. CONCLUSION
CONCLUSION
It's only a matter of time before augmented reality becomes part of our daily lives. With further developments, in future, the AR SYTEMS are going to become very compact, light weight and low cost units, so that it becomes very common in everyday life. Judging from the cell phones and palm-sized organizers that are already pervading our pockets, we can rightly predict that: "we'll feel left out if we don't have a personal augmented reality system to enhance our experience of the world."
9.BIBLIOGRAPHY
BIBLIOGRAPHY
1. Virtual architecture by zampi,guiiano
2. Optical and Optoelectronic instrumentation by Shanthi Prince,Anapurna
3. howstuffworks.com
4. imageguidedsurgery.com
5. gps.com
CONTENTS
ABSTRACT 1
1. INTRODUCTION 2
2. COMPARISON WITH VIRTUAL REALITY 5
3. EARLY DEVELOPMENT 8
4. COMPONENTS OF AUGMENTED REALITY SYSTEM 10
4.1 HEAD MOUNTED DISPLAYS 12
4.1.1 VIDEO SEE- THROUGH 13
4.1.2 OPTICAL SEE TROUGH 13
4.2 TRACKING AND ORIENTATION SYSTEMS 15
4.3 MOBILE COMPUTING POWER 20
5. APPLICATIONS 22
6. IMPACT ON FUTURE LIFE 28
7. CURRENT LIMITATIONS 30
8. CONCLUSION 32
9. BIBLIOGRAPHY 34
augmented reality
AUGMENTED REALITY - rathincr - 10-04-2017
ABSTRACT
Augmented reality (AR) refers to computer displays that add virtual information to a user's sensory perceptions. Most AR research focuses on "see-through" devices, usually worn on the head that overlay graphics and text on the user's view of his or her surroundings. AR systems track the position and orientation of the user's head so that the overlaid material can be aligned with the user's view of the world.
Consider what AR could make routinely possible. A repairperson viewing a broken piece of equipment could see instructions highlighting the parts that need to be inspected. A surgeon could get the equivalent of x-ray vision by observing live ultrasound scans of internal organs that are overlaid on the patient's body. Soldiers could see the positions of enemy snipers who had been spotted by unmanned reconnaissance planes.
Getting the right information at the right time and the right place is key in all these applications. Personal digital assistants such as the Palm and the Pocket PC can provide timely information using wireless networking and Global Positioning System (GPS) receivers that constantly track the handheld devices. But what makes augmented reality different is how the information is presented: not on a separate display but integrated with the user's perceptions. In augmented reality, the user's view of the world and the computer interface literally become one.
1.INTRODUCTION
INTRODUCTION
Video games have been entertaining us for nearly 30 years. Computer graphics have become much more sophisticated since then, and soon, game graphics will seem all too real. In the next decade, researchers plan to pull graphics out of your television screen or computer display and integrate them into real-world environments. This new technology, called augmented reality, will further blur the line between what's real and what's computer-generated by enhancing what we see, hear, feel and smell. The basic idea of augmented reality is to superimpose graphics, audio and other sense enhancements over a real-world environment in real-time. An augmented reality system generates a composite view for the user. It is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information.
Walk down the street, look at the world. This is reality. Now repeat, but wearing an odd-looking, bulky pair of glasses that place into your line of vision selective, relevant bits of data about the world or informative graphics and produce sound which will coincide with whatever you see.. This is augmented reality. An AR system, can superimpose computer generated text, graghics,3-D animation, sound, or any other digitised data on the real world. The augmented reality systems employ a see-through head-worn display that overlays graphics and sound on a person's naturally occurring sight and hearing. By tracking users and objects in space, these systems provide users with visual information that is tied to the physical environment. It not only superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective.
Augmented-reality displays will overlay
computer-generated graphics onto the real world.
On the spectrum between virtual reality, which creates immersible, computer-generated environments, and the real world, augmented reality is closer to the real world. Augmented reality adds graphics, sounds, hap tics and smell to the natural world as it exists. You can expect video games to drive the development of augmented reality, but this technology will have countless applications. Everyone from tourists to military troops will benefit from the ability to place computer-generated graphics in their field of vision.
Augmented reality will truly change the way we view the world. Picture yourself walking or driving down the street. With augmented-reality displays, which will eventually look much like a normal pair of glasses, informative graphics will appear in your field of view, and audio will coincide with whatever you see. These enhancements will be refreshed continually to reflect the movements of your head. In this article, we will take a look at this future technology, its components and how it will be used.
2. COMPARISON WITH VIRTUAL REALITY
COMPARISON WITH VIRTUAL REALITY
Augmented reality is very much close to virtual reality. Virtual reality is a technology that encompasses a broad spectrum of ideas. The term was defined as "a computer generated, interactive, three-dimensional environment in which a person is immersed." Virtual reality creates immersible, computer generated environments which replaces real world .Here the head mounted displays block out all the external world from the viewer and present a view that is under the complete control of the computer.
A very visible difference between the two types of systems is the immersiveness of the system. Virtual reality strives for a totally immersive environment. The visual, and in some systems aural and proprioceptive, senses are under control of the system. In contrast, an augmented reality system is augmenting the real world scene necessitating that the user maintains a sense of presence in that world. The virtual images are merged with the real view to create the augmented display. There must be a mechanism to combine the real and virtual that is not present in other virtual reality work. Developing the technology for merging the real and virtual image streams is an active research topic
Augmented reality is closer to the real world. Augmented reality adds graphics, sounds, hap tics and smell to the natural world, as it exists. Thus it augments the real world scene in such a way that the user can maintain a sense of presence in that world. That is, the user can interact with the real world , and at the same time can see, both the real and virtual world co-existing. For the same reason it has a large number of applications in the day to day life as compared to virtual reality.
The computer generated virtual objects must be accurately registered with the real world in all dimensions. Errors in this registration will prevent the user from seeing the real and virtual images as fused. The correct registration must also be maintained while the user moves about within the real environment. Discrepancies or changes in the apparent registration will range from distracting which makes working with the augmented view more difficult, to physically disturbing for the user making the system completely unusable. An immersive virtual reality system must maintain registration so that changes in the rendered scene match with the perceptions of the user. Any errors here are conflicts between the visual system and the kinesthetic or proprioceptive systems.
Milgram (Milgram and Kishino 1994; Milgram, Takemura et al. 1994) describes a taxonomy that identifies how augmented reality and virtual reality work are related. He defines the Reality-Virtuality continuum shown as Figure 2.
Figure 2 - Milgram's Reality-Virtuality Continuum
The real world and a totally virtual environment are at the two ends of this continuum with the middle region called Mixed Reality. Augmented reality lies near the real world end of the line with the predominate perception being the real world augmented by computer generated data. Augmented virtuality is a term created by Milgram to identify systems which are mostly synthetic with some real world imagery added such as texture mapping video onto virtual objects. This is a distinction that will fade as the technology improves and the virtual elements in the scene become less distinguishable from the real ones.
3. EARLY DEVELOPMENT
EARLY DEVELOPMENT
The first AR system was developed in the 1960s by computer graphics pioneer Ivan Sutherland and his students at Harvard University and the University of Utah. In the 1970s and 1980s a small number of researchers studied augmented reality at institutions such as the U.S. Air Force's Armstrong Laboratory, the NASA Ames Research Center and the University of North Carolina at Chapel Hill. It wasn't until the early 1990s that the term "augmented reality" was coined by scientists at Boeing who were developing an experimental AR system to help workers assemble wiring harnesses.
The past decade has seen a flowering of AR research as hardware costs have fallen enough to make the necessary lab equipment affordable. Scientists have gathered at yearly AR conferences since 1998. Eventually, possibly by the end of this decade, we will see the first mass-marketed augmented-reality system, which one researcher calls "the Walkman of the 21st century."
4.COMPONENTS OF AUGMENTED
REALITY SYSTEM
COMPONENTS OF AUGMENTED REALITY SYSTEM
What augmented reality attempts to do is not only superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective. Here are the three components needed to make an augmented-reality system work :
Head mounted displays
Tracking and orientation system
Mobile computing power
Photo courtesy Columbia University Computer Graphics and User Interfaces Lab
Early prototype of a mobile augmented-reality system
The goal of augmented-reality developers is to incorporate these three components into one unit, housed in a belt-worn device that wirelessly relays information to a display that resembles an ordinary pair of eyeglasses. Let's take a look at each of the components of this system.
4.1 HEAD MOUNTED DISPLAYS
Just as monitors allow us to see text and graphics generated by computers, head-mounted displays (HMDs) will enable us to view graphics and text created by augmented-reality systems. So far, there haven't been many HMDs created specifically with augmented reality in mind. These forms one of the main components of an augmented reality system. They are used to merge the virtual world and real world in front of the user in such a way that he feels he is looking at a single real scene . They resemble some type of skiing goggles.
There are two basic types of HMDS:
video see-through
optical see-through
VIDEO SEE- THROUGH
Video see-through displays block out the wearer's surrounding environment, using small video cameras attached to the outside of the goggles to capture images. On the inside of the display, the video image is played in real-time and the graphics are superimposed on the video. One problem with the use of video cameras is that there is more lag, meaning that there is a delay in image-adjustment when the viewer moves his or her head.
OPTICAL SEE-THROUGH
Optical see-through displays is not fully realized yet. It is supposed to consist of a ordinary-looking pair of glasses that will have a light source on the side to project images on to the retina.
COMPARISON
There are advantages and disadvantages to each of these types of displays. With both of the displays that use a video camera to view the real world there is a forced delay of up to one frame time to perform the video merging operation. At standard frame rates that will be potentially a 33.33 millisecond delay in the view seen by the user. Since everything the user sees is under system control compensation for this delay could be made by correctly timing the other paths in the system. Or, alternatively, if other paths are slower then the video of the real scene could be delayed. With an optical see-through display the view of the real world is instantaneous so it is not possible to compensate for system delays in other areas. On the other hand, with monitor based and video see-through displays a video camera is viewing the real scene. An advantage of this is that the image generated by the camera is available to the system to provide tracking information.
The optical see-through display does not have this additional information. The only position information available with that display is what can be provided by position sensors on the head mounted display itself.
4.2 TRACKING AND ORIENTATION SYSTEMS
The biggest challenge facing developers of augmented reality is the need to know where the user is located in reference to his or her surroundings.
In order to combine real and virtual worlds seamlessly so that the virtual objects align well with the real ones, an AR system needs to know two things precisely:
1) where the user is located, and
2) where he is looking.
A tracking system has to recognize these movements and project the graphics related to the real-world environment the user is seeing at any given moment. Currently, both video see-through and optical see-through displays typically have lag in the overlaid material due to the tracking technologies currently available. For augmented reality to reach its full potential, it must be usable both outdoors and indoors.
There are ways to increase tracking accuracy
Small area tracking and orientation systems
Tracking is easier in small spaces than in large spaces. Researchers at the University of North Carolina-Chapel Hill have developed a very precise system that works within 500 square feet. The HiBall Tracking System is an optoelectronic tracking system made of two parts:
six user-mounted, optical sensors
infrared-light-emitting diodes (LEDs) embedded in special ceiling panels
Photo courtesy Tracking Project at UNC-Chapel Hill
The Hiball Tracking System uses an optical sensing device and LED-embedded ceiling tiles to track movements over a short range.
The system uses the known location of the LEDs, the known geometry of the user-mounted optical sensors and a special algorithm to compute and report the user's position and orientation. The system resolves linear motion of less than .2 millimetres, and angular motions less than .03 degrees. It has an update rate of more than 1500 Hz, and latency is kept at about one millisecond
Large area tracking and orientation systems
For instance, the military uses multiple GPS (Global Positioning System) signals. There is also differential GPS, which involves using an area that has already been surveyed. Then the system would use a GPS receiver with an antenna that's location is known very precisely to track your location within that area. This will allow users to know exactly how inaccurate their GPS receivers are, and can adjust an augmented-reality system accordingly. Differential GPS allows for sub meter accuracy. A more accurate system being developed, known as real-time kinematic GPS, can achieve centimetre-level accuracy.
In case of out door application where the movement of user will be comparatively larger, his location with respect to his environments is tracked with the help of GPS RECEIVERS which works in coordination with the GPS satellites and the direction of vision of the user is calculated down to few degrees by INERTIAL/MAGNETIC TRACKER.
Tracking using GPS
The Global Positioning System (GPS) is actually a constellation of 27 Earth-orbiting satellites (24 in operation and three extras in case one fails). The U.S. military developed and implemented this satellite network as a military navigation system, but soon opened it up to everybody else.
Each of these 3,000- to 4,000-pound solar-powered satellites circles the globe at about 12,000 miles (19,300 km), making two complete rotations every day. The orbits are arranged so that at any time, anywhere on Earth, there are at least four satellites "visible" in the sky.
A GPS receiver's job is to locate four or more of these satellites, figure out the distance to each, and use this information to deduce its own location. This operation is based on a simple mathematical principle called trilateration
Positioning by 3-D trilateration
If we know we are 10 miles from satellite A in the sky, we could be anywhere on the surface of a huge, imaginary sphere with a 10-mile radius. If we also know we are 15 miles from satellite B, we can overlap the first sphere with another, larger sphere. The spheres intersect in a perfect circle. If we know the distance to a third satellite, we get a third sphere, which intersects with this circle at two points.
The Earth itself can act as a fourth sphere -- only one of the two possible points will actually be on the surface of the planet, so you can eliminate the one in space. Receivers generally look to four or more satellites, however, to improve accuracy and provide precise altitude information.
Measuring Distance
A GPS receiver calculates the distance to GPS satellites by timing a signal's journey from satellite to receiver. As it turns out, this is a fairly elaborate process. At a particular time (let's say midnight), the satellite begins transmitting a long, digital pattern called a pseudo-random code. The receiver begins running the same digital pattern also exactly at midnight. When the satellite's signal reaches the receiver, its transmission of the pattern will lag a bit behind the receiver's playing of the pattern. The length of the delay is equal to the signal's travel time. The receiver multiplies this time by the speed of light to determine how far the signal travelled. Assuming the signal travelled in a straight line, this is the distance from receiver to satellite. In order to make this measurement, the receiver and satellite both need clocks that can be synchronized down to the nanosecond. Every satellite contains an expensive atomic clock, but the receiver itself uses an ordinary quartz clock, which it constantly resets. In a nutshell, the receiver looks at incoming signals from four or more satellites and gauges its own inaccuracy.
When we measure the distance to four located satellites, we can draw four spheres that all intersect at one point. Three spheres will intersect even if our numbers are way off, but four spheres will not intersect at one point if we have measured incorrectly. Since the receiver makes all its distance measurements using its own built-in clock, the distances will all be proportionally incorrect. The receiver can easily calculate the necessary adjustment that will cause the four spheres to intersect at one point. Based on this, it resets its clock to be in sync with the satellite's atomic clock. The receiver does this constantly whenever it's on, which means it is nearly as accurate as the expensive atomic clocks in the satellites. In order for the distance information to be of any use, the receiver also has to know where the satellites actually are. This isn't particularly difficult because the satellites travel in very high and predictable orbits. The GPS receiver simply stores an almanac that tells it where every satellite should be at any given time. Things like the pull of the moon and the sun do change the satellites' orbits very slightly, but the Department of Defence constantly monitors their exact positions and transmits any adjustments to all GPS receivers as part of the satellites' signals
This system works pretty well, but inaccuracies do pop up. For one thing, this method assumes the radio signals will make their way through the atmosphere at a consistent speed (the speed of light). In fact, the Earth's atmosphere slows the electromagnetic energy down somewhat, particularly as it goes through the ionosphere and troposphere. The delay varies depending on where you are on Earth, which means it's difficult to accurately factor this into the distance calculations. Problems can also occur when radio signals bounce off large objects, such as skyscrapers, giving a receiver the impression that a satellite is farther away than it actually is. On top of all that, satellites sometimes just send out bad almanac data, misreporting their own position.
Differential GPS (DGPS) helps correct these errors. The basic idea is to gauge GPS inaccuracy at a stationary receiver station with a known location. Since the DGPS hardware at the station already knows its own position, it can easily calculate its receiver's inaccuracy. The station then broadcasts a radio signal to all DGPS-equipped receivers in the area, providing signal correction information for that area. In general, access to this correction information makes DGPS receivers much more accurate than ordinary receivers.
Thus the most essential function of a GPS receiver is to pick up the transmissions of at least four satellites and combine the information in those transmissions with information in an electronic almanac, all in order to figure out the receiver's position on Earth. Once the receiver makes this calculation, it can tell us the latitude, longitude and altitude (or some similar measurement) of its current position. To make the navigation more user-friendly, most receivers plug this raw data into map files stored in memory. We can use maps stored in the receiver's memory, connect the receiver to a computer that can hold more detailed maps in its memory. A standard GPS receiver will not only place us on a map at any particular location, but will also trace our path across a map as you move. If we leave our receiver on, it can stay in constant communication with GPS satellites to see how our location is changing.
ORIENTATION
For orientation, an inertial/magnetic tracker rides on a headband above the AR glasses. This device is a combination of miniature gyroscopes and accelerometers that detect head movements along with an electronic compass that establishes the direction of the viewer's gaze in relation to Earth's magnetic field.
4.3 MOBILE COMPUTING POWER
Mobile computing can be accomplished with the help of a wearable computer. A wearable computer is a battery-powered computer system worn on the user's body (on a belt, backpack or vest). It is designed for mobile and predominantly hands-free operations, often incorporating head-mounted displays and speech input.
The wearable computer is more than just a wristwatch or regular eyeglasses: it has the full functionality of a computer system but in addition to being a fully featured computer, it is also inextricably intertwined with the wearer. This is what sets the wearable computer apart from other wearable devices such as wristwatches, regular eyeglasses, wearable radios, etc
Three important features of wearable computers are
1.Constancy
The computer runs continuously, and is always ready'' to interact with the user. Unlike a hand-held device, laptop computer, or PDA, it does not need to be opened up and turned on prior to use. The signal flow from human to computer, and computer to human runs continuously to provide a constant user--interface.
2. Augmentation
Traditional computing paradigms are based on the notion that computing is the primary task. Wearable computing, however, is based on the notion that computing is NOT the primary task. The assumption of wearable computing is that the user will be doing something else at the same time as doing the computing. Thus the computer should serve to augment the intellect, or augment the senses[attachment=2406][attachment=2407][attachment=2406]
AUGMENTED REALITY - karanpatil1989 - 10-04-2017
Augmented Reality (AR)
Renjith.R & Bijin.V.S
(Department of computer application(MCA) ,Mohandas college of Engineering and technology
Anad, Trivandrum)
[attachment=10141]
Abstract
Technology has advanced to the point here realism in virtual reality is very achievable. However, in
our obsession to reproduce the world and human experience in virtual space, we overlook the most
important aspects of what makes us who we are our reality. On the spectrum between virtual reality,
which creates immersible, computer-generated environments, and the real world, augmented reality
is closer to the real world. Augmented reality adds graphics, sounds, haptics and smell to the natural
world as it exists .Augmented reality will truly change the way we view the world. Picture yourself
walking or driving down the street. With augmented-reality displays, which will eventually look much
like a normal pair of glasses, informative graphics will appear in your field of view and audio will
coincide with whatever you see. These enhancements will be refreshed continually to reflect the
movements of your head. In this article, we will take a look at this future technology, its components
and how it will be used. Augmented reality (AR) refers to computer displays that add virtual
information to a user's sensory perceptions. Most AR research focuses on see-through devices,
usually worn on the head that overlay graphics and text on the user's view of his or her surroundings.
In general it superimposes graphics over a real world environment in real time.Augmented reality is
far more advanced than any technology you've seen in television broadcasts, although early versions
of augmented reality are starting to appear in televised races and football games. These systems
display graphics for only one point of view. Next-generation augmented-reality systems will display
graphics for each viewer's perspective.
1. INTRODUCTION
1.1. DEFINITION
Augmented reality (AR) is a field of
computer research which deals with the
combination of real world and computer
generated data. Augmented reality (AR) refers to
computer displays that add virtual information to
a user's sensory perceptions. It is a method for
visual improvement or enrichment of the
surrounding environment by overlaying spatially
aligned computer-generated information onto a
human's view (eyes)
Augmented Reality (AR) was introduced as
the opposite of virtual reality: instead of
immersing the user into a synthesized, purely
informational environment, the goal of AR is to
augment the real world with information
handling capabilities.
AR research focuses on see-through
devices, usually worn on the head that overlay
graphics and text on the user's view of his or her
surroundings. In general it superimposes
graphics over a real world environment in real
time.
An AR system adds virtual computer-
generated objects, audio and other sense
enhancements to a real-world environment in
real-time. These enhancements are added in a
way that the viewer cannot tell the difference
between the real and augmented world.
1.2 PROPERTIES
AR system to have the following properties:
1. Combines real and virtual objects in a real
environment;
2. Runs interactively, and in real time; and
3. Registers (aligns) real and virtual objects with
each other.
Definition of AR to particular display
technologies, such as a head mounted display
(HMD). Nor do we limit it to our sense of sight.
AR can potentially apply to all senses, including
hearing, touch, and smell.
2. AUGMENTED REALITY Vs VIRTUAL
REALITY
The term Virtual Reality was defined as "a
computer generated, interactive, three-dimensional
environment in which a person is immersed." There
are three key points in this definition. First, this
virtual environment is a computer generated three-
dimensional scene which requires high
performance computer graphics to provide an
adequate level of realism. The second point is that
the virtual world is interactive. A user requires real-
time response from the system to be able to interact
with it in an effective manner. The last point is that
the user is immersed in this virtual environment
One of the identifying marks of a virtual reality
system is the head mounted display worn by users.
These displays block out all the external world and
present to the wearer a view that is under the
complete control of the computer. The user is
completely immersed in an artificial world and
becomes divorced from the real environment.
A very visible difference between these two
types of systems is the immersiveness of the
system. Virtual reality strives for a totally
immersive environment. The visual, and in some
systems aural and proprioceptive, senses are under
control of the system.
In contrast, an augmented reality system
is augmenting the real world scene necessitating
that the user maintains a sense of presence in that
world. The virtual images are merged with the real
view to create the augmented display. There must
be a mechanism to combine the real and virtual that
is not present in other virtual reality work.
Developing the technology for merging the real and
virtual image streams is an active research topic .
3. Different AR Techniques
There are two basic techniques for
combining real and virtual objects; optical and
video techniques. While optical technique uses
an optical combiner, video technique uses a
computer for combining the video of the real
world (from video cameras) with virtual
images (computer generated). AR systems use
either Head Mounted Display (HMD), which
can be closed-view or see-through HMDs, or
use monitor-based configuration. While
closed-view HMDs do not allow real world
direct view, see-through HMDs allow it, with
virtual objects added via optical or video
techniques
4. What Makes AR Work?
The main components that make an AR system
works are,
1. Display
This corresponds to head mounted
devices where images are formed. Many objects
that do not exist in the real world can be put into
this environment and users can view and exam on
physical properties etc. are just parameters in
simulation.
2. Tracking
Getting the right information at the
right time and the right place is the key in all these
applications. Personal digital assistants such as the
Palm and the Pocket PC can provide timely
information using wireless networking and Global
Positioning System (GPS) receivers that constantly
track the handheld devices
3. Environment Sensing
It is the process of viewing or sensing
the real world scenes or even physical environment
which can be done either by using an optical
combiner, a video combiner or simply retinal view.
4. Visualization and Rendering
Some emerging trends in the recent
development of human-computer interaction (HCI)
can be observed. The trends are augmented reality,
computer supported cooperative work, ubiquitous
computing, and heterogeneous user interface. AR is
a method for visual improvement or enrichment of
the surrounding environment by overlaying
spatially aligned computer-generated information
onto a human's view (eyes).
This is how AR works.
Pick A Real World Scene
Real world. User's view through
the see-through head-worn display of the real
world, showing two struts and a node without
any overlaid graphics.
Add your Virtual Objects in it
User's view of the virtual world
intended to overlay the view of the real
world.
Delete Real World Objects
Not Virtual Reality since Environment
Real
these objects. The properties such as complexity,
5.Augmented Reality Application
Domains
Only recently have the capabilities of
real-time video image processing, computer
graphic systems and new display technologies
converged to make possible the display of a virtual
graphical image correctly registered with registered
with a view of the 3D environment surrounding the
user. Researchers working with augmented reality
systems have proposed them as solutions in many
domains. The areas that have been discussed range
from entertainment to military training. Many of
the domains, such as medical are also proposed for
traditional virtual reality system
AUGMENTED REALITY - backstreet - 10-04-2017
Augmented Reality
Introduction
Augmented Reality (AR) is a growing area in virtual reality research. The world environment around us provides a wealth of information that is difficult to duplicate in a computer. This is evidenced by the worlds used in virtual environments. Either these worlds are very simplistic such as the environments created for immersive entertainment and games, or the system that can create a more realistic environment has a million dollar price tag such as flight simulators. An augmented reality system generates a composite view for the user. It is a combination of the real scene viewed by the user and a virtual scene generated by the computer that augments the scene with additional information. Augmented reality presented to the user enhances that person's performance in and perception of the world. The ultimate goal is to create a system such that the user cannot tell the difference between the real world and the virtual augmentation of it. To the user of this ultimate system it would appear that he is looking at a single real scene.
Augmented Reality vs. Virtual Reality
Virtual reality is a technology that encompasses a broad spectrum of ideas. The term is defined as "a computer generated, interactive, three-dimensional environment in which a person is immersed. There are three key points in this definition. First, this virtual environment is a computer generated three-dimensional scene, which requires high performance computer graphics to provide an adequate level of realism. The second point is that the virtual world is interactive. A user requires real-time response from the system to be able to interact with it in an effective manner. The last point is that the user is immersed in this virtual environment. One of the identifying marks of a virtual reality system is the head mounted display worn by users. These displays block out all the external world and present to the wearer a view that is under the complete control of the computer. The user is completely immersed in an artificial world and becomes divorced from the real environment. For this immersion to appear realistic the virtual reality system must accurately sense how the user is moving and determine what effect that will have on the scene being rendered in the head mounted display.
The discussion above highlights the similarities and differences between virtual reality and augmented reality systems. A very visible difference between these two types of systems is the immersiveness of the system. Virtual reality strives for a totally immersive environment. In contrast, an augmented reality system is augmenting the real world scene necessitating that the user maintains a sense of presence in that world. The virtual images are merged with the real view to create the augmented display. There must be a mechanism to combine the real and virtual that is not present in other virtual reality work. The computer generated virtual objects must be accurately registered with the real world in all dimensions. Errors in this registration will prevent the user from seeing the real and virtual images as fused. The correct registration must also be maintained while the user moves about within the real environment. Discrepancies or changes in the apparent registration will range from distracting which makes working with the augmented view more difficult, to physically disturbing for the user making the system completely unusable. An immersive virtual reality system must maintain registration so that changes in the rendered scene match with the perceptions of the user.
Download the full Report
AUGMENTED REALITY - sanu20074u - 10-04-2017
An Augmented Reality system supplements the real world with virtual (computer-generated) objects that appear to coexist in the same space as the real world. While many researchers broaden the definition of AR beyond this vision, we can generally find Augmented Reality system to have the following properties: 1) Combines real and virtual objects in a real environment; 2) Runs interactively, and in real time; 3) Aligns real and virtual objects with each other.
Augment Reality can be thought of as a ?middle ground? between Virtual Environment (completely synthetic) & Tele-presence (completely real).
Augmented Reality Vs Virtual Reality
VR was defined as a ?computer generated interactive 3-D environment in which a person is immersed?. The user is completely immersed in an aartificial world & is divorced from the real environment.
VR - Strives For Totally Immersive Environment.
AR ? Augmenting Real World Scenes.
Real Desk with Virtual Lamp and Two Virtual Chairs
? It shows a real desk with a real phone.
? Inside this room are also a virtual lamp and two virtual chairs.
? objects are combined in 3-D, so that the virtual lamp covers the real table, and the real table covers parts of the two virtual chairs
Milgram?s Reality Virtuality Continuum
Taxonomy for Mixed Reality
Reproduction Fidelity
Extent To Present Metaphor
Extent To World Knowledge
Characteristics of Augmented Reality
? Augmentation
? Optical vs. Visual
? Focus and Contrast
? Portability
? Comparison Against Virtual Environments
Application of Augmented Reality
1. Medical
? Virtual fetus inside womb of pregnant patient.
? Mockup of tumor biopsy.
2. Military Training
? Two Views Of A Combined Augmented Reality Virtual System
3. Maintenance & Repair
? Prototype laser printer maintenance application, displaying how to remove the paper tray
? Printer maintenance application
4. Robot and Telerobotics
? Virtual lines show a planned motion of a robot arm.
CONCLUSION Augmented Reality is far behind VE in maturity. The first deployed HMD-based Augmented Reality systems will probably be in the application of aircraft-manufacturing. For example, Boeing has made several trial runs with workers using a prototype system but has not yet made any deployment decisions. The next generation of combat aircraft will have Helmet - Mounted Signals with graphics registered to targets in the environment. These displays combined with short - range steerable missiles that can shoot at target off-boresight, give a tremendous combat advantage to pilots in dogfight. One area where a break-through is required is tracking an HMD outdoors at the accuracy required by Augmented Reality. If this is achieved several interesting applications will become possible.
AUGMENTED REALITY - student - 10-04-2017
(09-21-2008, 06:08 PM)remshad_m Wrote: if you want to get more information about this topics ,just reply with particular name of the topic
Adaptive Multipath Detection(34)
Resilient Packet Ring (RPR).
Organic electronics
The Vanadium Redox Flow Battery System(35)
Cellular Digital Packet Data (Cdpd)
Solid State Lighting
Fibre Optic Communication
Web based remote device monitoring
Quantum dots
Digital Imaging
Military Radars
WiMax
Virtual Reality
Multi threading microprocessors
Evolution Of Embedded System
Chameleon Chip
Imbricate cryptology
Digital steganography
Cryptography
Cellular technologies and security.
Eye gaze human ? computer interface.
Augmented reality.
Electronic Road Pricing System
Cellular geolocation.
Digit recognition using neural network
Microelectronic Pills
Ultra wide band technology.
Enhanced data rates for gsm evolution (edge).
Global Positioning System
The mp3 standard.
Thermal infrared imaging technology
Extreme ultraviolet lithography*
Mesh Radio(36)
Turbo codes.
Jseg-a method for unsupervised segmentation of color texture regions in images and video.
MILLIPEDE(37)
Multiple description coding.
Robotic balancing..
Molecular Electronics(38)
Intelligent transport.
The making of quantum dots.
Packet Cable Network(39)
E-paper.
Fpga offloads dsp?s.
Imaging radar.
Personal Area Network(40)
Wireless power transmission.
Voice recognition based on artificial neural networks.
Remote Accessible Virtual Instrumentation Control Lab(41)
Digital transmission content protection (dtcp)
Artificial immune system.
RTOS ? VXWORKS(42)
Resilient packet ring (rpr).
High performance Computing.
e-governance.
Software Radio(43)
Packet Switching chips
Printable RFID circuits
Adaptive Multipath Detection(34)
Resilient Packet Ring (RPR).
Organic electronics
The Vanadium Redox Flow Battery System(35)
Cellular Digital Packet Data (Cdpd)
Solid State Lighting
Fibre Optic Communication
Web based remote device monitoring
Quantum dots
Digital Imaging
Military Radars
WiMax
Virtual Reality
Multi threading microprocessors
Evolution Of Embedded System
Chameleon Chip
Imbricate cryptology
Digital steganography
Cryptography
Cellular technologies and security.
Eye gaze human ? computer interface.
Augmented reality.
Electronic Road Pricing System
Cellular geolocation.
Digit recognition using neural network
Microelectronic Pills
Ultra wide band technology.
Enhanced data rates for gsm evolution (edge).
Global Positioning System
The mp3 standard.
Thermal infrared imaging technology
Extreme ultraviolet lithography*