By Aaron Yip, , Game Developer :
See-through optics has a sort of inaccessible mysticism thanks to scifi (Iron Man, Star Trek, etc.) and dense hardware vocabulary. But it's actually super easy to get started. So this will be an accessible, layman’s introduction to the topic.
Let's begin with basics. We have these partially transparent displays that mix digital images with the real world. Light rays need to bounce off something to redirect into your eye. From the real world, we are already getting redirected rays. From the digital world, we need to create artificial light (e.g. from LEDs, OLEDs) and then redirect them. The optical device that combines this generated computer image with the real world is called a "combiner." Essentially, a combiner works like a partial mirror that redirects the display light and selectively lets light in through from the real world. Pretty simple.
Like the question suggests, the optical hardware solutions can break down to two categories: conventional HMD optical combiners and emerging waveguide combiners. And hopefully obvious, both of these are very different, and have very different tradeoffs, from opaque virtual reality displays.
There is an extensive history of see-through displays since the late 1960s (I’ll add some good places to get more technical background at the end). Consequently there is a huge range of optical technologies, but it all boils down to basic tradeoffs between resolution, field of view, eye box, image quality, hardware weight/fit, aesthetic form factor, and other features. Ideally, everyone wants stylish, easy-to-use glasses with 200x100 degree FOV (like human eyes) and perfect image quality invented by Tony Stark from Iron Man. But physical and optical limitations of HMDs/NEDs, e.g. how we bounce and bend light with actually wearable hardware, make that unrealistic in the foreseeable future. So we need to figure out which tradeoffs we care about.
Optical hardware is entirely about tradeoffs.
Traditional combiners produce reasonable see-through and imaging quality with consistent performance and affordable materials owed to decades of supply chain development. We can cover two popular varieties of the implementations pictured below: polarized beam combiners (top left) as an example of flat combiners, and off-axis combiners (top right) as an example of curved combiners.
Examples of polarized beam combiner include Google Glass as well as the smart glasses from Epson (Japan), Rockchip (China) and ITRI (Taiwan). Beam splitters can be polarized using LCOS microdisplays, like Google Glass does, or just by using simple half-tone mirrors. Unfortunately, PBC-based displays tend to be small due to weight and size constraints for the combiner, and there may be additional blurriness from the beam split. Google Glass gets 13 degree FOV, and Epson BT-300 gets 23 degree FOV with 1280x720 resolution. Both are among the low end of acceptable range for consumer displays. However, larger FOV and/or resolution would require much bigger and heavier hardware.
Pros: Light, small, relatively affordable ($500–700ish).
Cons: Very limited FOV and display, difficult to improve.
The best modern example of an off-axis, semi-spherical combiner is the Meta 2. Unlike other varieties of small and light combiners, Meta went the other direction in favor of larger FOV and display resolution. They tout a single OLED flat panel to support an “almost 90 degree FOV” and a 2560x1440 pixel display split between both eyes. However, their hardware is bulkier and comparable to VR headsets like the Oculus and HTC Vive. Additional concerns include their low angular resolution (less detailed/crisp images) and how the plastic material of their combiner maintains its quality (e.g. minor perturbations are emphasized and strain over time may lead to eventual visual artifacts) — which are choices they made to lower costs. An older example of curved combiners is Link's Advanced Helmet Mounted Display.
Pros: High FOV and resolution, relatively affordable ($900ish).
As you see, trying to improve traditional combiners with respect to FOV and resolution means a smaller eye box, thicker combiner optics, larger combiner further away, and/or worse imaging quality. It’s not about computational performance limits as much as physical ones from how light behaves with the hardware.
To address this hard tradeoff problem, new technology is pushing into non-conventional techniques like holographic and diffractive optics. These techniques use what’s called waveguide grating or waveguide hologram to progressively extract a collimated image guided by total internal reflection (TIR) in a waveguide pipe. The pipe is a thin sheet of glass or plastic that the light bounces through. In effect, you can think of a waveguide like a router that transmits the image at your eye.
Cons: Bulky, low angular resolution, material quality risk.
Waveguides are the most technically sophisticated type of see-through optics, and they are equally hard to design. These ideas however are not new; folks have been exploring waveguides for optics since the early 80s, and since then, companies like Sony (pictured top above), Konica/Minolta (pictured bottom above), Nokia/Microsoft (pictured below), Magic Leap, and many others have all worked on various waveguide combiners.
Surface relief slanted sub-wavelength gratings, for example, are the commonly presumed implementation used for Microsoft Hololens. Here, the waveguide has a series of very fine structures in a linear pattern (fine on the order of the wavelengths of light). Grating acts like a lens to bend the light through the TIR until it exits towards the eye. A pleasant result from this process is "pupil expansion"; the exiting light can slightly diffuse to increase its FOV.
All in all, state-of-the-art waveguide techniques might get you something close to 32Hx18V degree FOV at 1920x1080, potentially without as much of the bulk and weight of traditional combiner workarounds. Magic Leap patents suggest its technology aspires to get close to 120Hx80V degree horizontal FOV but may end up closer to 50-55 degree FOV. This could more promising, or at least least hyped, than traditional approaches - but very little of the promise has been demonstrated so far. Moreover, waveguide combiners have their own Set of challenges.
First, waveguides require a lot of precision and are finicky - volume holographic media like photopolymers, DCG, silver halides, etc. can change based off environmental temperature, humidity, and / or pressure. Second, the angular resolution weakens with more diffusion (ie FOV vs imaging details tradeoff). And finally, the supply chain has not been readily established for the technologies so mass production is difficult and expensive. Not to mention the hefty $ 1B + ongoing R & D cost by both companies.
Pros: Potentially better FOV and resolution on medium sized devices.
Cons: Expensive (estimated $ 3k +), technology still actively in development, maybe black magic.
In summary, this discussion is mostly between a proven and relatively well explored space with physical tradeoff limitations (traditional combiners) compared to a much hyped space of experimental technologies that could bypass these physical limitations (waveguide combiners). Personally, I think the distrust in waveguide technology is not entirely unmerited; no public demo has shown any work better than traditional combiners. On the other hand, I also think these huge investments make complete sense.
Consumers, calibrated on scifi expectations, have been underwhelmed at most/all traditional combiner hardware. In the past five decades of optics, AR existed as niche products. There may still be interesting improvements in traditional combiners (like ODG glasses), but for Microsoft and Magic Leap, waveguides are moonshot projects for AR optics promising enough to be accepted by the mass market.
Thanks for reading! I really just dabble. Hopefully this article can serve as a starting point with directions to better resources.