WO2001009685A1 - Display system with eye tracking - Google Patents

Display system with eye tracking Download PDF

Info

Publication number
WO2001009685A1
WO2001009685A1 PCT/US2000/021454 US0021454W WO0109685A1 WO 2001009685 A1 WO2001009685 A1 WO 2001009685A1 US 0021454 W US0021454 W US 0021454W WO 0109685 A1 WO0109685 A1 WO 0109685A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
holographic
eye
operable
optical
Prior art date
Application number
PCT/US2000/021454
Other languages
French (fr)
Inventor
Milan M. Popovich
Original Assignee
Digilens Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Digilens Inc. filed Critical Digilens Inc.
Priority to AU66232/00A priority Critical patent/AU6623200A/en
Publication of WO2001009685A1 publication Critical patent/WO2001009685A1/en

Links

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G09G3/003Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00Optical elements other than lenses
    • G02B5/32Holograms used as optical elements
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G09G3/002Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00Control of display operating conditions
    • G09G2320/02Improving the quality of display appearance
    • G09G2320/0261Improving the quality of display appearance in the context of movement of objects on the screen or movement of the observer relative to the screen

Definitions

  • the present invention relates generally to display systems, and more particularly, to a head mounted display system having an eye tracking device for tracking the change in the gaze direction of a user's eye and modifying the displayed image in response to the eye movement.
  • the head mounted display generally includes a support member for mounting the display on a head of a user and various optical and display components.
  • the components are arranged to magnify an image displayed on a compact image display panel such as a liquid crystal display (LCD) and to display the magnified image ahead of the user through the optical system.
  • LCD liquid crystal display
  • the user typically does not directly observe an image displayed on a monitor or screen, but instead observes a magnified virtual image converted from the image displayed on the image display panel.
  • Some head mounted displays include an eye tracking system which provides feedback to change the image or the focus of the image displayed as the user's eye moves so that the user perceives that he is focusing on a different portion of a panoramic scene as he shifts his eye.
  • Eye tracking systems may be used, for example, in flight control, flight simulation, and virtual reality imaging displays.
  • the eye tracking system may also be used to generate information based on the position of the eye with respect to an image on a display to enable the viewer to control hands-free movement of a cursor, such as a cross-hair on the display.
  • the eye tracking system allows an operator to manipulate a computer completely hands-free by tracking the eye and interpreting its movements as mouse commands to a computer application.
  • Conventional eye tracking systems for use with head mounted displays typically reflect light off of a user's eye to track movement of the eye.
  • An optical device is positioned to receive the reflected light and project the light onto a light detector.
  • the optical device is typically positioned on a different optical axis than used to display an image to the user so that the eye tracking system does not interfere with the image display. This arrangement increases the size and weight of the head mounted display.
  • conventional eye tracking devices can add significant weight to the front of the display.
  • a display system with eye tracking capability generally comprises a display device operable to display an image and an eye tracking system operable to track movement of an eye of a user of the system.
  • the tracking system comprises an optical device operable to receive light reflected from the user's eye and project the light towards a detector to detect movement of a user's eye based on the reflected light.
  • the detector is coupled to an image generator operable to modify the image in response to movement detected by the eye tracking system.
  • the system further includes a holographic diffraction device operable to direct the image towards a user of the system.
  • the optical device is positioned such that light received and projected by the optical device passes through the holographic diffraction device without substantial alteration and the holographic diffraction device is configured to diffract light having a wavelength band different than the light received and projected by the optical device.
  • the optical device may be configured to act upon infrared radiation and the holographic diffraction device may be configured to act upon visible wavelength light. Alternatively, the optical device may be configured to act upon visible wavelength light in a band outside or between the wavelength bands acted upon by the holographic diffraction device.
  • the holographic diffraction device may include a plurality of holographic optical elements switchable between an active state wherein light incident on the element is diffracted and a passive state wherein light incident on the element is transmitted without substantial alteration.
  • the holographic optical elements may each be configured to diffract light of a different wavelength band.
  • a controller may be coupled to each of the holographic optical elements to create a sequence of monochrome images which are combined to form a color image.
  • the system may also include a second holographic diffraction device configured to substantially compensate for chromatic dispersion created by the first holographic diffraction device.
  • a head mounted display system comprises a display device operable to display an image and an optical system operable to receive the image from the display device and project the image to a user of the display system.
  • the optical system includes at least one holographic optical element configured to diffract light having a first wavelength band.
  • the system further includes an eye tracking system operable to track movement of an eye of the user.
  • the eye tracking system comprises at least one optical device configured to act upon light having a second wavelength band different from the first wavelength band.
  • the holographic optical element is positioned adjacent to the optical device of the tracking system and on a common optical path therewith.
  • Fig. 1 is a schematic side view of a display system of the present invention.
  • Fig. 2 is a perspective of a holographic optical element and light source for use with the display system of Fig. 1.
  • Fig. 3 is a partial front view of the holographic optical element of Fig. 2 illustrating an electrode and electric circuit of the holographic optical element.
  • Fig. 4 is a schematic of a holographic device having three holographic optical elements and a control circuit.
  • the system 10 includes a display device 14 operable to produce an image for viewing by a user of the system, an optical relay system 16 positioned to receive the image from the display device and project the image onto a holographic diffraction device, generally indicated at 18. The image is directed by the holographic device 18 onto an eyepiece, generally indicated at 20, which reflects the image for viewing by the user.
  • the system 10 further includes an eye tracking system, generally indicated at 30, operable to track changes in the position or attitude of the viewer's eye 28 so that the image displayed by the display device 14 can be altered in response to such changes.
  • the display system 10 is preferably configured for use as a head mounted display.
  • the head mounted display includes a headpiece (not shown) designed to be worn by a viewer and the display system 10 for producing wide-angle, electronically generated virtual images to one or both eyes of the viewer.
  • the display system 10 may also be used in narrow field of view displays.
  • the headpiece may include a frame configured to fit over a viewer's head and a mask which fits over the viewer's eye region, as is well known by those skilled in the art.
  • the display system may comprise left and right optical systems or only one optical system. For example, two optical systems may be used as described in U.S. Patent Application Serial No. 09/405,444, filed September 23, 1999, by A. Preston, to provide a variable aspect ratio system.
  • the display system may also include a single image generator and projection system operable to project a left image to a left eye of a viewer and a right image to a right eye of the viewer as described in U.S. Patent Application Serial No. 09/439,063, filed November 12, 1999, by M. Popovich.
  • the display device 14 produces an image under control of an image generator 32 which may provide, for example, video or graphic information.
  • the display device 14 includes a display surface typically comprising an array of pixels for displaying monochromatic data or monochromatic images in accordance with signals generated by the image generator 32.
  • the display device 14 may comprise a flat panel display (e.g., a reflective liquid crystal display (LCD) panel, or other spatial light modulator (SLM) which reflects light produced externally).
  • the image display panel may be a miniature reflective LCD having either a nematic or ferroelectric material on a silicon backplane, for example.
  • the reflective display panel utilizes an external light source to reflect and modulate light off the front of the microdisplay.
  • the display panel may also be based on transmissive display technologies.
  • the display panel is color sequentially illuminated using separate red, green, and blue sources or, alternatively, a white source combined with a color sequential filter.
  • red, green, and blue sources or, alternatively, a white source combined with a color sequential filter.
  • the latter may be based on electro-mechanical techniques involving band pass filters which are rotated or displaced in some manner in front of the source, for example.
  • the display device 14 may be also a miniature reflective silicon backplane device, such as a SVGA (800x600 pixels) device available from Colorado MicroDisplay, of Boulder Colorado, for example.
  • SVGA 800x600 pixels
  • a micro-electromechanical system such as a Digital Light Processor (DLP) using a Digital Micromirror DeviceTM (DMD) available from Texas Instruments, may also be used as the display device 14.
  • the DMD is a micromechanical silicon chip having movable mirrors which reflect light to create high quality images. An image is formed on the reflective surface of the DMD by turning the mirrors on or off digitally at a high speed. Color is added to the image by filtering light through a color system.
  • the color system may comprise a light source which directs white light through a condenser lens and a red, green, and blue color filter and then onto the surface of the DMD chip, for example. Minors are turned on or off for different times depending upon how much light of each color is needed per pixel.
  • the display device 14 may also be a diffractive display device such as a Grating Light ValveTM (GLV) available from Silicon Light Machines (formerly Echelle, Inc.).
  • the GLV uses micro-electromechanical systems to create multiple ribbon structures which can move small distances to create a grating which selectively diffracts specified wavelengths of light.
  • Each grating defines a picture element (pixel) formed on the surface of a silicon chip and the array of pixels formed becomes the image source for display projection.
  • a white light source is filtered sequentially through red, green, and blue filters.
  • the display panel of the display device 14 may be coupled with a source of illumination and an inclined beamsplitter (not shown), which is used to direct light from the illumination source onto the display panel and allow light from the image itself to pass onto the optical relay system 16.
  • the optical relay system 16 may include components such as lenses and mirrors.
  • the lenses magnify the input image and are configured and positioned to provide appropriate focal length and other optical characteristics. Additional optical elements may be provided to correct for aberrations, as is well known by those skilled in the art.
  • the lenses may include cylinders, prisms, and off-axis aspheric elements to correct for aberrations due to the off-axis, non-symmetric nature of the display system.
  • the optical relay system 16 is preferably designed to minimize dispersion and chromatic aberrations contributed by the holograms (described below).
  • the optical relay system 16 is preferably fabricated from optical plastics (e.g., optical acrylics).
  • the optical relay system 16 may also include diffractive optics to reduce the weight of the system.
  • the image passes through the holographic diffraction device 18 which comprises three holographic optical elements 42, 44, 46 each holographically configured such that only a particular monochromatic light is diffracted by the element.
  • the elements 42, 44, 46 are preferably switchable so that the elements can be selectively activated and deactivated to transmit the image which is formed by sequentially manipulating different colors.
  • the holographic optical elements 42, 44, 46 each include a hologram interposed between two electrodes 62 (Figs. 1 and 3). The hologram is used to control transmitted light beams based on the principles of diffraction.
  • the hologram selectively directs an incoming light beam from light source 50 either towards or away from a viewer and selectively diffracts light at certain wavelengths into different modes in response to a voltage applied to the electrodes 62 (Figs. 2 and 3).
  • Light passing through the hologram in the same direction that the light is received from the light source 50 is referred to as the zeroth (0th) order mode 68.
  • liquid crystal droplets within the holographic optical element are oriented such that the hologram is present in the element and light is diffracted from the zeroth order mode to a first (1st) order mode 70 of the hologram.
  • a voltage is applied to the holographic optical element the liquid crystal droplets become realigned effectively erasing the hologram, and the incoming light passes through the holographic optical element in the zeroth order mode 68.
  • the holographic optical elements 42, 44, 46 may also be reflective rather than transmissive.
  • the arrangement of the holographic device and optical components would be modified to utilize reflective properties of the hologram rather than the transmissive properties described herein.
  • the light that passes through the hologram is diffracted to form an image by interference fringes recorded in the hologram.
  • the hologram is able to perform various optical functions which are associated with traditional optical elements, such as lenses and prisms, as well as more sophisticated optical operations which would normally require very complex systems of conventional components.
  • the hologram may be configured to perform operations such as deflection, focusing, or color filtering of the light, for example.
  • the hologram may be a Bragg (thick or volume) hologram or Raman- Nath (thin) hologram.
  • Raman-Nath holograms are thinner and require less voltage to switch light between various modes of the hologram, however, Raman-Nath holograms are not as efficient as Bragg holograms.
  • the Bragg holograms provide high diffraction efficiencies for incident beams with wavelengths close to the theoretical wavelength satisfying the Bragg diffraction condition and within a few degrees of the theoretical angle which also satisfies the Bragg diffraction condition.
  • the holograms are preferably recorded on a photopolymer/liquid crystal composite material (emulsion) such as a holographic photopolymeric film which has been combined with liquid crystal, for example.
  • a photopolymer/liquid crystal composite material emulsion
  • the presence of the liquid crystal allows the hologram to exhibit optical characteristics which are dependent on an applied electrical field.
  • the photopolymeric film may be composed of a polymerizable monomer having dipentaerythritol hydroxypentacrylate, as described in PCT Publication, Application Serial No. PCT/US97/12577, by Sutherland et al., which is incorporated herein by reference.
  • the liquid crystal may be suffused into the pores of the photopolymeric film and may include a surfactant.
  • the refractive properties of the holographic optical elements 42, 44, 46 depend primarily on the recorded holographic fringes in the photopolymeric film.
  • the interference fringes may be created by applying beams of light to the photopolymeric film. Alternatively, the interference fringes may be artificially created by using highly accurate laser writing devices or other replication techniques, as is well known by those skilled in the art.
  • the holographic fringes may be recorded in the photopolymeric film either prior to or after the photopolymeric film is combined with the liquid crystal. In the preferred embodiment, the photopolymeric material is combined with the liquid crystal prior to the recording.
  • the liquid crystal and the polymer material are pre-mixed and the phase separation takes place during the recording of the hologram, such that the holographic fringes become populated with a high concentration of liquid crystal droplets.
  • This process can be regarded as a "dry” process, which is advantageous in terms of mass production of the switchable holographic optical elements.
  • the electrodes (electrode layers) 62 are positioned on opposite sides of the emulsion and are preferably transparent so that they do not interfere with light passing through the hologram.
  • the electrodes 62 may be formed from a vapor deposition of Indium Tin Oxide (ITO) which typically has a transmission efficiency of greater than 80%, or any other suitable substantially transparent conducting material.
  • ITO Indium Tin Oxide
  • the transmission of the electrodes can be improved to greater than 97% by applying multi-layer anti-reflection coatings to the electrodes.
  • the electrodes 62 are connected to an electric circuit 78 operable to apply a voltage to the electrodes, to generate an electric field (Fig. 3).
  • the hologram is in the diffractive (active) state and the holographic optical element diffracts propagating light in a predefined manner.
  • the electrodes may be different than described herein. For example, a plurality of smaller electrodes may be used rather than two large electrodes which substantially cover surfaces of the holograms.
  • the holographic optical elements 42, 44, 46 may also be formed on curved substrates, as described in U.S. Patent Application Serial No. 09/416,076, by M. Popovich, filed October 12, 1999.
  • Each holographic optical element 42, 44, 46 of the holographic diffraction device 18 is holographically configured such that only a particular monochromatic light is diffracted by the hologram.
  • the red optical element 42 has a hologram which is optimized to diffract red light
  • the green optical element 44 has a hologram which is optimized to diffract green light
  • the blue optical element 46 has a hologram which is optimized to diffract blue light.
  • a holographic device controller 90 drives switching circuitry 94 associated with the electrodes 62 on each of the optical elements 42, 44, 46 to a ply a voltage to the electrodes (Figs. 3 and 4).
  • the electrodes 62 are individually coupled to the device controller through a voltage controller 102 which selectively provides an excitation signal to the electrodes 62 of a selected holographic optical element, switching the hologram to the passive state.
  • the voltage controller 102 also determines the specific voltage level to be applied to each electrode 62.
  • the voltage controller 102 operates to sequentially display three monochromatic images of the color input image.
  • the electrodes 62 attached to each of the holograms 42, 44, 46 are sequentially enabled such that a selected amount of red, green, and blue light is directed towards the viewer. For example, when a red monochromatic image is projected, the voltage controller 102 switches the green and blue holograms 44, 46 to the passive state by applying voltages to their respective electrodes 62.
  • the supplied voltages to the electrodes 62 of the green and blue holograms 44, 46 create a potential difference between the electrodes, thereby generating an electrical field within the green and blue holograms.
  • the presence of the generated electrical field switches the optical characteristic of the holograms 44, 46 to the passive state.
  • the green and blue holograms 44, 46 in the passive state and the red hologram 42 in the diffractive state only the red hologram optically diffracts the projected red image. Thus, only the portion of the visible light spectrum corresponding to the red light is diffracted to the viewer.
  • the green hologram 44 is next changed to the diffractive state by deenergizing the corresponding electrodes 62 and the electrodes of the red hologram 42 are energized to change the red hologram to the passive state so that only green light is diffracted.
  • the blue hologram 46 is then changed to the diffractive state by deenergizing its electrodes 62 and the electrodes of the green hologram 44 are energized to change the green hologram to the passive state so that only blue light is diffracted.
  • the holograms are sequentially enabled with a refresh rate which is faster than the response time of a human eye so that a color image will be created in the viewer's eye due to the integration of the red, green, and blue monochrome images created from each of the red, green, and blue holograms. Consequently, the holographic devices will sequentially transmit red, green, and blue lights so that the final viewable image will appear to be displayed as a composite color.
  • the red, green, and blue holographic elements 42, 44, 46 may be cycled on and off in any order.
  • the holographic diffraction 18 is preferably configured to compensate for chromatic dispersion introduced by the holographic diffraction elements of the eyepiece 20 and correct dispersion, chromatic, and geometric aberrations created due to the holographic diffraction elements of the eyepiece 20 operating off-axis and over large spectral bandwidths. More particularly, the characteristics of the holographic optical elements of the eyepiece 20 and the red, green, and blue holograms are preferably optimized so that the dispersion introduced by the holographic optical elements are substantially compensated for by the holograms of the holographic diffraction device 18.
  • the eyepiece 20 directs the image received from the holographic diffraction device 18 to the viewer.
  • the eyepiece 20 preferably comprises a plurality of holographic optical elements 112, 114, 116, each of which is operable to act upon a respective one of the wavelength bands produced by the display device, as described above with respect to the elements 42, 44, 46 of the holographic diffraction device 18.
  • the holographic elements 112, 114, 116 are switched rapidly in succession into and out of their active states by controller 120, which is coupled to the controller 90 of the holographic diffraction device 18.
  • the controllers 90, 120 are preferably operated together so that the corresponding holographic elements of the holographic diffraction device 18 and eyepiece 20 (42 and 112) (44 and 114) (46 and 116) are switched generally simultaneously into their active and passive states.
  • the holographic optical elements of either the diffraction device 18 or eyepiece 20 may also be nonswitchable (i.e., remain in active state at all times).
  • the elements 112, 114, 116 of the eyepiece 20 are preferably reflective.
  • the eyepiece 20 may also comprise conventional mirror devices, rather than holographic devices. In this case the optical devices are mounted on the front surface of the mirror (i.e., between the mirror and eye 28).
  • the eye tracking system 30 is used to track changes in the attitude of the viewer's eye 28 so that the image generated by the image generator 32 can be altered in response to the changes.
  • the eye tracker system 30 uses light (e.g., infrared light) reflected by the eye 28 (e.g., on the cornea or further in the eye) to track the direction of gaze by the eye.
  • the tracker system 30 includes a plurality of light emitters 120 positioned around an outer periphery of the eyepiece 20. The emitters 120 may be positioned, for example, every other light.
  • the emitters 120 are configured to project radiation in a broad wash onto the eye (as indicated by arrows A). Radiation reflected back from the eye (as indicated by arrows B) is directed by an optical device 124 (as indicated by a ⁇ ows C) onto a detector 126 located at a position displaced laterally from the main optical axis of the display system.
  • the detector 126 may include, for example, a miniature two- dimensional detector array, crossed one dimensional detector arrays, or a peak intensity detection device (e.g., position sensing device).
  • a detector 126 which may be used with the system described herein is a two- dimensional position sensing detector manufactured by Ffamamatsu of Japan under designation S4744. The device has a spectral response from 760 to 1 lOOnm (peak response at 960 nm), photosensitivity of 0.58A/watt and rise time of 25 microseconds. It is to be understood that other types of detectors having different characteristics may also be used.
  • Signals from the detector 126 are processed by a processor 130 to provide the image generator 32 data on changes in the attitude of the eye.
  • the image generator 32 alters the image displayed by the display device 14 so that the image seen by the observer moves with the direction of gaze of the user.
  • Additional optical elements such as focusing lenses and filters may be mounted adjacent to the detector 126 at a location between the optical device 124 and the detector, to facilitate the detection process.
  • the position of the eyeball 28 can be identified by tracking one of its visible features.
  • the components of the eye tracker system 30 and the wavelength of the radiation used are preferably selected so that their characteristics can be optimized to allow particular features of the eye to be easily recognized and tracked.
  • the tracking system may be optimized to generate a sharp image of the perimeter of the pupil of the eye, with eye orientation being determined by measuring the shape of the pupil perimeter.
  • the tracking system 30 may detect the peak corneal reflections which occur on the boundaries of the lens and cornea when infrared light is directed into the user's eye.
  • the tracking system 30 may also optically detect and track the limbus (boundary between the white sclera and the dark iris of the eye) since the sclera is white and the iris is darker. Other eye tracking methods may be used as well known by those skilled in the art.
  • the optical device 124 is disposed directly behind the eyepiece 20 (i.e., optically on the opposite side of the eyepiece than the eye 28). Thus, radiation reflected back to the eye must pass through the eyepiece 20 before and after being acted upon by the optical device 124 (as indicated by arrows B and C).
  • the system 10 is designed such that the eyepiece 20 has no substantial effect on this radiation. This allows the optical device 124 of the tracking system 30 to be positioned on a common optical axis with the eyepiece 20, thus providing a compact and lightweight arrangement.
  • the holographic optical elements 112, 114, 116 of the eyepiece 20 are designed to act upon relatively narrow wavelength bands of light in the red, green, and blue regions of the visible spectrum (e.g., 620 - 640 nm, 520 - 540 nm, 465 - 485 nm, respectively).
  • the radiation used by the eye tracker system e.g., 620 - 640 nm, 520 - 540 nm, 465 - 485 nm, respectively.
  • the eye tracker system 30 can utilize radiation in the near infrared band, which typically comprises wavelengths of 1000 nanometers or less.
  • the holographic diffraction elements each act on a wavelength bandwidth of around 20 nanometers, this leaves a substantial amount of the visible spectrum (which generally extends between around 400 nm - 465 nm, 485 nm - 520 nm, and 640 nm - 700 nm) unused and the eye tracking system 30 can utilize radiation having a wavelength band in one of the unused regions.
  • the term light as used herein includes visible and nonvisible light (e.g., red, green, and blue visible light and infrared radiation).
  • the optical device 124 may include a plurality of holographic diffraction elements configured to act on the wavelengths employed for the eye tracking function. For example, each element may be designed to image a particular area of the eye surface, thereby increasing accuracy in the determination of the eye gaze direction. In this case, multiple detectors 30 may be used, or a single detector may be time-multiplexed. Under these circumstances, it is advantageous for the holographic diffraction elements to be switchable. However, for other configurations, the holographic elements may be nonswitchable.
  • the optical device 124 may be formed integrally with the eyepiece 20 or as a separate device.
  • the optical device 124 may also be positioned on the opposite side of the eyepiece 20 (i.e., between the eyepiece and eye 28) in which case the optical device would be configured such that it does not act upon the wavelength band of light that is diffracted by the eyepiece 20.
  • the holographic elements of the diffraction device 18 are shown configured to be transmissive and the elements of the eyepiece 20 and optical device 124 are shown configured to be reflective. It is to be understood that the arrangement of components and reflective or transmissive configuration of the elements may be different than described herein without departing from the scope of the invention.
  • the transmission holograms described above are sensitive to the polarization state of incident light and exhibit maximum diffraction efficiency for p-polarized light, with the response to s-polarized light being around 1% for that of p-polarized light.
  • the system 10 may include elements which make use of both the p- polarized light and s-polarized light, such as disclosed in U.S. Patent Application Serial Number 09/478,150, filed January 5, 2000, by M. Popovich et al, which is incorporated herein by reference in its entirety.
  • pairs of holographic diffraction elements may be used with one element in the pair acting on the p-polarized component and the other acting on the s- polarized components.
  • reflection holograms are used, these additional provisions are not required since reflection holograms only start to become polarization sensitive at large angles of incidence, typically much greater than 45 degrees.

Abstract

A display system comprising a display device operable to display an image and eye tracking system operable to track movement of an eye of a user of a system. The tracking system comprises an optical device (124) operable to receive light reflected from the user's eye (28) and project the light towards the detector (126) to detect movement of the user's eye based on the reflected light. The detector is coupled to an image generator operable to modify the image in response to movement detected by the eye tracking system. The system further includes a holographic diffraction device operable to direct the image towards the user of the system. The optical device without substantial alteration and a holographic diffraction device is positioned such that light received and projected by the optical device passes through the holographic diffraction device is configured to diffract light having a wavelength band different than the light received and projected by the optical device.

Description

DISPLAY SYSTEM WITH EYE TRACKING
RELATED APPLICATIONS
The present application claims the benefit of U.S. Provisional Application Serial No. 60/146,980, filed August 3, 1999.
BACKGROUND OF THE INVENTION
The present invention relates generally to display systems, and more particularly, to a head mounted display system having an eye tracking device for tracking the change in the gaze direction of a user's eye and modifying the displayed image in response to the eye movement.
Head mounted displays have received considerable attention as a technique for displaying high magnification, large field of view and high definition virtual images. The head mounted display generally includes a support member for mounting the display on a head of a user and various optical and display components. The components are arranged to magnify an image displayed on a compact image display panel such as a liquid crystal display (LCD) and to display the magnified image ahead of the user through the optical system. The user typically does not directly observe an image displayed on a monitor or screen, but instead observes a magnified virtual image converted from the image displayed on the image display panel.
Some head mounted displays include an eye tracking system which provides feedback to change the image or the focus of the image displayed as the user's eye moves so that the user perceives that he is focusing on a different portion of a panoramic scene as he shifts his eye. Eye tracking systems may be used, for example, in flight control, flight simulation, and virtual reality imaging displays. The eye tracking system may also be used to generate information based on the position of the eye with respect to an image on a display to enable the viewer to control hands-free movement of a cursor, such as a cross-hair on the display. Thus, the eye tracking system allows an operator to manipulate a computer completely hands-free by tracking the eye and interpreting its movements as mouse commands to a computer application.
Conventional eye tracking systems for use with head mounted displays typically reflect light off of a user's eye to track movement of the eye. An optical device is positioned to receive the reflected light and project the light onto a light detector. The optical device is typically positioned on a different optical axis than used to display an image to the user so that the eye tracking system does not interfere with the image display. This arrangement increases the size and weight of the head mounted display. Furthermore, conventional eye tracking devices can add significant weight to the front of the display. SUMMARY OF THE INVENTION
A display system with eye tracking capability is disclosed. The display system generally comprises a display device operable to display an image and an eye tracking system operable to track movement of an eye of a user of the system. The tracking system comprises an optical device operable to receive light reflected from the user's eye and project the light towards a detector to detect movement of a user's eye based on the reflected light. The detector is coupled to an image generator operable to modify the image in response to movement detected by the eye tracking system. The system further includes a holographic diffraction device operable to direct the image towards a user of the system. The optical device is positioned such that light received and projected by the optical device passes through the holographic diffraction device without substantial alteration and the holographic diffraction device is configured to diffract light having a wavelength band different than the light received and projected by the optical device.
The optical device may be configured to act upon infrared radiation and the holographic diffraction device may be configured to act upon visible wavelength light. Alternatively, the optical device may be configured to act upon visible wavelength light in a band outside or between the wavelength bands acted upon by the holographic diffraction device. The holographic diffraction device may include a plurality of holographic optical elements switchable between an active state wherein light incident on the element is diffracted and a passive state wherein light incident on the element is transmitted without substantial alteration. The holographic optical elements may each be configured to diffract light of a different wavelength band. A controller may be coupled to each of the holographic optical elements to create a sequence of monochrome images which are combined to form a color image. The system may also include a second holographic diffraction device configured to substantially compensate for chromatic dispersion created by the first holographic diffraction device.
In another aspect of the invention a head mounted display system comprises a display device operable to display an image and an optical system operable to receive the image from the display device and project the image to a user of the display system. The optical system includes at least one holographic optical element configured to diffract light having a first wavelength band. The system further includes an eye tracking system operable to track movement of an eye of the user. The eye tracking system comprises at least one optical device configured to act upon light having a second wavelength band different from the first wavelength band. The holographic optical element is positioned adjacent to the optical device of the tracking system and on a common optical path therewith. The above is a brief description of some deficiencies in the prior art and advantages of the present invention. Other features, advantages, and embodiments of the invention will be apparent to those skilled in the art from the following description, drawings, and claims.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 is a schematic side view of a display system of the present invention.
Fig. 2 is a perspective of a holographic optical element and light source for use with the display system of Fig. 1.
Fig. 3 is a partial front view of the holographic optical element of Fig. 2 illustrating an electrode and electric circuit of the holographic optical element.
Fig. 4 is a schematic of a holographic device having three holographic optical elements and a control circuit.
Corresponding reference characters indicate corresponding parts throughout the several views of the drawings. DETAILED DESCRIPTION OF THE INVENTION
The following description is presented to enable one of ordinary skill in the art to make and use the invention. Descriptions of specific embodiments and applications are provided only as examples and various modifications will be readily apparent to those skilled in the art. The general principles described herein may be applied to other embodiments and applications without departing from the scope of the invention. Thus, the present invention is not to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features described herein. For purpose of clarity, details relating to technical material that is known in the technical fields related to the invention have not been described in detail.
Referring now to the drawings, and first to Fig. 1 , a display system of the present invention is shown and generally indicated at 10. The system 10 includes a display device 14 operable to produce an image for viewing by a user of the system, an optical relay system 16 positioned to receive the image from the display device and project the image onto a holographic diffraction device, generally indicated at 18. The image is directed by the holographic device 18 onto an eyepiece, generally indicated at 20, which reflects the image for viewing by the user. The system 10 further includes an eye tracking system, generally indicated at 30, operable to track changes in the position or attitude of the viewer's eye 28 so that the image displayed by the display device 14 can be altered in response to such changes.
The display system 10 is preferably configured for use as a head mounted display. The head mounted display includes a headpiece (not shown) designed to be worn by a viewer and the display system 10 for producing wide-angle, electronically generated virtual images to one or both eyes of the viewer. The display system 10 may also be used in narrow field of view displays. The headpiece may include a frame configured to fit over a viewer's head and a mask which fits over the viewer's eye region, as is well known by those skilled in the art. The display system may comprise left and right optical systems or only one optical system. For example, two optical systems may be used as described in U.S. Patent Application Serial No. 09/405,444, filed September 23, 1999, by A. Preston, to provide a variable aspect ratio system. The display system may also include a single image generator and projection system operable to project a left image to a left eye of a viewer and a right image to a right eye of the viewer as described in U.S. Patent Application Serial No. 09/439,063, filed November 12, 1999, by M. Popovich.
The display device 14 produces an image under control of an image generator 32 which may provide, for example, video or graphic information. The display device 14 includes a display surface typically comprising an array of pixels for displaying monochromatic data or monochromatic images in accordance with signals generated by the image generator 32. The display device 14 may comprise a flat panel display (e.g., a reflective liquid crystal display (LCD) panel, or other spatial light modulator (SLM) which reflects light produced externally). The image display panel may be a miniature reflective LCD having either a nematic or ferroelectric material on a silicon backplane, for example. The reflective display panel utilizes an external light source to reflect and modulate light off the front of the microdisplay. The display panel may also be based on transmissive display technologies. Preferably, the display panel is color sequentially illuminated using separate red, green, and blue sources or, alternatively, a white source combined with a color sequential filter. The latter may be based on electro-mechanical techniques involving band pass filters which are rotated or displaced in some manner in front of the source, for example.
The display device 14 may be also a miniature reflective silicon backplane device, such as a SVGA (800x600 pixels) device available from Colorado MicroDisplay, of Boulder Colorado, for example.
A micro-electromechanical system, such as a Digital Light Processor (DLP) using a Digital Micromirror Device™ (DMD) available from Texas Instruments, may also be used as the display device 14. The DMD is a micromechanical silicon chip having movable mirrors which reflect light to create high quality images. An image is formed on the reflective surface of the DMD by turning the mirrors on or off digitally at a high speed. Color is added to the image by filtering light through a color system. The color system may comprise a light source which directs white light through a condenser lens and a red, green, and blue color filter and then onto the surface of the DMD chip, for example. Minors are turned on or off for different times depending upon how much light of each color is needed per pixel.
The display device 14 may also be a diffractive display device such as a Grating Light Valve™ (GLV) available from Silicon Light Machines (formerly Echelle, Inc.). The GLV uses micro-electromechanical systems to create multiple ribbon structures which can move small distances to create a grating which selectively diffracts specified wavelengths of light. Each grating defines a picture element (pixel) formed on the surface of a silicon chip and the array of pixels formed becomes the image source for display projection. A white light source is filtered sequentially through red, green, and blue filters. By synchronizing the image data stream's red, green, and blue pixel data with the appropriate filtered source light, combinations of red, green, and blue light are diffracted to the lens group for projection of the image into the holographic device. It is to be understood that display panels other than those described herein may be used without departing from the scope of the invention.
The display panel of the display device 14 may be coupled with a source of illumination and an inclined beamsplitter (not shown), which is used to direct light from the illumination source onto the display panel and allow light from the image itself to pass onto the optical relay system 16. The optical relay system 16 may include components such as lenses and mirrors. The lenses magnify the input image and are configured and positioned to provide appropriate focal length and other optical characteristics. Additional optical elements may be provided to correct for aberrations, as is well known by those skilled in the art. For example, the lenses may include cylinders, prisms, and off-axis aspheric elements to correct for aberrations due to the off-axis, non-symmetric nature of the display system. Since the holographic display system is preferably designed to be as compact as possible, the projection optical system will be highly off-axis. Thus, the optical relay system is preferably designed to minimize dispersion and chromatic aberrations contributed by the holograms (described below). In order to minimize weight of the display system, as well as the manufacturing cost, the optical relay system 16 is preferably fabricated from optical plastics (e.g., optical acrylics). The optical relay system 16 may also include diffractive optics to reduce the weight of the system.
After passing through the optical relay system 16, the image passes through the holographic diffraction device 18 which comprises three holographic optical elements 42, 44, 46 each holographically configured such that only a particular monochromatic light is diffracted by the element. The elements 42, 44, 46 are preferably switchable so that the elements can be selectively activated and deactivated to transmit the image which is formed by sequentially manipulating different colors. The holographic optical elements 42, 44, 46 each include a hologram interposed between two electrodes 62 (Figs. 1 and 3). The hologram is used to control transmitted light beams based on the principles of diffraction. The hologram selectively directs an incoming light beam from light source 50 either towards or away from a viewer and selectively diffracts light at certain wavelengths into different modes in response to a voltage applied to the electrodes 62 (Figs. 2 and 3). Light passing through the hologram in the same direction that the light is received from the light source 50 is referred to as the zeroth (0th) order mode 68. When no voltage is applied to the electrodes 62, liquid crystal droplets within the holographic optical element are oriented such that the hologram is present in the element and light is diffracted from the zeroth order mode to a first (1st) order mode 70 of the hologram. When a voltage is applied to the holographic optical element the liquid crystal droplets become realigned effectively erasing the hologram, and the incoming light passes through the holographic optical element in the zeroth order mode 68.
It is to be understood that the holographic optical elements 42, 44, 46 may also be reflective rather than transmissive. In the case of a reflective holographic optical element, the arrangement of the holographic device and optical components would be modified to utilize reflective properties of the hologram rather than the transmissive properties described herein.
The light that passes through the hologram is diffracted to form an image by interference fringes recorded in the hologram. Depending on the recording, the hologram is able to perform various optical functions which are associated with traditional optical elements, such as lenses and prisms, as well as more sophisticated optical operations which would normally require very complex systems of conventional components. The hologram may be configured to perform operations such as deflection, focusing, or color filtering of the light, for example.
The hologram may be a Bragg (thick or volume) hologram or Raman- Nath (thin) hologram. Raman-Nath holograms are thinner and require less voltage to switch light between various modes of the hologram, however, Raman-Nath holograms are not as efficient as Bragg holograms. The Bragg holograms provide high diffraction efficiencies for incident beams with wavelengths close to the theoretical wavelength satisfying the Bragg diffraction condition and within a few degrees of the theoretical angle which also satisfies the Bragg diffraction condition.
The holograms are preferably recorded on a photopolymer/liquid crystal composite material (emulsion) such as a holographic photopolymeric film which has been combined with liquid crystal, for example. The presence of the liquid crystal allows the hologram to exhibit optical characteristics which are dependent on an applied electrical field. The photopolymeric film may be composed of a polymerizable monomer having dipentaerythritol hydroxypentacrylate, as described in PCT Publication, Application Serial No. PCT/US97/12577, by Sutherland et al., which is incorporated herein by reference. The liquid crystal may be suffused into the pores of the photopolymeric film and may include a surfactant.
The refractive properties of the holographic optical elements 42, 44, 46 depend primarily on the recorded holographic fringes in the photopolymeric film. The interference fringes may be created by applying beams of light to the photopolymeric film. Alternatively, the interference fringes may be artificially created by using highly accurate laser writing devices or other replication techniques, as is well known by those skilled in the art. The holographic fringes may be recorded in the photopolymeric film either prior to or after the photopolymeric film is combined with the liquid crystal. In the preferred embodiment, the photopolymeric material is combined with the liquid crystal prior to the recording. In this preferred embodiment, the liquid crystal and the polymer material are pre-mixed and the phase separation takes place during the recording of the hologram, such that the holographic fringes become populated with a high concentration of liquid crystal droplets. This process can be regarded as a "dry" process, which is advantageous in terms of mass production of the switchable holographic optical elements.
The electrodes (electrode layers) 62 are positioned on opposite sides of the emulsion and are preferably transparent so that they do not interfere with light passing through the hologram. The electrodes 62 may be formed from a vapor deposition of Indium Tin Oxide (ITO) which typically has a transmission efficiency of greater than 80%, or any other suitable substantially transparent conducting material. The transmission of the electrodes can be improved to greater than 97% by applying multi-layer anti-reflection coatings to the electrodes. The electrodes 62 are connected to an electric circuit 78 operable to apply a voltage to the electrodes, to generate an electric field (Fig. 3). Initially, with no voltage applied to the electrodes 62, the hologram is in the diffractive (active) state and the holographic optical element diffracts propagating light in a predefined manner. When an electrical field is generated in the hologram by applying a voltage to the electrodes 62 of the holographic optical element, the operating state of the hologram switches from the diffractive state to a passive (inactive) state and the holographic optical element does not optically alter the propagating light. It is to be understood that the electrodes may be different than described herein. For example, a plurality of smaller electrodes may be used rather than two large electrodes which substantially cover surfaces of the holograms.
The holographic optical elements 42, 44, 46 (or the holographic elements of the eyepiece 20 described below) may also be formed on curved substrates, as described in U.S. Patent Application Serial No. 09/416,076, by M. Popovich, filed October 12, 1999.
Each holographic optical element 42, 44, 46 of the holographic diffraction device 18 is holographically configured such that only a particular monochromatic light is diffracted by the hologram. The red optical element 42 has a hologram which is optimized to diffract red light, the green optical element 44 has a hologram which is optimized to diffract green light, and the blue optical element 46 has a hologram which is optimized to diffract blue light. A holographic device controller 90 drives switching circuitry 94 associated with the electrodes 62 on each of the optical elements 42, 44, 46 to a ply a voltage to the electrodes (Figs. 3 and 4). The electrodes 62 are individually coupled to the device controller through a voltage controller 102 which selectively provides an excitation signal to the electrodes 62 of a selected holographic optical element, switching the hologram to the passive state. The voltage controller 102 also determines the specific voltage level to be applied to each electrode 62.
Preferably, only one pair of the electrodes 62 associated with one of the three holographic optical elements 42, 44, 46 is energized at one time. In order to display a color image, the voltage controller 102 operates to sequentially display three monochromatic images of the color input image. The electrodes 62 attached to each of the holograms 42, 44, 46 are sequentially enabled such that a selected amount of red, green, and blue light is directed towards the viewer. For example, when a red monochromatic image is projected, the voltage controller 102 switches the green and blue holograms 44, 46 to the passive state by applying voltages to their respective electrodes 62. The supplied voltages to the electrodes 62 of the green and blue holograms 44, 46 create a potential difference between the electrodes, thereby generating an electrical field within the green and blue holograms. The presence of the generated electrical field switches the optical characteristic of the holograms 44, 46 to the passive state. With the green and blue holograms 44, 46 in the passive state and the red hologram 42 in the diffractive state, only the red hologram optically diffracts the projected red image. Thus, only the portion of the visible light spectrum corresponding to the red light is diffracted to the viewer. The green hologram 44 is next changed to the diffractive state by deenergizing the corresponding electrodes 62 and the electrodes of the red hologram 42 are energized to change the red hologram to the passive state so that only green light is diffracted. The blue hologram 46 is then changed to the diffractive state by deenergizing its electrodes 62 and the electrodes of the green hologram 44 are energized to change the green hologram to the passive state so that only blue light is diffracted.
The holograms are sequentially enabled with a refresh rate which is faster than the response time of a human eye so that a color image will be created in the viewer's eye due to the integration of the red, green, and blue monochrome images created from each of the red, green, and blue holograms. Consequently, the holographic devices will sequentially transmit red, green, and blue lights so that the final viewable image will appear to be displayed as a composite color. The red, green, and blue holographic elements 42, 44, 46 may be cycled on and off in any order.
The holographic diffraction 18 is preferably configured to compensate for chromatic dispersion introduced by the holographic diffraction elements of the eyepiece 20 and correct dispersion, chromatic, and geometric aberrations created due to the holographic diffraction elements of the eyepiece 20 operating off-axis and over large spectral bandwidths. More particularly, the characteristics of the holographic optical elements of the eyepiece 20 and the red, green, and blue holograms are preferably optimized so that the dispersion introduced by the holographic optical elements are substantially compensated for by the holograms of the holographic diffraction device 18.
The eyepiece 20 directs the image received from the holographic diffraction device 18 to the viewer. The eyepiece 20 preferably comprises a plurality of holographic optical elements 112, 114, 116, each of which is operable to act upon a respective one of the wavelength bands produced by the display device, as described above with respect to the elements 42, 44, 46 of the holographic diffraction device 18. The holographic elements 112, 114, 116 are switched rapidly in succession into and out of their active states by controller 120, which is coupled to the controller 90 of the holographic diffraction device 18. The controllers 90, 120 are preferably operated together so that the corresponding holographic elements of the holographic diffraction device 18 and eyepiece 20 (42 and 112) (44 and 114) (46 and 116) are switched generally simultaneously into their active and passive states. The holographic optical elements of either the diffraction device 18 or eyepiece 20 may also be nonswitchable (i.e., remain in active state at all times). The elements 112, 114, 116 of the eyepiece 20 are preferably reflective. The eyepiece 20 may also comprise conventional mirror devices, rather than holographic devices. In this case the optical devices are mounted on the front surface of the mirror (i.e., between the mirror and eye 28).
The eye tracking system 30 is used to track changes in the attitude of the viewer's eye 28 so that the image generated by the image generator 32 can be altered in response to the changes. The eye tracker system 30 uses light (e.g., infrared light) reflected by the eye 28 (e.g., on the cornea or further in the eye) to track the direction of gaze by the eye. The tracker system 30 includes a plurality of light emitters 120 positioned around an outer periphery of the eyepiece 20. The emitters 120 may be positioned, for example, every
20 degrees around the perimeter of the eyepiece 20. The emitters 120 are configured to project radiation in a broad wash onto the eye (as indicated by arrows A). Radiation reflected back from the eye (as indicated by arrows B) is directed by an optical device 124 (as indicated by aπows C) onto a detector 126 located at a position displaced laterally from the main optical axis of the display system.
The detector 126 may include, for example, a miniature two- dimensional detector array, crossed one dimensional detector arrays, or a peak intensity detection device (e.g., position sensing device). One example of a detector 126 which may be used with the system described herein is a two- dimensional position sensing detector manufactured by Ffamamatsu of Japan under designation S4744. The device has a spectral response from 760 to 1 lOOnm (peak response at 960 nm), photosensitivity of 0.58A/watt and rise time of 25 microseconds. It is to be understood that other types of detectors having different characteristics may also be used. Signals from the detector 126 are processed by a processor 130 to provide the image generator 32 data on changes in the attitude of the eye. The image generator 32 alters the image displayed by the display device 14 so that the image seen by the observer moves with the direction of gaze of the user. Additional optical elements such as focusing lenses and filters may be mounted adjacent to the detector 126 at a location between the optical device 124 and the detector, to facilitate the detection process.
The position of the eyeball 28 can be identified by tracking one of its visible features. The components of the eye tracker system 30 and the wavelength of the radiation used are preferably selected so that their characteristics can be optimized to allow particular features of the eye to be easily recognized and tracked. For example, the tracking system may be optimized to generate a sharp image of the perimeter of the pupil of the eye, with eye orientation being determined by measuring the shape of the pupil perimeter. It is to be understood that other features or properties of the eye may be used for tracking purposes, without departing from the scope of the invention. For example, the tracking system 30 may detect the peak corneal reflections which occur on the boundaries of the lens and cornea when infrared light is directed into the user's eye. The tracking system 30 may also optically detect and track the limbus (boundary between the white sclera and the dark iris of the eye) since the sclera is white and the iris is darker. Other eye tracking methods may be used as well known by those skilled in the art.
The optical device 124 is disposed directly behind the eyepiece 20 (i.e., optically on the opposite side of the eyepiece than the eye 28). Thus, radiation reflected back to the eye must pass through the eyepiece 20 before and after being acted upon by the optical device 124 (as indicated by arrows B and C). The system 10 is designed such that the eyepiece 20 has no substantial effect on this radiation. This allows the optical device 124 of the tracking system 30 to be positioned on a common optical axis with the eyepiece 20, thus providing a compact and lightweight arrangement.
The holographic optical elements 112, 114, 116 of the eyepiece 20 are designed to act upon relatively narrow wavelength bands of light in the red, green, and blue regions of the visible spectrum (e.g., 620 - 640 nm, 520 - 540 nm, 465 - 485 nm, respectively). The radiation used by the eye tracker system
30 is chosen to lie either outside or between these wavelength bands. For example, the eye tracker system 30 can utilize radiation in the near infrared band, which typically comprises wavelengths of 1000 nanometers or less. Alternatively, since the holographic diffraction elements each act on a wavelength bandwidth of around 20 nanometers, this leaves a substantial amount of the visible spectrum (which generally extends between around 400 nm - 465 nm, 485 nm - 520 nm, and 640 nm - 700 nm) unused and the eye tracking system 30 can utilize radiation having a wavelength band in one of the unused regions. It is to be understood that the term light as used herein includes visible and nonvisible light (e.g., red, green, and blue visible light and infrared radiation).
The optical device 124 may include a plurality of holographic diffraction elements configured to act on the wavelengths employed for the eye tracking function. For example, each element may be designed to image a particular area of the eye surface, thereby increasing accuracy in the determination of the eye gaze direction. In this case, multiple detectors 30 may be used, or a single detector may be time-multiplexed. Under these circumstances, it is advantageous for the holographic diffraction elements to be switchable. However, for other configurations, the holographic elements may be nonswitchable. The optical device 124 may be formed integrally with the eyepiece 20 or as a separate device. The optical device 124 may also be positioned on the opposite side of the eyepiece 20 (i.e., between the eyepiece and eye 28) in which case the optical device would be configured such that it does not act upon the wavelength band of light that is diffracted by the eyepiece 20.
The holographic elements of the diffraction device 18 are shown configured to be transmissive and the elements of the eyepiece 20 and optical device 124 are shown configured to be reflective. It is to be understood that the arrangement of components and reflective or transmissive configuration of the elements may be different than described herein without departing from the scope of the invention.
The transmission holograms described above are sensitive to the polarization state of incident light and exhibit maximum diffraction efficiency for p-polarized light, with the response to s-polarized light being around 1% for that of p-polarized light. In order to make use of the full output of the light source, the system 10 may include elements which make use of both the p- polarized light and s-polarized light, such as disclosed in U.S. Patent Application Serial Number 09/478,150, filed January 5, 2000, by M. Popovich et al, which is incorporated herein by reference in its entirety. For example, pairs of holographic diffraction elements may be used with one element in the pair acting on the p-polarized component and the other acting on the s- polarized components. This may be achieved either by interposing a polarization rotator between the elements in the pair or by arranging for the interference fringes in the elements of each pair to be mutually crossed. If reflection holograms are used, these additional provisions are not required since reflection holograms only start to become polarization sensitive at large angles of incidence, typically much greater than 45 degrees.
Although the present invention has been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations made to the embodiments without departing from the scope of the present invention. Accordingly, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.

Claims

CLAIMSWHAT IS CLAIMED IS:
1. A display system comprising:
a display device operable to display an image;
a holographic diffraction device operable to direct said image towards a user of the system; and
an eye tracking system operable to track movement of an eye of the user of the system, the tracking system comprising an optical device operable to receive light reflected from the user's eye and project said light towards a detector to detect movement of a user' s eye based on said reflected light, the detector coupled to an image generator device operable to modify said image in response to movement detected by the eye tracking system;
wherein the optical device is positioned such that light received and projected by the optical device passes through the holographic diffraction device and the holographic diffraction device is configured to diffract light having a wavelength band different than the light received and projected by the optical device so that the light received by the optical device passes through the diffraction device without substantial alteration.
2. The system of claim 1 wherein the optical device is configured to act upon infrared radiation and wherein the holographic diffraction device is configured to act upon visible wavelength light.
3. The system of claim 1 wherein the optical device and holographic diffraction device are configured to act upon visible wavelength light with different wavelength bands.
4. The system of claim 1 wherein the holographic diffraction device and the optical device are positioned along a common optical axis.
5. The system of claim 1 wherein the eye tracker system further includes a light emitter operable to project light onto the user's eye.
6. The system of claim 5 wherein the light emitter comprises a plurality of emitters positioned around the periphery of the holographic diffraction device.
7. The system of claim 1 wherein the eye tracker system further includes a detector operable to detect light after reflection by the user's eye.
8. The system of claim 7 wherein the detector is positioned laterally from an optical axis of the holographic diffraction device.
9. The system of claim 7 further comprising a processor operable to process signals received from the detector and provide data corresponding to changes in attitude of the user's eye to the image generator coupled to the display device.
10. The system of claim 1 wherein the display system is configured for use as a head mounted display system.
11. The system of claim 1 wherein the display device is a liquid crystal display device.
12. The system of claim 1 wherein the display device is operable to display a plurality of monochromatic images.
13. The system of claim 1 wherein the holographic diffraction device comprises a plurality of holographic optical elements switchable between an active state wherein light incident on the element is diffracted and a passive state wherein light incident on the element is transmitted without substantial alteration.
14. The system of claim 13 wherein each of the holographic optical elements is configured to diffract light of a different wavelength band.
15. The system of claim 13 wherein each of the holographic optical elements comprises a hologram interposed between two electrode layers operable to apply an electrical field to the hologram.
16. The system of claim 15 wherein the hologram is formed from a polymer and liquid crystal material.
17. The system of claim 13 wherein said plurality of holographic optical elements comprises three holographic optical elements.
18. The system of claim 17 wherein the three holographic optical elements each have a hologram recorded therein which is optimized to diffract red, green, or blue light.
19. The system of claim 18 wherein each of the holograms is interposed between two electrode layers operable to apply an electrical field to the hologram to diffract the red, green, or blue light.
20. The system of claim 19 further comprising a controller operable to sequentially supply voltage to and remove voltage from the electrode layers of each of the holographic optical elements to create a sequence of monochrome images which are combined to form a color image.
21. The system of claim 1 further comprising an optical relay system operable to project said image received from the display device.
22. The system of claim 1 further comprising a second holographic diffraction device positioned between the display device and the first holographic diffraction device.
23. The system of claim 22 wherein the second holographic diffraction device comprises a plurality of holographic optical elements switchable between an active state wherein light incident on the element is diffracted and a passive state wherein light incident on the element is transmitted without substantial alteration.
24. The system of claim 23 wherein each of the holographic optical elements is configured to diffract light of a different wavelength band.
25. The system of claim 22 wherein the second holographic diffraction device is configured to substantially compensate for chromatic dispersion created by the first holographic diffraction device.
26. A display system comprising:
a display device operable to display an image;
an optical system operable to receive said image from the display device and project said image to a user of the display system, the optical system comprising at least one holographic optical element configured to diffract light having a first wavelength band; and
an eye tracking system operable to track movement of an eye of the user, the eye tracking system comprising at least one optical device configured to act upon light having a second wavelength band different from said first wavelength band;
wherein the holographic optical element is positioned adjacent to the optical device of the tracking system and on a common optical path therewith.
27. The system of claim 26 wherein the optical device is configured to act upon infrared radiation and wherein the holographic diffraction device is configured to act upon visible wavelength light.
28. The system of claim 26 wherein the optical device and holographic diffraction device are configured to act upon visible wavelength light with different wavelength bands.
29. The system of claim 26 wherein the eye tracker system further includes a light emitter operable to project light onto the user's eye.
30. The system of claim 29 wherein the light emitter comprises a plurality of emitters positioned around the periphery of the holographic diffraction device.
31. The system of claim 26 wherein the eye tracker system further includes a detector operable to detect light after reflection from the user's eye.
32. The system of claim 31 wherein the detector is positioned laterally from an optical axis of the holographic diffraction device.
33. The system of claim 31 further comprising a processor operable to process signals received from the detector and provide data corresponding to changes in attitude of the user's eye to the image generator coupled to the display device.
34. The system of claim 26 wherein the display device is operable to display a plurality of monochromatic images.
35. The system of claim 26 wherein the optical system comprises a plurality of holographic optical elements switchable between an active state wherein light incident on the element is diffracted and a passive state wherein light incident on the element is transmitted without substantial alteration.
36. The system of claim 35 wherein each of the holographic optical elements is configured to diffract light of a different wavelength band.
37. The system of claim 36 wherein each of the holographic optical elements comprises a hologram interposed between two electrode layers operable to apply an electrical field to the hologram.
38. The system of claim 35 wherein said plurality of holographic optical elements comprises three holographic optical elements.
39. The system of claim 38 wherein the three holographic optical elements each have a hologram recorded therein which is optimized to diffract red, green, or blue light.
40. The system of claim 39 wherein each of the holograms is interposed between two electrode layers operable to apply an electrical field to the hologram to diffract the red, green, or blue light.
41. The system of claim 40 further comprising a controller operable to sequentially supply voltage to and remove voltage from the electrode layers of each of the holographic optical elements to create a sequence of monochrome images which are combined to form a color image.
42. The system of claim 26 wherein the eye tracking system further includes an emitter operable to emit light onto the user's eye and wherein the optical device is positioned to receive the light reflected from the user's eye.
43. The system of claim 42 wherein the eye tracking system further includes a detector positioned to receive light reflected from the optical device and detect a change in attitude of the user's eye based on the light reflected therefrom.
PCT/US2000/021454 1999-08-03 2000-08-03 Display system with eye tracking WO2001009685A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU66232/00A AU6623200A (en) 1999-08-03 2000-08-03 Display system with eye tracking

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14698099P 1999-08-03 1999-08-03
US60/146,980 1999-08-03

Publications (1)

Publication Number Publication Date
WO2001009685A1 true WO2001009685A1 (en) 2001-02-08

Family

ID=22519851

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/021454 WO2001009685A1 (en) 1999-08-03 2000-08-03 Display system with eye tracking

Country Status (2)

Country Link
AU (1) AU6623200A (en)
WO (1) WO2001009685A1 (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007083215A2 (en) * 2006-01-17 2007-07-26 Ferrari S.P.A. Method of controlling a road vehicle hud system
FR2959022A1 (en) * 2010-04-16 2011-10-21 Thales Sa OPTICAL COMBINER VISUALIZATION DEVICE CORRECTING CHROMATIC ABERRATIONS
WO2011015843A3 (en) * 2009-08-07 2012-04-12 Light Blue Optics Ltd Head up displays
WO2017005614A1 (en) * 2015-07-03 2017-01-12 Essilor International (Compagnie Generale D'optique) Methods and systems for augmented reality
WO2017005608A1 (en) * 2015-07-03 2017-01-12 Essilor International (Compagnie Generale D'optique) Methods and systems for augmented reality
WO2017069946A1 (en) * 2015-10-23 2017-04-27 Microsoft Technology Licensing, Llc Holographic display
US9658453B1 (en) 2013-04-29 2017-05-23 Google Inc. Head-mounted display including diffractive combiner to integrate a display and a sensor
CN107783297A (en) * 2017-11-22 2018-03-09 深圳创维新世界科技有限公司 Nearly eye display device
US10394034B2 (en) 2017-08-15 2019-08-27 Microsoft Technology Licensing, Llc Eye-tracking with MEMS scanning and optical relay
WO2020112527A1 (en) * 2018-11-28 2020-06-04 Facebook Technologies, Llc Polarization-sensitive components in optical systems for large pupil acceptance angles
CN111708163A (en) * 2020-04-26 2020-09-25 北京理工大学 Head-up display system and method with eye movement tracking function
US11175508B2 (en) 2018-03-15 2021-11-16 Facebook Technologies, Llc Display device with varifocal optical assembly
US11327306B2 (en) 2018-03-15 2022-05-10 Facebook Technologies, Llc Angular performance of apochromatic pancharatnam berry phase components using a C-plate
US11846779B2 (en) 2018-03-15 2023-12-19 Meta Platforms Technologies, Llc Display device with varifocal optical assembly

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3635538A (en) * 1970-02-27 1972-01-18 Sperry Rand Corp Stacked hologram apparatus
US5453854A (en) * 1993-04-09 1995-09-26 Sextant Avionique Device to check the position and orientation of a mirror
US5734357A (en) * 1994-12-02 1998-03-31 Fujitsu Limited Vehicular display device adjusting to driver's positions
US5799231A (en) * 1996-07-25 1998-08-25 International Business Machines Corporation Variable index distributed mirror

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3635538A (en) * 1970-02-27 1972-01-18 Sperry Rand Corp Stacked hologram apparatus
US5453854A (en) * 1993-04-09 1995-09-26 Sextant Avionique Device to check the position and orientation of a mirror
US5734357A (en) * 1994-12-02 1998-03-31 Fujitsu Limited Vehicular display device adjusting to driver's positions
US5799231A (en) * 1996-07-25 1998-08-25 International Business Machines Corporation Variable index distributed mirror

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007083215A3 (en) * 2006-01-17 2007-12-13 Ferrari Spa Method of controlling a road vehicle hud system
WO2007083215A2 (en) * 2006-01-17 2007-07-26 Ferrari S.P.A. Method of controlling a road vehicle hud system
WO2011015843A3 (en) * 2009-08-07 2012-04-12 Light Blue Optics Ltd Head up displays
FR2959022A1 (en) * 2010-04-16 2011-10-21 Thales Sa OPTICAL COMBINER VISUALIZATION DEVICE CORRECTING CHROMATIC ABERRATIONS
US9658453B1 (en) 2013-04-29 2017-05-23 Google Inc. Head-mounted display including diffractive combiner to integrate a display and a sensor
JP2018528452A (en) * 2015-07-03 2018-09-27 エシロール アンテルナショナルEssilor International Methods and systems for augmented reality
US10852541B2 (en) 2015-07-03 2020-12-01 Essilor International Methods and systems for augmented reality
WO2017005608A1 (en) * 2015-07-03 2017-01-12 Essilor International (Compagnie Generale D'optique) Methods and systems for augmented reality
KR102593113B1 (en) 2015-07-03 2023-10-25 에씰로 앙터나시오날 Methods and systems for augmented reality
CN107924057A (en) * 2015-07-03 2018-04-17 依视路国际公司 Method for augmented reality and system
KR20180044238A (en) * 2015-07-03 2018-05-02 에씰로 앙터나시오날 Methods and systems for augmented reality
US11762199B2 (en) 2015-07-03 2023-09-19 Essilor International Methods and systems for augmented reality
JP7074478B2 (en) 2015-07-03 2022-05-24 エシロール アンテルナショナル Methods and systems for augmented reality
JP2018525662A (en) * 2015-07-03 2018-09-06 エシロール アンテルナショナルEssilor International Methods and systems for augmented reality
WO2017005614A1 (en) * 2015-07-03 2017-01-12 Essilor International (Compagnie Generale D'optique) Methods and systems for augmented reality
CN107924057B (en) * 2015-07-03 2020-12-22 依视路国际公司 Method and system for augmented reality
US10416456B2 (en) 2015-07-03 2019-09-17 Essilor International Methods and systems for augmented reality
CN108351614A (en) * 2015-10-23 2018-07-31 微软技术许可有限责任公司 Holographic display device
US10067346B2 (en) 2015-10-23 2018-09-04 Microsoft Technology Licensing, Llc Holographic display
WO2017069946A1 (en) * 2015-10-23 2017-04-27 Microsoft Technology Licensing, Llc Holographic display
US10394034B2 (en) 2017-08-15 2019-08-27 Microsoft Technology Licensing, Llc Eye-tracking with MEMS scanning and optical relay
CN107783297A (en) * 2017-11-22 2018-03-09 深圳创维新世界科技有限公司 Nearly eye display device
US11175507B2 (en) 2018-03-15 2021-11-16 Facebook Technologies, Llc Polarization-sensitive components in optical systems for large pupil acceptance angles
US11175508B2 (en) 2018-03-15 2021-11-16 Facebook Technologies, Llc Display device with varifocal optical assembly
US11327306B2 (en) 2018-03-15 2022-05-10 Facebook Technologies, Llc Angular performance of apochromatic pancharatnam berry phase components using a C-plate
US11846779B2 (en) 2018-03-15 2023-12-19 Meta Platforms Technologies, Llc Display device with varifocal optical assembly
WO2020112527A1 (en) * 2018-11-28 2020-06-04 Facebook Technologies, Llc Polarization-sensitive components in optical systems for large pupil acceptance angles
CN111708163A (en) * 2020-04-26 2020-09-25 北京理工大学 Head-up display system and method with eye movement tracking function
CN111708163B (en) * 2020-04-26 2022-06-03 北京理工大学 Head-up display system and method with eye movement tracking function

Also Published As

Publication number Publication date
AU6623200A (en) 2001-02-19

Similar Documents

Publication Publication Date Title
US6377369B1 (en) Holgraphic display with switchable aspect ratio
US20010024177A1 (en) Holographic display system
US6185016B1 (en) System for generating an image
US6082862A (en) Image tiling technique based on electrically switchable holograms
JP6320451B2 (en) Display device
US6124954A (en) Projection screen based on reconfigurable holographic optics for implementation in head-mounted displays
US6646636B1 (en) Display system utilizing ambient light and a dedicated light source
JP2022091982A (en) Waveguide with uniform output illumination
US6295145B1 (en) Switchable holographic device
US20200301239A1 (en) Varifocal display with fixed-focus lens
US6373603B1 (en) Image generating system including compensation for chromatic dispersion
TW201300834A (en) Display device, in particular a head-mounted display
WO2001009685A1 (en) Display system with eye tracking
US6175431B1 (en) Projection systems based on reconfigurable holographic optics
US6040928A (en) Holographic desktop monitor
WO2000023831A1 (en) Light diffusion control by electrically reconfigurable holographic optical elements
US6301027B1 (en) Holographic desktop monitor
JP4443046B2 (en) Projection system based on reconfigurable holographic optics
WO2000030066A1 (en) Holographic display system
WO2000043841A1 (en) System for generating images using switchable holographic optical elements
WO2000043841A9 (en) System for generating images using switchable holographic optical elements
WO2000068715A1 (en) Image generating system
JP2024054106A (en) Display device

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AL AM AT AU AZ BA BB BG BR BY CA CH CN CR CU CZ DE DK DM EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP