应用于3D全息成像

Real-time color electroholography using multiplegraphics processing units and multiplehigh-definition liquid-crystal display panels Hirotaka Nakayama，1* Naoki Takada， Yasuyuki Ichihashi，Shin Awazu，Tomoyoshi Shimobaba， Nobuyuki Masuda， and Tomoyoshi Ito' 1Graduate School of Engineering， Chiba University， 1-33 Yayoi-cho， Inage-ku， Chiba City， Chiba 263-8522， Japan"Department of Information Media， Shohoku University， 428， Nurumizu， Atsugi City， Kanagawa 243-8501， Japan *Corresponding author： h-nakayama@graduate.chiba-u.jp Received 9 July 2010； accepted 14 September 2010； posted 24 September 2010 (Doc. ID 131349)； published 22 October 2010 We succeeded in developing a prototype three-dimensional (3-D) color television system by holographyusing three graphics processing units and three liquid-crystal display panels for displaying three primarycolored images. A 3-D animation system featuring holography is said to have potential for ultimate 3-Dtelevision. Because of the technical complexity involved， however， the practical use of 3-D television isdifficult. In our system， the image processing speed for holography is more than 300 times faster thanthat of today's personal computers， and a 3-D image composed of 1000 points can be animated at virtuallyvideo rate， although the image size is as small as 5 cm. The system succeeded in producing realistic 3-Dcolor animation. @ 2010 Optical Society of America OCIS codes： 090.1760，090.2870，090.5694. 1. Introduction Holography is the ultimate three-dimensional (3-D)vision technology because it is the only techniquecapable of rebuilding complete 3-D structures. Aswell as the optical method， we can also make holo-grams by computing， called computer-generatedholograms (CGHs) [1]. Animation is possible bytreating holography electronically with CGH tech-nology， called electroholography， which has been stu-died from about 1990 [2，3]. A real-time 3-D displaysystem featuring electroholography is thought to re-present the ultimate 3-D television [4，5]. However，due to the technical complexity involved， the practi-cal use of 3-D television is difficult. The biggest problem is accelerating the CGH cal-culation [6]. Holography records a 3-D image throughlight interference and reconstructs the same throughlight diffraction， and， hence， requires a display de- ( 0003-6935/10/315993-04$15.00/0 ) ( 2010 Optical Society of America ) vice with a large area and a narrow pixel intervalto achieve a sufficient image and viewing angle.The pixel interval must also be smaller than 1pm，which is comparable with the wavelength of visiblelight. Today， miniaturization of the pixel intervalof a liquid-crystal display (LCD) has advanced toaround 5 um，while in the future， the finest value willbe around 1um [7]. However， the number of pixelsbecomes 1 trillion (1012) when the display size is1mx1m， and no current computing system can han-dle such enormous calculations. In fact， we cannotcalculate in real time even to 1920×1080 CGHs.which is equivalent to the current video standard.However， researchers have been striving to improvecomputing speed by using graphics pro 111cessing units(GPUs)， which have undergone rapid development inrecent years [8-11]. CGH calculations are suitablefor parallel processing， while a GPU can operate withthe same efficiency via built-in stream processors. In this study， we succeeded in developing an inter-active color electroholography system using threeGPU chips， which calculate three CGHs using the colors red， green， and blue (RGB). Moreover， eachGPU board is connected to each LCD panel. TheGPU chips generate CGHs at high speed and sendthem to LCD panels directly without returning thedata to the central processing unit (CPU) in the hostpersonal computer (PC). Therefore， the data commu-nication is strictly one way without bottlenecks.Thanks to the GPU computing and the unencum-bered communication system， the performance ofour electroholography system is more than 300 timessuperior to that of a PC alone. Consequently， werealize real-time electroholography for a 3-D imageconstructed with around 1000 colored points. In Section 2， we describe the real-time color elec-troholography systemthat we developed usingmultiple GPUs and multiple LCDs. In Section 3，we exhibit the reconstructed images achieved withthis system. The final section is our conclusion anddiscussion. 2. Color Electroholography System with MultipleGPUs and Multiple LCDs A. Optical System Figure 1 shows a schematic illustration of our colorelectroholography system. The optical system con-sists of three transmissive LCDs and three colorlasers. These LCDs connect to three GPU-equippedvideo cards. The laser beams are converted into par-allel lights by the objective and collimator lenses.The parallel lights are incident on the transmissiveLCDs. The GPUs compute three CGHs for RGB inparallel， and each CGH is displayed directly onthe corresponding LCD， whereupon each image fromthe CGH is reconstructed by the parallel light. Thesereconstructed images are then coaxially overlappedby beam splitters (BS)， thus allowing us to observethe full-color image. The LCD， which we extractedfrom a liquid-crystal projector (Epson Inc. EMP-TW1000) is the L3C07U series， with specificationsas follows： a pixel interval of 8.5 um， resolution of1920×1080，and size of 16 mmx9mm. We use threelasers for the RGB colors as reference lights， withspecifications for wavelength and output power，respectively， asfollows：： red1Jlaser， 633 nm and15 mW； green laser， 532 nm and 20mW； and blue Fig. 1. (Color online) Color electroholography system with multi-ple GPUs and multiple LCDs. laser， 473 nm and 100 mW. The variation in the out-put of the lasers was regulated by neutral density(ND) filters. Several color reconstruction techniques have beenproposed as follows： (1) a method using three holo-gram display panels and three reference lights(RGB) [12-14]；(2) a time division switching methodusing one hologram display panel and three refer-ence lights (RGB)， which are changed sequentiallyby an electronic shutter [15]， and (3) a method usingone hologram display panel and three referencelights (RGB) without a control unit such as an elec-tronic shutter [16]. In this study， we used method (1)，which is the simplest .B. Calculation System Three GPU-equipped video cards are setup on thePC. The GPU we used in this study was an NVIDIAGeForce GTX 285， with the following specifications：the number of streaming processors was 240 and theclock frequency of the processor was 1476 MHz. Forthe computational algorithm of a CGH with a GPU，we use the following formula： where parameters x，y， and z represent the horizon-tal， vertical， and depth components， while indices aand j indicate the hologram and object， respectively.N is the number of points that compose a 3-D object，A； is the intensity of the object point， and A is thewavelength of the reference light. Equation (1) isthe Fresnel approximate calculation[1]. The above equation requires N×M times the cal-culations， where M is the total number of pixels on aCGH (LCD)； hence， the calculation amount becomesenormous. However， because Eq.(1) shows that eachpixel on a CGH can be calculated independently， com-puter-generated holography is suitable for a parallelcalculation. Therefore， first， the CPU on the PCdivides an original color object data into R， G， andB data， and transmits them to each GPU with a mes-sage passing interface (MPI). Next， the streamingprocessors on each GPU calculate the CGH by Eq. (1)in parallel. The synchronization of each GPU is per-formed by the MPI. Finally， each CGH is output toeach LCD. Our system can display an animated 3-D objectconsisting of around 1000 pointsper color inreal time. Therefore， the data volume to be handledby a CPUisapproximately 1000×3(colors)=3000 points， with a similar quantity to be sent tothree GPU chips from a CPU. Conversely， theamount of CGH calculations per GPU is proportionalto the product of the total numbers of object pointsand pixels on the CGH. Therefore， the amount of pro-cesses by the CPU can be ignored compared with thatof CGH calculation by the GPU. C. Performance The specifications of the CPU that we used to com-pare the calculation times are as follows： Intel Corei7 920 (2.66GHz) (we used all cores for the calcula-tion)， 3.0 Gbytes of memory， Linux operating system(Fedora 10， kernel-2.6.27)， and gcc4.3.2. We used the Table 1..Acceleration Achieved by Our GPU System Calculation Times (ms) Object CPU Three GPU Acceleration Points only chips Ratio 512 7119 27.03 263 1024 14158 45.27 312 2048 28804 81.82 352 4096 56621 155.2 364 Color olo Colori Fig. 2..((Color online) Reconstructed single-color images (red，green， and blue). Fig. 3.((Color online) Colors by the additive color process： (a) ob-ject point data image and (b) reconstructed image (Media 1). Fig.4.(((Color online) Realistic color reconstructeddimage(Media 2). Fig. 5. (Color online) Reconstructed images made from full-coloranimations： (a) dancing “Degidori” on finger (Media 3) and(b)jumping “Nanami"(Media 4). CUDA programming environment (NVIDIA) for theGPU calculations. The CUDA SDK version is 2.3. Weused the MPICH2 as an MPI library to synchronizeeach GPU. Three LCDs with resolution of 1920×1080 operatesimultaneously， hence， we measured the time takento calculate three CGHs with resolution of 1920x1080×3 while changing the number of object points.Table 1 shows the performance， demonstrating thatour system is around 350 times faster than a singleCPU. When the number of points is about 1000 percolor， real-time reconstruction is possible at 22frames per second (fps). Even for 2，000 points percolor， it is 12 fps， virtually real time. 3..FReconstruction of a Real-Time Color Hologram Figure 2 shows reconstructed single-color images(red， green， and blue). Figure 3 is the overlapped im-age of RGB， and Figs. 3(a) and 3(b) are the originaland reconstructed CGimages，respectively. This is ananimation scene whereby the color phase of the char-acter “Color”is changing and the color circle is rotat-ing. We can recognize full color by additive and colormixing. Next， we tried to reconstruct a more artistic imageusing this system， with the results shown in Fig.4. Inthis animation， the motif is a fish swimming， wherethe red presents the skin with 1497 points， the greenpresents the skeleton with 1498 points， and the bluepresents the bowels with 710 points， respectively.Figure 4 is one scene of the overlapped RGB anima-tion. Within the limited number of object pointsavailable， we succeeded in expressing an artisticimage. Finally， we demonstrated color electroholographyvia the procedure shown in Fig. 5.Namely， any color3-D animation is divided into RGB components andreconstructed in the air by our system. Figure 5(a) is“Degidori” and Fig. 5(b) is"Nanami，" both of whichare Japan Broadcasting Corp. (NHK) characters.As well as their smooth movement， we can clearly ob-serve the mixed color“yellow”in the beak in Fig. 5(a)and“white” in Fig. 5(b). 4l.. Conclusion In this study， we developed a color electroholographysystem toward a prototype of 3-D television fromperspectives with color， unidirectional data flow， and video rate. We can reconstruct a realistic colorimage at near video rate (20-30 fps) when the num-ber of points is less than around 1000 for each RGB.Study ofelectroholography is ongoing with the aim ofmaking 3-D television a reality. With technical pro-gress， the image quality also becomes important.Currently， the field of view angle has only several de-grees and the size of the reconstruction image is sev-eral cubic centimeters. Practical use is seeminglystill some way off. Under these circumstances， thisstudy is expected to help advance research， and itis hoped that more artistic expression will be enabledin the future by accelerating the CGH calculationspeed and increasing the number of points. We thank NHK for the use of the characters. Thisresearch was partially supported by the Japan So-ciety for the Promotion of Science (JSPS)， a Grant-in-Aid for Scientific Research (C) (21500094)， andthe Ministry of Internal Affairs and Communica-tions， Strategic Information and CommunicationsR&D Promotion Program (SCOPE)， 2009. References 1. T. C. Poon， ed.， Digital Holography and Three-DimensionalDisplay (Springer， 2006)，Chap. 8， pp. 235-255. 2. P. S. Hilaire， S. A. Benton， M. Lucente， M. L. Jepsen， J. Kollin，H. Yoshikawa， and J. Underkoffler，“Electronic display systemfor computational holography，”Proc. SPIE 1212， 174-182(1990). 3. N. Hashimoto， S. Morokawa， and K. Kitamura，“Real-time ho-lography using the high-resolution LCTV-SLM，”Proc. SPIE1461，291-302(1991). 4. C. W. Slinger， C. D. Cameron，S. D. Coomber， R. J. Miller，D. A.Payne， A. P. Smith， M. G. Smith，M. Stanley， and P. J. Watson，“Recent developments in computer-generated holography： to-ward a practical electroholography system for interactive 3Dvisualization，”Proc. SPIE 5290， 27-41 (2004). 5. E. Zschau， R. Missbach， A. Schwerdtner， and H. Stolle"Generation， encoding， and presentation ofcontent on holo- graphic displays in real time” Proc. SPIE 7690，76900E(2010). 6. M. Lucente and T. A. Galyean，“Rendering interactive holo-graphic images，” in Proceedings of SIGGRAPH 1995 (ACMPress， 1995)， pp. 387-394. 7. S. Tay， P. A. Blanche， R. Voorakaranam， A. V. Tunc， W. Lin，S. Rokutanda， T. Gu， D. Flores， P. Wang， G. Li， P.S. Hilaire， J.Thomas，R. A. Norwood，M. Yamamoto，andN.Peyghambarian，“An updatable holographic three-dimensional display，"Nature451，694-698(2008). 8. N.Masuda， T. Ito， T. Tanaka， A. Shiraki， and T. Sugie，“Com-puter generated holography using a graphics processing unit，"Opt. Express 14， 603-608(2006). 9. A. Shiraki，N. Takada，M.Niwa， Y. Ichihashi， T. Shimobaba，N.Masuda，and T. Ito，“Simplified electroholographic color recon-struction system using graphics processing unit and liquidcrystal display projector，” Opt. Express 17， 16038-16045(2009). 10. H. Kang， F. Yaras， and L. Onural，“Graphics processing unitaccelerated computation of digital holograms，”Appl.Opt. 48，H137-H143(2009). 11. L. Ahrenberg， A. J. Page， B. M. Hennelly， J. B. McDonald， andT. J. Naughton，“Using commodity graphics hardware forreal-time digital hologram view-reconstruction，”J. DisplayTechnol. 5， 111-119 (2009). 12. K.Sato，“Animated color 3D image display using kinoforms byliquid crystal devices，”J. Inst. Telev. Eng. Jpn. 48， 1261-1266(1994)[in Japanese]. 13. Y. Sando， M. Itoh， and T. Yatagai，“Full-color computer-generated holograms using 3-D Fourier spectra，”Opt. Express12，6246-6251(2004). 14. F. Yaras， H. Kang， and L. Onural，“Real-time phase-only colorholographic video display system using LED illumination，"Appl. Opt. 48， H48-H53 (2009). 15. T. Shimobaba， A. Shiraki， N. Masuda， and T. Ito， “An electro-holographic colour reconstruction by time division switchingof reference lights，”J. Opt. A Pure Appl. Opt. 9， 757-760(2007). 16. T.Ito and K. Okano， “Color electroholography by three coloredreference lights simultaneously incident upon one hologrampanel，”Opt. Express 12， 4320-4325(2004). ovember / Vol. No. APPLIED OPTICS PPLIED OPTICS /Vol. No. / November