I. INTRODUCTION
Recently, the commercial success of 3D contents leads to the rise of 3D related markets for various 3D devices, such as 3DTV, 3DVR and 3D cameras [1-6]. Also, as 3D cameras become cheaper and smaller, it is possible for users to buy a dual lens stereoscopic camera and make 3D video contents easily. The dual lens stereoscopic camera is very convenient for taking 3D videos without post image processing and provides various functions to affect the videos [6]. Since one of the functions which non-professionals, such as home users, usually exploit in the classic handy cam is a zoom function, it is necessary for the users to exploit the zoom function easily in the dual lens stereoscopic camera.
However, if the users zoom the object in the same way they deal with the classic handy cam, it can give viewers dizziness and visual discomfort coming from watching 3D videos. The recent commercial dual lens stereoscopic camera has the zoom function precisely synchronized on two lenses and can adjust the convergence point manually. The convergence point is an intersection point of two lenses for stereopsis effects. Since the appropriate adaptation of the convergence point requires professional knowledge, it is difficult for non-professionals to adjust the convergence point while shooting. To avoid the difficulty, the commercial dual lens stereoscopic cameras fix the distance of the convergence point while the users zoom the object. Since the object is magnified in high ratio with the convergence point fixed during the zoom operation, it may lead to the excessive binocular disparity of the taken videos.
In [1-5], the binocular disparity is defined as the difference of images seen by each eye. Reference [3] recommends the 1° of the binocular disparity for visual comfort. In [6], they present the parallax setting of the recent commercial dual lens stereoscopic cameras for the zoom. References [7, 8] suggest the methods to adjust the depth of the 3D images. To adjust the convergence, they measure the disparity value of images shot by a camera using image processing. In [9-11], they conducted the evaluation to find out the factors affecting the visual comfort of 3D videos.
In this paper, to ease visual discomfort in a zoom operation on a dual lens stereoscopic camera, we build a relational model using the locations of a focus, an object and a convertgence point. In the model, nine possible cases are derived by the position between the convergence point and the focus. To deal with nine cases, we propose the four methods adjusting the convergence point by the distance of movements caused by the zoom. We also evaluate the disparity measurements and the subjective test on one hundred eight 3D video clips shot by the proposed methods in order to investigate the visual discomfort. The results show that one of the proposed methods outperforms the others.
The composition of this paper is as follows. Chapter 2 covers the current research trends. The excessive disparity caused by a zoom is presented in Chapter 3. In Chapter 4, we propose a relational model between a focus and a convergence point, and four methods to adjust a convergence point in order to prevent visual discomfort. In Chapter 5, the results of the disparity measurements and the subjective evaluation are shown and the performances of proposed methods are compared. Finally, conclusion and future research are suggested in Chapter 6
II. RELATED WORKS
In [1-5], the binocular disparity was defined as the difference in an image location of an object seen by the left and the right eyes. The binocular disparity determines depth perception. People get dizzy and feel visual discomfort with increased depth perception. This visual discomfort occurs because the main view point and accommodation do not correspond. In [3], 1° of binocular disparity was recommended as the condition for the visual comfort zone to content producers. But, further research is required since there is no analysis for parallax value when an image changes because of altered accommodation, as in zoom.
Reference [6] presented the parallax setting of recent commercial cameras providing the zoom function. The Sony camera provides the depth control using a knob or a fixed distance for the convergence point. The JVC camera also presents the parallax adjustment using 10 feet from the camera or the fixed distance from the specific point. However, both cameras recommend the professional adjustments of the convergence point because it may suffer from the visual discomfort or the depth tolerance.
References [7-8] proposed the methods to adjust the depth of the 3D images. Reference [7] suggested a stereo auto convergence algorithm to improve the 3D viewing comfort. Reference [8] also presented the method extracting the disparity information from the image and adjusting the vergence using the extracted data. However, both researches require additional costs for the image processing measuring the binocular disparity.
In [9-11], they conducted the evaluation to find out the factors affecting the visual comfort of 3D videos. Reference [9] evaluated the subjective tests of the visual discomfort by adjusting the convergence distance. The results showed that the faster the convergence changes, the higher visual discomfort the viewers will get. In [10], they showed that negative conflicts are less comfortable at far distance but the positive are less comfortable at near distance. In [11], visual tiredness was measured for each camera movement (pan, tilt, roll, and zoom) which causes visually-induced motion sickness. Thereby, a correlation has been suggested between motion speed and visual discomfort. But the experiment is based on a single lens image, which makes it hard to infer direct impact of motion in a dual lens image. It is far more difficult to figure out the impact in zooming which is greatly influenced by a convergence point.
References [12-14] presented the methods to extract data and correct errors of the images resulting from a zoom. In [12], they showed the gradient based as well as the feature-based methods recovering the depth on one camera with a zoom lens. Reference [13] proposed the method to estimate the distance information on the stereoscopic display using the stereo zoom lens module in the wide range. In [14], they also presented an optical property correction technique on a low cost heterogeneous stereoscopic camera especially for a zoom and a depth. However, this previous research does not deal with a binocular disparity caused by a zoom.
III. CONVERGENCE POINT IN ZOOM
The convergence point is the point where the views of two lenses converge in toed-in configuration in a dual lens stereoscopic camera and is defined as C as shown in Fig. 1. If the distance of C from a camera is shorter than the distance of an object, then it is the positive parallax. If the distance of C is longer than that of the object, it is the negative parallax. Since the adjustment of the convergence point requires professional knowledge and techniques, the convergence point is not coupled with the zoom function and is fixed while zooming the object in a conventional dual lens stereoscopic camera. However, if the user exploits the zoom function with the convergence point fixed, the object which the camera is focusing on gets enlarged in high magnification ratio on a screen, resulting in an excessive binocular disparity.
FIG. 1.The binocular disparity when zooming in the object.
For example, suppose that ZO is an object the camera is shooting and trying to zoom in. When the user zooms in ZO, ZO is magnified by the zoom ratio. It is similar that ZO moves to the camera after zoom-in by the rate as shown in Fig. 1. Let us assume that the camera always focuses on the object which the camera is shooting using an auto focus function. Fb denotes the focus point focusing ZO before zoom-in and Fa is the focus point focusing ZO after zoom-in. If ZO is zoomed in, Fb moves to Fa following the magnification of ZO as shown in Fig. 1. Since the distance of C, d(C), is fixed where d() is the distance from the lens of the camera, Fa gets farther from C where d(Fa) < d(Fb). Since d(Fa) < d(Fb) leads to the larger angle of Fa, the binocular disparity after zoom-in should be larger than before as shown in Fig. 1.
Likewise, as the binocular disparity caused by the larger difference between the convergence point and the focus point gets larger, the user views the more uncomfortable stereoscopic images and feels dizzy. In this paper, we define this phenomenon as visual discomfort caused by zoom. To solve this problem, we suggest the adjustment methods of the convergence point in zoom.
IV. CONVERGENCE POINT ADJUSTMENT
4.1. Relational Model by a Convergence Point and a Focus
To consider all cases which the adjustment methods deal with, it is required to classify the position of a focus point and a convergence point before zoom since the object which is zoomed can be focused or not. When the object is zoomed, the point of the focus before zoom is classified into three cases by the position of Fb relative to the position of ZO. The first case is where Fb is located closer to the camera than ZO. In the second case, Fb overlaps with ZO. The third case is that Fb is farther from the camera than ZO.
The convergence point C before the zoom is also classified into three cases by the position of C relative to the position of Fb and ZO. In the first, C is positioned closer to a camera than Fb and ZO. The second case is that C is between Fb and ZO. In the last case, C is farther than Fb and ZO.
Using the classification, Table 1 shows the possible phenomena when the user zooms the object ZO. As shown in Table 1, if the focus point is overlapping with the convergence point and the object, the excessive binocular disparity does not occur. However, since other cases may raise the excessive binocular disparity during a zoom operation, the adjustment method should deal with all cases.
TABLE 1.Relational Model by the position of the focus, the object and the convergence point
4.2. Adjustment Methods
To minimize excessive binocular disparity, it is required to adjust the convergence point during a zoom operation and we design four methods: fixed convergence point (FCP) method, focus-convergence correspondence (FCC) method, coupled focus-convergence point (CFC) method, and convergence point comfort threshold (CCT) method that is based on the stereoscopic threshold.
Let Cb be the convergence point before zoom and Ca be the convergence point after zoom. In the FCP method, Cb does not move after the zoom. It is based on the principles of a current conventional dual-lens stereoscopic camera. This method is easy to use but may suffer from the excessive binocular disparity. FCP can be expressed by the following equation:
In the FCC method, Cb moves to ZO upon zoom-in. It has the benefit of producing comfortable images but should eliminate the depth of images since the convergence point is overlapping with both the focus and the object during the zoom. This can be expressed in terms of the distance value as follows:
In the CFC method, Cb moves towards the focus by a distance equal to the movement distance of the focus. To minimize the change in the visual distance that can occur in the FCC method, the convergence point is moved to the movement direction of the focus, by a distance equal to the movement distance of the focus. Using this method, it is possible to diminish the distance difference between the convergence point and the focus after zoom and produce comfortable images. However, it may cause the depth distortion of images, such as the changes of stereoscopic depth from the positive to the negative parallax or vice versa, if the movement distance of the focus is larger than the distance between Fa and Cb. This distance value can be expressed as follows:
In the CCT method, when zoom starts, Cb moves towards the focus by a distance equal to the movement distance of the focus but within the range of the pre-defined threshold value. If the convergence point moves closer to the focus after zoom than the threshold, the convergence point after zoom, Ca, is set to be the threshold distance, d(θ), which is the maximum distance where the visual discomfort is not caused. This can be defined as follows.
However, if the convergence point after zoom, Ca, is within the threshold distance but |d(Fa) − d(Fb)| is larger than the distance between Cb and ZO, it is possible to cause the depth distortion. To prevent the depth distortion, Ca is set to be the closest point to Fa as follows.
In [5], a maximum binocular disparity of 1°, which is a parallax angle, is recommended for the visual comfort zone. The parallax angle is |α − β| where α and β denote the convergence angle, as shown in Fig. 2. Since |α − β|=1° which is the maximum parallax angle, the half of parallax angle, |α′ − β′| where α′ is the half of α and β′ is the half of β , can be expressed as follows.
FIG. 2.The threshold distance, d(θ).
The distance between two lenses is called the interangle distance(IAD) as shown in Fig. 2. By using the convergence angle and IAD, β′ can be derived as follows.
Since 0.5°≅0.008722 rad, α′ can be expressed by using the equation (6) and (7) as follows.
The distance of convergence point after zoom, d(Ca), can be derived by the equation (8) as follows.
By using the equation (9), we define the threshold distance for the visual comfort, d(θ), as follows:
V. EXPERIMENTS
For the experiments, we evaluate two experiments. The first is to measure the binocular disparity of images by the distance for nine cases. The second is the subjective test conducted to investigate the visual comfort.
5.1. Test Environments
For the disparity evaluation, we make 3D sample videos where the nine cases are applied under the four different distances using the four proposed methods as shown in Fig. 3. To measure the disparity of images, we exploit Sony 3D Box(MPE-200, MPES-3D01). However, we exploit stopped images only in the coupled focus convergence point method since it is impossible to make video clips applied by the CFC method using conventional dual lens stereoscopic cameras.
FIG. 3.Example of sample images.
For the subjective test, the subjects are 48 male and female participants in the age group of 20-40 years (average, 23.2 years); 64.6% of these participants are male and 35.4%, female. All participants have prior experience of watching 3D movies. However, none of them have viewed 3DTV images before. To display the 3D sample videos, 55' 3DTV is exploited, and all participants are asked to wear polarized glasses. The viewing distance for these videos is 205.5 cm, which is three times the height of the TV display (68.5 cm). After watching the videos, each participant answers a questionnaire. All questions related to visual discomfort and the feeling amounts of depth transition are answered using a five-point scale.
5.2. Results of Disparity Measurements
In Sony 3D box, the disparity for the comfortable 3D images is ±3% derived from the binocular disparity of 1°. Figure 4 shows the distribution graph measuring the binocular disparity of 3D images shot by FCP, FCC, CFC and CCT for nine cases. As shown in Fig. 4, when FCC is exploited, the disparity values of the images are close to 0% resulting in the elimination of the 3D depth. In most cases, the disparity values of images shot by FCP and CFC exceed ±3%. It means that the viewers may feel the visual discomfort caused by the excessive disparity when they watch the images. However, for all cases, the disparity values using CCT are less than or equal to ±3% since the difference between the focus and the convergence point is forced to be the threshold distance when it exceeds the threshold.
FIG. 4.Disparity value using FCP, FCC, CFC and CCT.
5.3. Results of the Subjective Test
We obtain meaningful results from the subjective test using three of the proposed methods with four distances and nine cases. As shown in Fig. 5, the FCP method has the lowest score and the CCT method has the highest score for visual comfort under all conditions. The reason to get the highest score in CCT is that the binocular disparity is forced to keep under 1° due to the threshold distance, d(θ). Also as shown in Fig. 6, the viewers always feel less depth transition under all conditions when they watch the videos shot by CCT method during the zoom. Since the more depth transitions rise, the more dizzy the viewers feel, the CCT makes more comfortable 3D videos for the viewers.
FIG. 5Average score of comfortability in FCP, FCC and CCT.
FIG. 6.Average score of feeling amounts for a visual depth transition.
5.4 Side effects
As shown in Fig. 4 and 5, we obtain meaningful results to ease visual discomfort by preventing the excessive binocular disparity. However, the proposed method may cause a few side effects. First, it is possible for the reduction of the binocular disparity to eliminate 3D effect. The binocular disparity is the key concept to make 3D effect. If the binocular disparity is lessened, it is possible for the viewers to watch less 3D effect on a screen, but feel more comfortable. To make 3D be effective efficiently and the shots be comfortable, the proposed method sets the convergence point after zoom to be the threshold distance, d(θ), when the excessive disparity arise. The threshold distance, d(θ), is the maximum distance from the subject where 3D is effective, but the visual discomfort is not caused.
Second, it is possible for a viewer to watch other objects, not the zoomed object. The zoom is the technique to magnify or shrink an object on a camera for various purposes. When the object is zoomed in, the zoomed object is enlarged and the some objects around the zoomed object are cropped due to the narrowed angle of view. Also, the background objects remaining on the zoomed shot are blurring since the depth of zoomed shots is shallow. These lead to the out-of-focus effect to the backgrounds which concentrate the attention of the viewers on the zoomed object. When a viewer watches and concentrates on the zoomed object intended by the shooter under our proposed methods, the viewer feels more comfortable. However, if the viewer watches the background objects, it may be difficult for the viewer to recognize the background objects due to more blur and the reduced depth perception.
VI. CONCLUSION
One of the frequently used functions is a zoom to magnify or reduce an object in a dual lens stereoscopic camera. However, since a convergence point is not coupled with a zoom function, it is possible for the zoom to cause a visual discomfort resulting from an excessive disparity. To solve this problem, we propose and implement four methods that adjust a convergence point according to the distance of a focus, an object, and a convergence point. We also evaluate the experiment measuring a binocular disparity and the subjective test in order to investigate the visual comfort using the sample videos shot by proposed methods. The results show that the convergence point comfort threshold (CCT) method is more effective than the others in order to improve the visual comfort when the zoom is exploited. We expect that these proposed high-effective methods will be applied to produce a dual-lens stereoscopic camera that allows convenient and comfortable stereoscopic photography. Further research is planned to improve the recognition of backgrounds on zoomed shots.
참고문헌
-
S. Yano, M. Emoto, and T. Mitsuhashi, “Two factors in visual fatigue caused by stereoscopic HD TV images,” Displays
25 , 141-150 (2004). https://doi.org/10.1016/j.displa.2004.09.002 -
T. Motoki, H. Isono, and I. Yuyama, “Present status of three dimensional television research,” Proc. IRE
83 , 1009-1021 (1995). - 3D Consortium, “3DC Safety Guidelines for Popularization of Human-Friendly 3D,” (2006).
-
H. Ujike, “Estimation of visually induced motion sickness from velocity component of moving image,” In
Proc . of the 3rd International Conference on Virtual and Mixed Reality, R. Shumaker, ed. (San Diego, CA USA, 2009). -
S. Knorr, K. Ide, M. Kunter, and T. Sikora, “The avoidance of visual discomfort and basic rules for producing good 3D pictures,” SMPTE Motion Imaging J.
121 , 72-79 (2012). - Cyclopital3D, “3D camera Parallax adjustments, the Stereo Window, and using attachments for Close-Up photography,” (2015).
-
B. Zhang, S. Kothandaraman, and A. U. Batur, “Auto Convergence for Stereoscopic 3-D Cameras,” In
Proc . of SPIE8288 , 828809-1-6 (2012). -
K. C. Kwon, Y. T. Lim, and N. Kim, “Vergence Control of Binocular Camera Using Disparity Information,” J. Opt. Soc. Korea
13 , 379-385 (2009). https://doi.org/10.3807/JOSK.2009.13.3.379 -
J. Chen, J. Zhou, and J. Sun, “Visual Discomfort Induced by Adjustment of Convergence Distance in Stereoscopic Video,” in
Proc. IEEE Int’l Sym. On Broadband Multimedia Systems and Broadcasting , (Brunel Univ., UK, Jun. 2013) pp. 1-5. -
Y. J. Jung, S. Lee, H. Sohn, H. W. Park, and Y. M. Ro, “Visual comfort assessment metric based on salient object motion information in stereoscopic video,” J. Electron. Imaging
21 , 011008-1 (2012). -
T. Shibata, J. Kim, D. M. Hoffman, and M. S. Banks, “The zone of comfort: predicting visual discomfort with stereo displays,” J. Vis.
11 , 1-29 (2011). -
J. Ma and S. I. Olsen, “Depth from zooming,” J. Opt. Soc. Am.
7 , 1883-1890 (1990). https://doi.org/10.1364/JOSAA.7.001883 -
E. S. Hwang and N. Kim, “A Method for Estimating a Distance Using the Stereo Zoom Lens Module,” Korean J. Opt. Photon.
17 , 537-543 (2006). https://doi.org/10.3807/KJOP.2006.17.6.537 -
E. K. Jung, S. H. Baek, S. Y. Park, and H. W. Jang, “Optical Property Correction of a Heterogeneous Stereoscopic Camera,” Journal of IKEEE
49 , 74-85 (2012).