1. INTRODUCTION
Today, texts to read are changing into images to watch and feel. Kinetic typography is that it improves the emotional presentation and delivery of information through the text. In particular, it has been widely used in the fields of media art, music video and advertising that require effective delivery and expression of a text. Visual representation form combined with the concept of the time, space and acoustic will have the attraction effect and attentionability. Therefore, the formal language is to maximize the information transfer efficiency. Kinetic typography mixing text and motion is going beyond the text image expressed in digital typography and combined with new temporality in space, developing into a media giving the sense of space. Kinetic typography has also established itself as a strong and dynamic communication tools in the new digital environment beyond the role of information exchange with the traditional print media.
In this study, kinetic typography will be used as a medium to induce bilateral communication promoting active participation of user as well as the realization of non-linearity, metaphor and multiple meanings. The purpose of this study is to identify the design factors in the formativeness of Hangul and to present and implement the interaction design method of unique Hangul kinetic typography based on the scientific structure of Hangul. The study method is to investigate and research Hangul structure and motion recognition technology to propose a interaction method based Hangul typography movement method and exhibit in a media art event. It is expected that this study stimulates the momentum to improve the value of Hangul as a communication language and media art language and develop the branding of Hangul for more added values.
Previous kinetic typography was mostly made using professional motion manufacturing software such as Adobe Premiere, After Effect and Flash. However, this method may require specialized skills and complex and difficult manufacturing process. Beginners to learn the interface of new software takes a lot of time. Therefore, recent studies on kinetic typography are focused on simplifying the production process, using programming technique. These studies on kinetic typography visually communicate with viewer by changing the typography locations (motion path) linearly along the time sequence [1]-[5].
Recently, with the development of motion recognition technology, studies have been active on typography motion path setting according to viewer gesture information [6], [7]. Although such studies express stories including structure by assigning motion paths to standstill words, the letters move only passively to provide rather insufficient instant visual pleasure. If letter movement and sound are connected with viewers’ motion information, messages will be more effectively delivered through movements of this visual language, especially to the present cyber generation communicating usually in the context of instant enjoyment.
This study focuses on the visual aspect in interaction with viewers excluding the sound thereof. Although sound is a very significant factor for spatial appeal, it will not be dealt with in this study for the vast scope of its principal and effectiveness.
The remainder of this paper is organized as follows: some related works of design characteristics of Hangul are reviewed and discussed in Section 2. In Section 3, the proposed interaction design method based on motion recognition technology is illustrated. In Section 4, the implementation of kinetic typography and media art exhibition is illustrated. Lastly, in Section 5, the conclusion of this work is drawn.
2. DESIGN CHARACTERISTICS OF HANGUL
Hangul is a unique and scientific writing system which was developed based on a sound and form. The excellence of Hangul has been already proven in the global linguistics area. Hangul shows ideal design characteristics in its background and process of creation.
2.1 Formative characteristics
Many studies have already researched the formative beauty of Hangul and evaluated Hangul as a scientific and logical language. Hangul was created with 6 basic morphemes – dot, horizontal line, vertical line, circle, square and triangle. All of them are in geometric forms. These morphemes can make any shape freely in Hangul. Such simplified and easy basic morphemes generate harmony between each letter, overall rhythmic sense and dynamism. Hangul forms each complete character by combining vowel and consonant. Consonants are based on four forms ‘ㄴ, ㅁ, ㅅ, ㅇ’. Every consonant developed from the four forms have a precise symmetry in a certain imaginary square. Such is also a basic formative element in modern designs. The basic morphemes of vowel are dot representing the sky (universe) and from which horizontal line (representing the land) and vertical line (representing human) are mixed in diverse forms to consist the vowel system [8].
Fig. 1.Deployment principles of Hangul, 14 consonants and 10 vowels
2.2 Structural characteristics
Hangul is a typical alphabetic writing where vowels and consonants are separated and very well-organized featural alphabets writing consisted of 14 consonants and 10 vowels. Alphabetic writing refers to letter system based on phonemes and these phonemes are combined to form a syllable and word. Featural alphabets mean letters whose nature is changed to become a different letter every time a stroke is added. Unlike the Roman alphabet, Hangul combines 2~4 vowels and consonants together to form one complete letter. It is a syllable based writing in a multidimensional composition. Depending upon consonant and vowel combination structure, Hangul letters are divided into horizontal gather letter, vertical gather letter and mixed gather letter.
Fig. 2.Consonant/Vowel combination structure
3. INTERACTION DESIGN BASED ON MOTION RECOGNITION TECHNOLOGY
3.1 Neologism ‘Hur’
Amid the recent growing internet and mobile use, a lot of neologisms have been created. It is even impossible to guess the original meaning of some new exclamations. And as online chatting spread widely, many tend not to write full sentences and reduce them for quick conversations. Many new coined words are leading a new culture of language use either in a serious language destruction or adolescent cultural trend to maximize language efficiency.
‘Hur(헐)’ is a neologism used in the internet space of South Korea when one is surprised, embarrassed or made to laugh. It is also used as an exclamation to express one’s emotion generally in unexpected situations or at unpredicted news without a known cause.
In this study, the neologism ‘Hur’ was selected as a content for media art exhibition to exteriorize provocatively the emotion of preposterousness when facing unforeseen situation or story in one’s own generation or the contemporary observational experience implied therein. The neologism ‘Hur’ was selected for two reasons. One is that since ‘Hur’ is a horizontal gather letter combining two consonants and one vowel to be well matched to human joint(skeleton). Another reason is that ‘Hur’ reflects the diversified and complicated psychological status in the simplest way as well as the postmodern inclination of the contemporary youths, generation communion, practicality, etc.
3.2 Motion interface design
In general, the digital medial space is limited to a squareformed place. In order to express the depth in space within the limited screen, letter sizes can be adjusted or story unfolding method. There are two digital video unfolding methods. Linear structure method is to unfold video consecutively and nonlinear structure method is for user to discretionarily access and selectively explore just as in the web(hypermedia).
In this study, both typography size and story development were synchronized with viewers’ motions to reveal situational characteristics. The camera installed on the screen captures viewers and adjusts typography sizes. Also, viewers are allowed to seek the nonlinear structure to adjust the rhythm and pace thereof for their own to build own moving path instead of following the moving path randomly set by a writer. We designed an immersive user interface using gesture-based contactless interface. Gesture recognition interface shows the results of immediate response to the user's movements on the display.
A lot of studies in real-time gesture recognition area using depth camera they have been made in a few years. In particular MS's Kinect sensor provides RGB image, depth value and joints tracking information in real time in spite of the low-priced depth camera. The appearances of these sensors reduce your troubles for body detection and pose estimation needed for gesture recognition [9]-[13].
We have used the Kinect sensor for the design of the interface of Kinetic Typography Hangul to interact with the viewer. Kinect recognizes 20 joints of human using a camera and an infrared sensor and passes the coordinate values. To obtain joint information from a human, body of a human should be within a Kinect sensor's FOV(Field Of View).
We used 15 joint information of recognition that Kinect implements typography. Kinect sensor is able to recognize 20 joints of a human. We used 15 joints partially in this study. Fig. 3 outlines how to build the letter ‘Hur’ by connecting 15 joints. We are considering aesthetic factors when shaping the letters. To do this, we set up an offset value for each letters. The offset value is used in transforming the actual coordinates of each joint. The offset value is a very important element to natural interactive typography. Therefore, a lot of time and the test was conducted for the setting of the offset value.
Fig. 3.Matching between viewer joint information and typography
4. MOVEMENT IMPLEMENTATION
In this study, personified typography was expressed by matching a viewer’s movement in the 3 dimensions and typography movement in the plane 2-dimensional screen. The personified typography is characterized to move to the x, y and z axes directions at the same time, provoking extreme tension and interest from viewers.
The x axis of typography was set along the horizontal direction (left-right) in a viewer’s space; y axis, along the vertical (up-down) direction; and z axis, along the distance between a viewer and the screen. Viewers feel different sense of space according to typography movement directions reflecting their own appearances. The horizontal direction grants the basic stability of the screen, the calm lyricism as if oriental paintings. On the other hand, the vertical direction stimulates dynamism and interest. The z axis adds the sense of distance in space.
Kinetic Typography system proposed in this study activates at the moment they recognize the joint information of the viewer. And the system will draw a shape during the interaction time according to the information of the tracked skeleton. We set interaction time to 20 seconds in consideration of the fun of the user. After interaction time passes, the system captures image of the viewer and store it in the memory space. Snapshots of the viewer is used to play in the idle state where there is no viewer, it is played from the most recent image.
The figure below shows the interaction flow of the proposed system in this study.
Fig. 4.Interaction flow of Hangul Kinetic typography
The system proposed in this study is a two-player program developed in the environment of Visual Studio 2010 by combining the open source vision library of OpenCV 2.3.5 with Kinect SDK 1.7 of Microsoft. Fig. 5 shows viewers were actually interacting in the media art exhibition with a developed interface.
Fig. 5.User interaction with kinetic typography in the media art exhibition
5. CONCLUSION
Recently, in line with the emergence of new media, digital typography was developed to use movement itself as an expression form for the pursuit of formative beauty. And as the idea of time was introduced, movement forces function as a significant part of works. This means that, based on viewers’ movement, effects like tension and optical illusion can be achieved. Sight perception such as a variety of changes in forms depending upon the viewer’s location would add significance to work. And the form changes with time, viewer is more interested and their sight perception is more stimulated.
This study was initiated by the question that; can the kinetic typography be recognized as a new communication factor in the era of digital video for its oral historicity and dynamism? We selected the Hangul as the target of kinetic typography. The reason is because identity of the Korean remains intact in Hangul. In this study, a Hangul kinetic typography was proposed and realized, which varies in its sizes and motion directions according to a viewer’s movement and exhibited in a media art event. Exhibition participants felt maximized sense of space and absorption while doing interaction with the typography.
As a result, it was found possible herein to use the Hangul kinetic typography not only as a means to communicate with viewer but also as a medium to show the artistic value of Hangul. In addition, we could see the Hangul has high development potential as a major design elements as well as an element of simple information communication in accordance with the times. In the future on the basis of this study results, it is deemed necessary to expand the study into a kinetic typography library applicable to more diverse letters.
References
- J. Forlizzi, J. Lee, and S. Hudson, "The kinedit system: affective messages using dynamic texts," Proceeding of the SIGCHI Conference on Human Factors in Computing Systems, 2003, pp. 377-384.
- J. Kim, M. Kim, S. Lim, and Y. Choy, "Kinetic Typography Motion Propagation Technique based on Text Hierarchy," Journal of KIISE Transactions on Computing Practices, vol. 20, no. 8, 2014, pp. 441-449.
- J. Lee, J. Forlizzi, and S. Hudson, "The kinetic typography engine: an extensible system for animating expressive text," Proceeding of the 15th annual ACM symposium on User interface software and technology, 2002, pp. 81-90.
- J. Lewis and A. Weyers, "ActiveText: a method for creating dynamic and interactive texts," Proceeding of the 12th annual ACM symposium on User interface software and technology, 1999, pp. 131-140.
- Y. Cho, S. Woo, and S. Lim, "Design and Implementation of the Primitive Motion API for Kinetic Typography," Journal of Korea Multimedia Society, vol. 18, no. 6, 2015, pp. 763-771. https://doi.org/10.9717/kmms.2015.18.6.763
- S. Shin and H. Kim, "A Study on the Interface Based on NUI for Creative Hangul Kinetic Typography," The Korean Journal of animation, vol. 11, no. 1, 2015, pp. 124-137.
- T. Pimentel and V. Branco, "Dynamic and interactive typography in digital art," Computers & Graphics, vol. 29, issue. 6, 2005, pp. 882-889. https://doi.org/10.1016/j.cag.2005.09.005
- J. Kim, Letters representation of Hangul, Mijin Publisher, 1983.
- A. Barmpoutis, "Tensor Body: Real-Time Reconstruction of the Human Body and Avatar Synthesis From RGB-D," IEEE Transactions on Cybernetics, vol. 43, issue. 5, 2013, pp. 1347-1356. https://doi.org/10.1109/TCYB.2013.2276430
- L. Oikonomidis, N. Kyriazis, and A. Argyros, "Efficient model-based 3D tracking of hand articulations using Kinect," Proceeding of British Machine Vision Conference, 2011, pp. 3-13.
- L. Xia, C. Chen, and J. Aggarwal, "Human detection using depth information by kinect," Proceeding of Workshop on Human Activity Understanding from 3D Data at Conference on Computer Vision and Pattern Recognition, 2011, pp. 15-22.
- S. Lim and S. Kim, "Holographic Projection System with 3D Spatial Interaction," Proceeding of the 1st International Conference on Advanced Data and Information Engineering, 2013, pp 409-416.
- T. Xiaolong, X. Pin, and Y. Xing, "Research on Skeleton Animation Motion Data Based on Kinect," Proceeding of 2012 Fifth International Symposium on Computational Intelligence and Design, vol. 2, 2012, pp. 347-350.