• Journal of Intelligence and Information Systems
• /
• v.25 no.1
• /
• pp.163-177
• /
• 2019

As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.

• MISULJARYO - National Museum of Korea Art Journal
• /
• v.96
• /
• pp.89-122
• /
• 2019

In this paper, I argue that the Handscroll of a Sightseeing Trip to the Hangang and Imjingang Rivers by Jeong Su-yeong (1743~1831, pseudonym: Jiwujae) is a record of his private journeys to several places on the outskirts of Hanyang (present-day Seoul) and that it successfully embodies the painter's subjective perspective while boating on these rivers and going on outings to nearby mountains. Around 1796, Jeong Su-yeong traveled to different places and documented his travels in this 16-meter-long handscroll. Several leaves of paper, each of which depicts a separate landscape, are pieced together to create this long handscroll. This indicates that the Handscroll of a Sightseeing Trip to the Hangang and Imjingang Rivers reflected the painter's personal subjective experiences as he went along his journey rather than simply depicts travel destinations. The Handscroll of a Sightseeing Trip to the Hangang and Imjingang Rivers features two types of travel: boating and mountain outings on foot. Traveling by boat takes up a large portion of the handscroll, which illustrates the channels of the Hangang and Imjingang Rivers. Mountain outings correspond to the sections describing the regions around Bukhansan, Gwanaksan, and Dobongsan Mountains. Jeong Su-yeong traveled to this wide span of places not just once, but several times. The fact that the Hangang River system are not presented in accordance with their actual locations shows that they were illustrated at different points. After visiting the riversides of the Hangang and Namhangang Rivers twice, Jeong Su-yeong delineated them in fourteen scenes. Among them, the first eight illustrate Jeong's initial trip by boat, while the other six scenes are vistas from his second trip. These fourteen scenes occupy half of this handscroll, indicating that the regions near the Hangang River are painted most frequently. The scenes of Jeong Su-yeong's first boating trip to the system of the Hangang River portray the landscapes that he personally witnessed rather than famous scenes. Some of the eight scenic views of Yeoju, including Yongmunsan Mountain, Cheongsimru Pavilion, and Silleuksa Temple, are included in this handscroll. However, Jeong noted spots that were not often painted and depicted them using an eye-level perspective uncommon for illustrating famous scenic locations. The scenes of Jeong's second boating trip include his friend's villa and a meeting with companions. Moreover, Cheongsimru Pavilion and Silleuksa Temple, which are depicted in the first boating trip, are illustrated again from different perspectives and in unique compositions. Jeong Su-yeong examined the same locations several times from different angles. A sense of realism is demonstrated in the scenes of Jeong's first and second boating trips to the channels of the Hangang River, which depict actual roads. Furthermore, viewers can easily follow the level gaze of Jeong from the boat. The scenes depicting the Imjingang River begin from spots near the Yeongpyeongcheon and Hantangang Rivers and end with places along the waterways of the Imjingang River. Here, diverse perspectives were applied, which is characteristic of Imjingang River scenes. Jeong Su-yeong employed a bird's-eye perspective to illustrate the flow of a waterway starting from the Yeongpyeongcheon River. He also used an eye-level perspective to highlight the rocks of Baegundam Pool. Thus, depending on what he wished to emphasize, Jeong applied different perspectives. Hwajeogyeon Pond located by the Hantangang River is illustrated from a bird's-eye perspective to present a panoramic view of the surroundings and rocks. Similarly, the scenery around Uhwajeong Pavilion by the Imjingang River are depicted from the same perspective. A worm's-eye view was selected for Samseongdae Cliff in Tosangun in the upper regions of the Imjingang River and for Nakhwaam Rock. The scenes of Jeong Su-yeong's mountain outings include pavilions and small temple mainly. In the case of Jaeganjeong Pavilion on Bukhansan Mountain, its actual location remains unidentified since the pavilion did not lead to the route of the boating trip to the system of the Hangang River and was separately depicted from other trips to the mountains. I speculate that Jaeganjeong Pavilion refers to a pavilion either in one of the nine valleys in Wooyi-dong at the foot of Bukhansan Mountain or in Songajang Villa. Since these two pavilions are situated in the valleys of Bukhansan Mountain, their descriptions in written texts are similar. As for Gwanaksan Mountain, Chwihyangjeong and Ilganjeong Pavilions as well as Geomjisan Mountain in the Bukhansan Mountain range are depicted. Ilganjeong Pavilion was a well-known site on Gwanaksan that belonged to Shin Wi. In this handscroll, however, Jeong Su-yeong recorded objective geographic information on the pavilion rather than relating it to Shin Wi. "Chwihyangjeong Pavilion" is presented within the walls, while "Geomjisan Mountain" is illustrated outside the walls. Handscroll of a Sightseeing Trip to the Hangang and Imjingang Rivers also includes two small temples, Mangwolam and Okcheonam, on Dobongsan Mountain. The actual locations of these are unknown today. Nevertheless, Gungojip (Anthology of Gungo) by Yim Cheonsang relates that they were sited on Dobongsan Mountain. Compared to other painters who stressed Dobong Seowon (a private Confucian academy) and Manjangbong Peak when depicting Dobongsan Mountain, Jeong Su-yeong highlighted these two small temples. Jeong placed Yeongsanjeon Hall and Cheonbong Stele in "Mangwolam small temple" and Daeungjeon Hall in front of "Okcheonam small temple." In addition to the buildings of the small temple, Jeong drew the peaks of Dobongsan Mountain without inscribing their names, which indicates that he intended the Dobongsan peaks as a background for the scenery. The Handscroll of a Sightseeing Trip to the Hangang and Imjingang Rivers is of great significance in that it embodies Jeong Su-yeong's personal perceptions of scenic spots on the outskirts of Hanyang and records his trips to these places.

• (The) Research of the performance art and culture
• /
• no.43
• /
• pp.5-46
• /
• 2021

The film Moranbong is the product of a trip to North Korea in 1958, when Armangati, Chris Marker, Claude Lantzmann, Francis Lemarck and Jean-Claude Bonardo left at the invitation of Joseon Film. However, for political reasons, the film was not immediately released, and it was not until 2010 that it was rediscovered and received attention. The movie consists of the narratives of Young-ran and Dong-il, set in the Korean War, that are folded into the narratives of Chunhyang and Mongryong in the classic Chunhyangjeon of Joseon. At this time, Joseon's classics are reproduced in the form of the drama Chunhyangjeon, which shares the time zone with the two main characters, and the two narratives are covered in a total of six scenes. There are two layers of middle-story frames in the movie, and if the same narrative is set in North Korea in the 1950s, there is an epic produced by the producers and actors of the Changgeuk Chunhyangjeon and the Changgeuk Chunhyangjeon as a complete work. In the outermost frame of the movie, Dong-il is the main character, but in the inner double frame, Young-ran, who is an actor growing up with the Changgeuk Chunhyangjeon and a character in the Changgeuk Chunhyangjeon, is the center. The following three OST albums are Corée Moranbong released in France in 1960, Musique de corée released in 1970, and 朝鮮の伝統音樂-唱劇 「春香伝」と伝統樂器- released in 1968 in Japan. While Corée Moranbong consists only of the music from the film Moranbong, the two subsequent albums included additional songs collected and recorded by Pyongyang National Broadcasting System. However, there is no information about the movie Moranbong on the album released in Japan. Under the circumstances, it is highly likely that the author of the record label or music commentary has not confirmed the existence of the movie Moranbong, and may have intentionally excluded related contents due to the background of the film's ban on its release. The results of analyzing the detailed scenes of the Changgeuk Chunhyangjeon, Farewell Song, Sipjang-ga, Chundangsigwa, Bakseokti and Prison Song in the movie Moranbong or OST album in the 1950s are as follows. First, the process of establishing the North Korean Changgeuk Chunhyangjeon in the 1950s was confirmed. The play, compiled in 1955 through the Joseon Changgeuk Collection, was settled in the form of a Changgeuk that can be performed in the late 1950s by the Changgeuk Chunhyangjeon between 1956 and 1958. Since the 1960s, Chunhyangjeon has no longer been performed as a traditional pansori-style Changgeuk, so the film Moranbong and the album Corée moranbong are almost the last records to capture the Changgeuk Chunhyangjeon and its music. Second, we confirmed the responses of the actors to the controversy over Takseong in the North Korean creative world in the 1950s. Until 1959, there was a voice of criticism surrounding Takseong and a voice of advocacy that it was also a national characteristic. Shin Woo-sun, who almost eliminated Takseong with clear and high-pitched phrases, air man who changed according to the situation, who chose Takseong but did not actively remove Takseong, Lim So-hyang, who tried to maintain his own tone while accepting some of modern vocalization. Although Cho Sang-sun and Lim So-hyang were also guaranteed roles to continue their voices, the selection/exclusion patterns in the movie Moranbong were linked to the Takseong removal guidelines required by North Korean musicians in the name of Dang and People in the 1950s. Second, Changgeuk actors' response to the controversy over the turbidity of the North Korean Changgeuk community in the 1950s was confirmed. Until 1959, there were voices of criticism and support surrounding Taksung in North Korea. Shin Woo-sun, who showed consistent performance in removing turbidity with clear, high-pitched vocal sounds, Gong Gi-nam, who did not actively remove turbidity depending on the situation, Cho Sang-sun, who accepted some of the vocalization required by the party, while maintaining his original tone. On the other hand, Cho Sang-seon and Lim So-hyang were guaranteed roles to continue their sounds, but the selection/exclusion patterns of Moranbong was independently linked to the guidelines for removing turbidity that the Gugak musicians who crossed to North Korea had been asked for.