• 한국HCI학회:학술대회논문집
• /
• 2009.02a
• /
• pp.278-284
• /
• 2009

Multimodal data to have several modalities such as videos, images, sounds and texts for one contents is increasing. Since this type of data has ill-defined format, it is not easy to represent the crossmodal information for them explicitly. So, we proposed new method to extract and analyze vision-language crossmodal association information using the documentaries video data about the nature. We collected pairs of images and captions from 3 genres of documentaries such as jungle, ocean and universe, and extracted a set of visual words and that of text words from them. We found out that two modal data have semantic association on crossmodal association information from this analysis.

• Language and Information
• /
• v.9 no.2
• /
• pp.1-22
• /
• 2005

In this paper I propose the semantics of two modality markers in Korean, keyss and (u)1 kes. I compare the two modality markers with respect to some properties. First, keyss is used to express logical necessity while (u)1 kes can be used to express a simple prediction as well. Second, keyss expresses some logical conclusion from the speaker's own information state without claiming it is true. On the other hand, (u)1 kes expresses the claim that the speaker's prediction will be true. Third, the prediction of keyss is non-monotonic: it can be reversed without being inconsistent. However, that of (u)1 kes cannot. Fourth, (u)1 kes can be used freely in epistemic conditionals, but keyss cannot. Finally, when keyss is used, the prediction cannot be repeated. The prediction from the use of (u)1 kes can be repeated. To account for these differences, I propose that keyss is used when the speaker makes a purely logical presumption based on his/her own information state, and that (u)1 kes is used to make a prediction which is asserted to be true. This proposal accounts for all the differences of the two modality markers.

• The Journal of Korean Orthopaedic Ultrasound Society
• /
• v.1 no.2
• /
• pp.94-111
• /
• 2008

Traditionally the diagnosis and treatment of the diseases of the knee is based on the findings of the x-rays and the MRI. The x-rays provide good information of the changes of the internal structure of the bone. However, there is a limitation in providing information of the soft tissue and the cartilage. The MRI is one of the most expensive diagnostic modalities and it can not give us a dynamic and real time information. The sonography has a role in diagnosis and treatment of the soft tissue disease and surface of the bone. It gives us a real time dynamic information and it is really cheap. In this article the sonographic findings of the normal and pathologic conditions of the knee joint are introduced in relation to the findings of the x-rays and the MRI.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.12 no.1
• /
• pp.392-412
• /
• 2018

Due to the semantic gap problem across different modalities, automatically retrieval from multimedia information still faces a main challenge. It is desirable to provide an effective joint model to bridge the gap and organize the relationships between them. In this work, we develop a deep image annotation and classification by fusing multi-modal semantic topics (DAC_mmst) model, which has the capacity for finding visual and non-visual topics by jointly modeling the image and loosely related text for deep image annotation while simultaneously learning and predicting the class label. More specifically, DAC_mmst depends on a non-parametric Bayesian model for estimating the best number of visual topics that can perfectly explain the image. To evaluate the effectiveness of our proposed algorithm, we collect a real-world dataset to conduct various experiments. The experimental results show our proposed DAC_mmst performs favorably in perplexity, image annotation and classification accuracy, comparing to several state-of-the-art methods.

• International Journal of Contents
• /
• v.18 no.3
• /
• pp.11-20
• /
• 2022

Human Emotion Recognition is an exciting topic that has been attracting many researchers for a lengthy time. In recent years, there has been an increasing interest in exploiting contextual information on emotion recognition. Some previous explorations in psychology show that emotional perception is impacted by facial expressions, as well as contextual information from the scene, such as human activities, interactions, and body poses. Those explorations initialize a trend in computer vision in exploring the critical role of contexts, by considering them as modalities to infer predicted emotion along with facial expressions. However, the contextual information has not been fully exploited. The scene emotion created by the surrounding environment, can shape how people perceive emotion. Besides, additive fusion in multimodal training fashion is not practical, because the contributions of each modality are not equal to the final prediction. The purpose of this paper was to contribute to this growing area of research, by exploring the effectiveness of the emotional scene gist in the input image, to infer the emotional state of the primary target. The emotional scene gist includes emotion, emotional feelings, and actions or events that directly trigger emotional reactions in the input image. We also present an attention-based fusion network, to combine multimodal features based on their impacts on the target emotional state. We demonstrate the effectiveness of the method, through a significant improvement on the EMOTIC dataset.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.18 no.1
• /
• pp.105-125
• /
• 2024

Gliomas are the most common malignant brain tumor and cause the most deaths. Manual brain tumor segmentation is expensive, time-consuming, error-prone, and dependent on the radiologist's expertise and experience. Manual brain tumor segmentation outcomes by different radiologists for the same patient may differ. Thus, more robust, and dependable methods are needed. Medical imaging researchers produced numerous semi-automatic and fully automatic brain tumor segmentation algorithms using ML pipelines and accurate (handcrafted feature-based, etc.) or data-driven strategies. Current methods use CNN or handmade features such symmetry analysis, alignment-based features analysis, or textural qualities. CNN approaches provide unsupervised features, while manual features model domain knowledge. Cascaded algorithms may outperform feature-based or data-driven like CNN methods. A revolutionary cascaded strategy is presented that intelligently supplies CNN with past information from handmade feature-based ML algorithms. Each patient receives manual ground truth and four MRI modalities (T1, T1c, T2, and FLAIR). Handcrafted characteristics and deep learning are used to segment brain tumors in a Global Convolutional Neural Network (GCNN). The proposed GCNN architecture with two parallel CNNs, CSPathways CNN (CSPCNN) and MRI Pathways CNN (MRIPCNN), segmented BraTS brain tumors with high accuracy. The proposed model achieved a Dice score of 87% higher than the state of the art. This research could improve brain tumor segmentation, helping clinicians diagnose and treat patients.

• Progress in Medical Physics
• /
• v.9 no.1
• /
• pp.47-53
• /
• 1998

Nuclear Medicine Images have comparatively poor spatial resolution, making it difficult to relate the functional information which they contain to precise anatomical structures. Anatomical structures useful in the interpretation of SPECT /PET Images were radiolabelled. PET/SPECT Images Provide functional information, whereas MRI mainly demonstrate morphology and anatomical. Fusion or Image Registration improves the information obtained by correlating images from various modalities. Brain Scan were studied on one or more occations using MRI and SPECT. The data were aligned using a point pair methods and surface matching. SPECT and MR Images was tested using a three dimensional water fillable Hoffman Brain Phantom with small marker and PET and MR Image was tested using a patient data. Registration of SPECT and MR Images is feasible and allows more accurate anatomic assessment of sites of abnormal uptake in radiolabeled studies. Point based registration was accurate and easily implemented three dimensional registration of multimodality data set for fusion of clinical anatomic and functional imaging modalities. Accuracy of a surface matching algorithm and homologous feature pair matching for three dimensional image registration of Single Photon Emission Computed Tomography Emission Computed Tomography (SPECT), Positron Emission Tomography (PET) and Magnetic Resonance Images(MRD was tested using a three dimensional water fill able brain phantom and Patients data. Transformation parameter for translation and scaling were determined by homologous feature point pair to match each SPECT and PET scan with MR images.

• Proceedings of the IEEK Conference
• /
• 2002.07a
• /
• pp.285-288
• /
• 2002

Compression of magnetic resonance images (MRI) has proved to be more difficult than other medical imaging modalities. In an average sized hospital, many tora bytes of digital imaging data (MRI) are generated every year, almost all of which has to be kept. The medical image compression is currently being performed by using different algorithms. In this paper, Fuzzy C-Means (FCM) algorithm is used for the Vector Quantization (VQ). First, a digital image is divided into subblocks of fixed size, which consists of 4${\times}$4 blocks of pixels. By performing 2-D Discrete Cosine Transform (DCT), we select six DCT coefficients to form the feature vector. And using FCM algorithm in constructing the VQ codebook. By doing so, the algorithm can make good time quality, and reduce the processing time while constructing the VQ codebook.

• Proceedings of the KSPS conference
• /
• 2002.11a
• /
• pp.205-208
• /
• 2002

Bimodal speech recognition based on lip reading has been studied as a representative method of speech recognition under noisy environments. There are three integration methods of speech and lip modalities as like direct identification, separate identification and dominant recording. In this paper we evaluate the robustness of lip reading methods under the assumption that lip parameters are estimated with errors. We show that the dominant recording approach is more robust than other methods with lip reading experiments. Also, a measure of lip parameter degradation is proposed. This measure can be used in the determination of weighting values of video information.

• Asian Oncology Nursing
• /
• v.10 no.1
• /
• pp.19-29
• /
• 2010

Purpose: The purpose of this article was to review and discuss the current status, issues, and nursing perspectives of cancer survivorship. Methods: A comprehensive literature review was conducted. Results: The major areas of concern for the survivors included recurrence, secondary malignancies, and long-term treatment sequalae which affect their quality of life. The four essential components of survivorship are prevention, surveillance, intervention, and coordination. Cancer survivorship care plan should address survivor's long-term care, such as types of cancer, treatment modalities, potential side effects, and recommendations for follow-up. It also needs to include preventive practices, health maintenance and well-being, information on legal protections regarding employment and health insurance, as well as psychosocial services in the community. Survivorship care for cancer patients requires multidisciplinary efforts and team approach. Conclusion: Nurses are uniquely positioned to play a key role in ensuring quality services for cancer survivors and family members. Nurses should review the care plans for cancer survivorship with patients and families by instructing them when to seek medical treatment, promoting any recommended surveillance protocols, and encouraging healthy life styles for health promotion and quality of life.