딥러닝 모델과 이미지 피처 벡터 클러스터링을 활용한 관광지별 사진 분류 및 선호장면 추출

Abstract

소셜 미디어 플랫폼의 일상화로 관광객은 자신의 경험을 사진과 텍스트를 통해 온라인상에 공유하고 이는 잠재적 관광객에게 정보원의 역할을 하고 있다(i Agusti., 2018). 이러한 데이터는 관광객의 시각에서 목적지 이미지를 파악할 수 있다는 점에서 관심을 받고 있다. 특히 사진은 특정 지역에 대한 관광객의 시각적 선호도를 담고 있어 소수의 전문가들보다 실제 관광객들의 선호도를 더 직접적으로 반영할 수 있다(Sun et al., 2021). 이와 같은 사진의 가치에 주목하여 관광객이 촬영한 사진을 분석하고 이로부터 관광지의 매력요인을 추출하려는 연구들이 진행되고 있다. 딥러닝 모델을 활용한 방법론이 활용되기 전에는 목측을 이용한 연구가 주를 이루었다. 그러나 이러한 연구는 분석할 수 있는 사진의 양에 한계가 존재하여 관광지에 대한 종합적인 분석이 힘들다는 한계를 가진다. 최근 컴퓨터 비전 분야의 급속한 발전과 함께 딥러닝 모델을 활용하여 대량의 관광 사진을 분석하려는 연구가 시도되고 있다. 그러나 하나의 카테고리를 기준으로 여러 관광지를 분석하기 때문에 관광지의 차별적 특징을 파악하기 어렵다는 한계가 존재한다. 또한 동일한 카테고리로 분류된 사진 속에서도 다양한 시각적 내용이 존재하므로 이를 분석하는 연구가 필요하다. 본 연구는 관광객이 촬영한 사진을 분석하여 개별 관광지의 특징을 파악할 수 있는 방법을 제안한다. 본 연구의 목적은 딥러닝 모델과 이미지 피처 벡터 클러스터링 기법을 활용하여 관광지별 이미지 카테고리를 생성하고 이에 따라 사진을 분류한 후 카테고리별 선호장면을 추출하는 방법을 개발하는 것이다. 이를 위해 첫째, Tripadvisor 사이트에 외국인 관광객이 게시한 리뷰에 첨부된 사진을 수집했다. 둘째, 이미지의 특징을 추출하고 이를 카테고리화 하기 위해 먼저 Places365로 사전 훈련된 VGG16 네트워크를 활용하여 개별 이미지를 512차원의 피처 벡터로 임베딩했다. 이를 t-SNE로 2차원으로 축소하고 HDBSCAN 분석을 통해 클러스터를 추출하고 이를 지역의 사진 카테고리로 정하였다. 이후 카테고리에 포함되지 않는 노이즈를 제거하고, 카테고리에 따라 사진을 분류하기 위해 샴 네트워크를 학습시켰다. 셋째, 생성된 카테고리별로 대표사진을 추출하기 위해 클러스터링 방법을 사용했다. 카테고리별로 t-SNE를 활용하여 512차원의 이미지 피처 벡터를 2차원으로 축소하고 HDBSCAN으로 분석하여 추출된 클러스터로 카테고리별 선호장면을 찾아냈다. 본 연구에서 제안한 방법론을 활용하여‘경복궁’과 ‘인사동’ 지역을 분석했다. 경복궁에서는 ‘경회루’, ‘근정전’, ‘흥례문’, ‘광화문’, ‘향원정’, ‘민속박물관’, ‘어좌’, ‘한복’, ‘수문장’, ‘나무’ 총 10개로 이루어진 카테고리가 생성되었다. 인사동은 ‘쌈지길’, ‘인사동 거리’, ‘식음료’, ‘기념품’으로 이루어진 카테고리가 생성되었다. 카테고리 생성을 통해 해당 관광지에서 어떠한 사진들이 많이 촬영되는지 알 수 있었다. 또한 카테고리 항목별 선호장면 추출을 통해 이러한 매력요인에 기여하는 구체적인 장면을 확인할 수 있었다. 이는 외국인 관광객을 대상으로 하는 홍보 및 마케팅 전략 수립에 있어 기초자료의 역할을 할 수 있다. 특히 추출된 선호장면의 경우 홍보 및 마케팅에 직접적으로 활용될 수 있다. 향후 과제로는 다음과 같은 연구를 진행할 수 있다. 첫째, 관광목적지 이미지에 대한 내국인과 외국인의 차이를 분석하는 연구를 제안한다. 둘째, 시간의 흐름에 따른 관광목적지 이미지의 변화를 분석하는 연구를 제안한다. 셋째, 문화권별 혹은 출신 국가별로 관광목적지 이미지를 분석하는 연구를 제안한다. ;With the dailyization of social media platforms, tourists share their experiences online through photos and text, which serves as an informantion for potential tourists. These data are attracting attention in that they can grasp the destination image from the perspective of tourists. In particular, photos contain tourists' visual preferences for a specific area, so they can reflect actual tourists' preferences more directly than a small number of experts. Focusing on the value of such photos, studies are being conducted to analyze photos taken by tourists and extract attractive factors of tourist attractions. Before the methodology using the deep learning model was used, researchers conducted studies by observing data. However, these studies have limitations in the amount of photos that can be analyzed, making it difficult to comprehensively analyze tourist attractions. With the rapid development of the computer vision field in recent years, research is being attempted to analyze large amounts of tourism photos using deep learning models. However, since several tourist attractions are analyzed based on one category, there is a limitation that it is difficult to grasp the unique characteristics of tourist attractions. In addition, It is in one category item that there are various photo with different visual contents. so research to analyze them is needed. This study proposes a method that can analyze photos taken by tourists to grasp the characteristics of individual tourist attractions. The purpose of this study is to develop a method of building photo categories for each tourist destination and extracting favorite scene for each items of category using deep learning models and clustering techniques. To this end, first, photos attached to reviews posted by foreign tourists were collected on Tripadvisor. Second, in order to extract and categorize the features of the photos, individual photos were embedded as 512-dimensional feature vectors using the VGG16 network pre-trained with Place365. These vectors were reduced using t-SNE in two dimensions, the cluster was extracted through HDBSCAN analysis, and it was determined as a regional photo category. Afterwards, noise not included in the category was removed, and a Siamese network was learned to classify photos according to the category. Third, a clustering method was used to extract representative photos for each generated category. Using t-SNE for each item of category, 512-dimensional image feature vectors were reduced to two dimensions and analyzed by HDBSCAN to find representative images for each category. Using the methodology proposed in this study, ‘Gyeongbokgung Palace’ and ‘Insa-dong’ were analyzed. In Gyeongbokgung Palace, A category consisting of 10 items was created: Gyeonghoeru, Geunjeongjeon, Heungryemun, Gwanghwamun, Hyangwonjeong Folk Museum, Eojwa, Hanbok, Sumunjang, and Tree. In Insa-dong, A category consisting of 10 items was created: Ssamji-gil, Insa-dong Street, Food & Beverage, and Souvenirs. Through building the category, it was possible to know what photos were taken a lot at the tourist destination. In addition, through the extraction of favorite photos, specific scenes contributing to these attractive factors could be identified. This can be used as basic data in establishing promotional and marketing strategies for foreign tourists. In particular, favorite photos can be directly used for promotion and marketing. As a future studies, the following research can be conducted. First, a study is proposed to analyze the difference between Koreans and foreigners on the image of tourist destinations. Second, a study is proposed to analyze changes in the image of tourist destinations over time. Third, a study is proposed to analyze the image of tourist destinations by culture or country of origin. With the dailyization of social media platforms, tourists share their experiences online through photos and text, which serves as an informantion for potential tourists. These data are attracting attention in that they can grasp the destination image from the perspective of tourists. In particular, photos contain tourists' visual preferences for a specific area, so they can reflect actual tourists' preferences more directly than a small number of experts. Focusing on the value of such photos, studies are being conducted to analyze photos taken by tourists and extract attractive factors of tourist attractions. Before the methodology using the deep learning model was used, researchers conducted studies by observing data. However, these studies have limitations in the amount of photos that can be analyzed, making it difficult to comprehensively analyze tourist attractions. With the rapid development of the computer vision field in recent years, research is being attempted to analyze large amounts of tourism photos using deep learning models. However, since several tourist attractions are analyzed based on one category, there is a limitation that it is difficult to grasp the unique characteristics of tourist attractions. In addition, It is in one category item that there are various photo with different visual contents. so research to analyze them is needed. This study proposes a method that can analyze photos taken by tourists to grasp the characteristics of individual tourist attractions. The purpose of this study is to develop a method of building photo categories for each tourist destination and extracting favorite scene for each items of category using deep learning models and clustering techniques. To this end, first, photos attached to reviews posted by foreign tourists were collected on Tripadvisor. Second, in order to extract and categorize the features of the photos, individual photos were embedded as 512-dimensional feature vectors using the VGG16 network pre-trained with Place365. These vectors were reduced using t-SNE in two dimensions, the cluster was extracted through HDBSCAN analysis, and it was determined as a regional photo category. Afterwards, noise not included in the category was removed, and a Siamese network was learned to classify photos according to the category. Third, a clustering method was used to extract representative photos for each generated category. Using t-SNE for each item of category, 512-dimensional image feature vectors were reduced to two dimensions and analyzed by HDBSCAN to find representative images for each category. Using the methodology proposed in this study, ‘Gyeongbokgung Palace’ and ‘Insa-dong’ were analyzed. In Gyeongbokgung Palace, A category consisting of 10 items was created: Gyeonghoeru, Geunjeongjeon, Heungryemun, Gwanghwamun, Hyangwonjeong Folk Museum, Eojwa, Hanbok, Sumunjang, and Tree. In Insa-dong, A category consisting of 10 items was created: Ssamji-gil, Insa-dong Street, Food & Beverage, and Souvenirs. Through building the category, it was possible to know what photos were taken a lot at the tourist destination. In addition, through the extraction of favorite photos, specific scenes contributing to these attractive factors could be identified. This can be used as basic data in establishing promotional and marketing strategies for foreign tourists. In particular, favorite photos can be directly used for promotion and marketing. As a future studies, the following research can be conducted. First, a study is proposed to analyze the difference between Koreans and foreigners on the image of tourist destinations. Second, a study is proposed to analyze changes in the image of tourist destinations over time. Third, a study is proposed to analyze the image of tourist destinations by culture or country of origin.