Machine Learning Approach to the Effects of the Superstore Mandatory Closing Regulation

Abstract

Purpose - This paper is aimed to analyze the effects of the mandatory closing regulation targeting large retailers, which has been implemented since 2012 to protect small retailers. We examine the changes in consumers' choice of retailers and their purchasing patterns of agri-food following the implementation of such regulation. Research design, data, and methodology - Household spending patterns were identified through the historical data of household food purchase, consumer panel provided by the Rural Development Administration. Clustering was employed to determine the household spending patterns. Moreover, the different household spending patterns before and after the regulation were comparatively studied. The patterns of consumers' choice of retail stores and shopping baskets by the type of retailers, derived from the respective datasets before and after the regulation, were compared to analyze the effects of the regulation. Results -After the regulation, some consumers who used to shop at large retailers before the regulation changed their shopping places to small retailers. However, the product categories that consumers had mainly purchased before the regulation were rarely changed even after the regulation. Conclusions - Although the regulation helped migrate some of the consumers to small retailers, the regulation seemed to have failed to stimulate consumers to purchase the goods, normally bought at large retailers, from traditional markets. In other words, traditional markets are not effective substitutes for regulation-affected retailers.

1. Introduction

While revising the “Distribution Industry Development Act” in January 2012, the National Assembly included a new article that aimed to restrict the operating hours of superstores and super-supermarkets (SSM). In accordance with Article 12-2 of the Act, heads of the local governments were granted authority to restrict these stores' operating hours in their districts or designate mandatory closing days up to two days per month. Following the passing of the amendment, local governments implemented their version of regulations with different timing and methods. These days, however, almost all local governments implement the mandatory closing regulation, designating a specific day every other week. Additionally, a bill is being advanced to extend the scope of target stores beyond superstores and SSM to include multiple shopping complexes and duty-free shops. The regulation is also expected to become tightened against large-sized retail stores by doubling the number of mandatory closing days from the current one-day closing biweekly to every Sunday. All the market participants ranging from superstores to small-sized stores and traditional markets are overly sensitive to every policy announcement because any policy favorable to one party may translate into a matter of survival for others (Park & Chung, 2018).

Consequently, considering a great ripple effect of prospective policy moves to expand the scope of target stores and designate every Sunday as a mandatory closing day, a close examination is required to assess the effectiveness of current policies and reasonably foresee potential impacts following policy changes.

Preceding studies utilized the sales of superstores and SSM, credit card-based data, and price information collected from regulation-affected retail stores in the neighborhood. In other words, they analyzed the policy effects mainly through the changes in sales by the type of store and in consumers' expenditures at those stores. However, this study analyzed the effects of the mandatory closing regulation while considering the spending patterns that consumers exhibit when they visit stores.

To do this, this study used scanner data that contains individual households’ daily food purchase history. Since the data was collected from the real receipts submitted by households, the data includes information of a date of shopping, a place of shopping and a type of stores visited. Recently the importance of data-driven insight has increased in academic field and research based on big data, such as Park (2019) has been frequently conducted. In this paper, taking advantage of the scanner data, we could identify consumers’ daily choice of retail shops and products. The main methodology employed in this study was clustering, which is generally utilized to classify the groups of consumers with varied spending patterns and identify characteristics by group. The spending patterns were defined as how consumers choose the place to shop among alternative options and the product categories they purchase when they visit a specific store. The groups of consumers were subsequently classified depending on the spending patterns, and the policy effectiveness was analyzed based on the changes in the spending patterns of each group following the regulation.

This study differs from previous studies in that scanner data is not simply viewed as expenditure history, but used to identify consumer's choice of retail stores and product purchase pattern. In addition, this study is important in that it analyzes and understands changes in consumer behavior to suggest future policy improvements and practical implications.

Table 4: definition of types of retail store

*Retail stores whose sales of agricultural products accounts for more than 51% of the total sales are exempted from the regulation despite store size.

2. Literature Reviews

Many governments in the world implement various initiatives, like the mandatory closing restriction, to regulate the retail industry. Nonetheless, policies vary from country to country based on their objectives that can be broadly divided into two categories. First, policies are implemented due to religious reasons and workers' right to rest. According to Reddy (2012), Germany enacted the “Shop Closing Law” in 1956, which forced retail stores to be closed to the public during weekday nights and on Sundays, to ensure workers' rest time and encourage religious activities. According to Kok (2008), the French government legislated the law in 1906 to prevent workers from working on Sundays and has since promoted rest and religious activities. The other policy objective is to protect small and medium-sized stores (SMS). According to Grier (2001), Japan established the large-scale “Retail Store Law” in 1973 prior to the introduction of superstores. The objective of the law was to assist SMS for their stable operation by strictly regulating business activities of large-scale retailers. Any would-be store with over 1,500 square meters was required to submit its business plan to the Ministry of Economy, Trade, and Industry. Local merchants and consumers then reviewed the plan whether to allow the store to operate in the area, and when approved, the store's operating hours were then scrutinized. However, with the results of subsequent research studies revealing the negative effects of the policy on SMS and businesses in the neighborhood, the law was relaxed to minimally regulate large retailers except those that caused traffic congestion and noise pollution. France also implemented the regulation on large-scale retailers to protect small retailers. Choi and Lee (2012) analyzes the management strategies of the largest supermarket chain in France during the regulation period. However, the mandatory closing regulation for superstores has been loosened or abolished in most countries. Many of the later empirical studies focused on analyzing the effects of relaxed regulations. Some theoretical studies delved into business restrictions (Skuterud, 2005). Moreover, a majority of theoretical studies analyzed the economic effects of eased restrictions. For instance, the analyses of Inderst and Irmen (2005) and Shy and Stenbacka (2008) concluded that the operating hours of supermarkets under relaxed regulations were determined by consumers' available time, preference of weekday shopping, and supermarkets' operating expenses. The analyses also revealed that eased regulations would enhance social welfare.

The mandatory closing regulation in Korea, which was initially implemented to protect SMS and traditional markets, has invited many discussions on its effectiveness. Quite a few domestic studies have been conducted on the relationship between the mandatory closing regulation and the domestic retail industry. Choi and Jeong (2016) analyzed the effect of business restriction based on the daily sales data of superstores and SSM. The study confirmed that about 16–17% of the sales of superstores and SSM had been transferred to traditional markets. Using the price data of 300 SMS in Seoul, Chung et al. (2018) analyzed the potential competition between giant stores and SMS. The results revealed no rivalry between them with one type of retailer not serving as a substitute for the other.

Most preceding studies relied on the sales data for analysis. In other words, they analyzed ex-post measure in assessing the effectiveness of the policy. However, the effect of the sales data cannot be reasonably measured because of macroeconomic factors, changes in distribution structure, and other unobserved elements. Therefore, a demand-oriented approach, rather than supply-oriented one, is needed to properly address the policy effect. As many previous studies suggest that consumers’ behavior can be affected by external and internal changes, it is important to understand how consumers respond to the policy. (Tham et al., 2019; Prashar et al., 2015b; Ryu & Bringhurst, 2015). As this regulation indirectly supports small retailers by restricting consumers’ behavior, research on consumer responses to the regulation is essential.

That is, future studies should place a heavy emphasis on identifying changes in consumers' choice for a place to shop and in their purchasing behavior based on what they buy at each retailer. A close examination of the effects of regulatory policies, while looking at the changes in consumers' retail-store-visit patterns and spending patterns, if any, following the mandatory closing regulation, is believed to help predict regulatory policy-driven changes in the retail industry and draw practical and policy implications.

3. Data and Research Methodology

3.1. Research Questions

This paper intends to analyze how the mandatory closing regulation changes consumers' spending patterns. For this objective, two types of consumers' “spending patterns” are defined, and the respective patterns before and after the regulation are compared to confirm consumers' reaction to the mandatory closing system. In this paper, spending patterns are defined as follows: (i) consumers' choice of retailer, a spending pattern where a specific retailer is selected and visited, and (ii) the composition of shopping basket by store, a spending pattern where specific lines of goods are purchased depending on the type of store. Based on these definitions, this study aims to answer the following two research questions:

1. Has the mandatory closing regulation changed the store of consumer's choice?

2. Has the mandatory closing regulation changed consumers' basket composition?

To answer these questions, this study identified the household spending pattern through clustering methodology and comparatively analyzed the household spending patterns before and after the regulation.

3.2. Research Designs

3.2.1. Data

This study employed the agri-food consumers' panel data of 2010–2017 provided by the Rural Development Administration. The data was compiled from the receipt-based daily agri-food household expenditures in the Seoul metropolitan area. The data also provides information on the type of store visited and purchased items. Along with this data, the information on the implementation of the ordinance by each local government in the Seoul metropolitan area regarding the mandatory closing regulation was utilized for the analysis. With different implementation dates, classifying the data for consistency was a challenging task. However, despite the difficulties involved, we collected each region’s different ordinance information and were able to categorize the enormous volume of data by successfully sorting them out.

For the two categories (consumers' retailer choice and shopping basket patterns) subject to the analysis, the raw data was converted to fit each category and analyzed. As for the pattern of consumers' retailer choice, a spending pattern defined as the proportion of each retailer – superstores, SSM, SMS, and traditional markets – out of the total monthly visits by a household was analyzed. In other words, the spending pattern shows how consumers allocate their monthly visits for grocery shopping by retailer. The rationale behind the definition of spending pattern lies in the fact that consumers visit different types of stores for food consumption instead of adhering to one type of retailer. After calculating the proportion of households' monthly visits, the proportion of each retailer by household was averaged against the sample period, thus variabalizing the percentage of a specific type of store out of the number of average monthly visits of a household for food consumption. Additionally, reflecting the differences in implementation dates of the regulation by consumers' region, two subsets of the data were compiled to separate the pre-regulation visits from the post-regulation visits. To track the changes in consumers' visiting patterns, the subjects of the analysis were confined to those households whose purchasing records existed for both pre-regulation and post-regulation periods. Furthermore, for consistency, the analysis narrowed down the regions to those areas where the mandatory closing days fell on the second and fourth Sundays. Finally, after filtering, a total of 515 households became available for the analysis. Table 2 displays the descriptive statistics of 504 households whose records were included in the 2017 baseline data, the most recent year of the sample period.

Table 2: Descriptive Statistics

Next, the pattern of basket composition is defined as the combination of product categories purchased by households at each type of store. The monetary weight of each of the five categories (processed food, vegetables, fruits, fishery products, and livestock products) that consumers purchased during one visit at a specific retailer was variabalized. A total of 439,165 purchases of the five categories, including those purchases of at least one category, were employed for the analysis. When a consumer visited a specific retailer twice on the same day, two visits were treated as one; the purchase amounts from each visit were added up. As with the data on consumer's retailer choice, the basket pattern data was divided into two subsets, one for the pre-regulation period and the other for the post-regulation period, by taking into account the purchase dates and regions where purchases were made. The distribution of purchases by retailer for each period is shown in Table 3.

Table 3: Distribution of Purchase data

3.2.2. Methodology

Clustering, one of the unsupervised machine learning techniques that classify and group samples with similar characteristics based on the given data, is mainly utilized to divide consumers into various groups. In particular, purchase records make it possible to classify consumers into different groups depending on their spending patterns. Since clustering has many advantages, such as its ease of use when identifying similar patterns based on the data, it is frequently utilized in recent studies in social science, such as Shadkam (2014) and Jeong (2017). Out of the various clustering methodologies, this study employed K-means Clustering, a technique used to classify the given data into a fixed number (K) of clusters that do not overlap, to look for K number of centroids and group data points nearest to each centroid together, forming a cluster. The number of centroids is located through iterative algorithms. This data clustering generates data points that are similar to other data points within a cluster, but that set themselves apart from data points in other clusters. In this context, similarity and dissimilarity between datasets are calculated based on Euclidean distance. In this paper, the optimum number of clusters was identified using the elbow point method (Murphy, 2012) for the most effective application of clustering to the given datasets.

Taking advantage of consumers' spending patterns derived from the clustering technique, we conducted a comparative analysis for the changes in spending patterns before and after the mandatory closing regulation. First, the results of consumer clustering based on consumers' visiting patterns were compared for pre and post-regulation periods. For this, a transition matrix was constructed to confirm whether households' visiting patterns remained the same or shifted to other types of stores following the implementation of the regulation. The changes in spending patterns before and after the regulation were successfully traced, thanks to the panel data employed in this study.

Next, to identify whether any changes occurred in households' basket patterns at each store following the regulation, the distribution of basket clusters at each retailer was examined. Then, a comparison was established by looking at how the basket patterns derived from clustering were distributed across retailers.

4. Estimation Results

4.1. Changes in Consumers’ Choice of Retailer

4.1.1. Clustering Results

Consumers were grouped into five clusters for each pre-regulation and post-regulation dataset. Depending on the weight of consumers' choice of retailer exhibited by each cluster, we arbitrarily assigned cluster names. When the location of a cluster centroid is extremely lopsided toward a specific type of store, that type of retailer was named after the cluster of consumers who frequented the retailer.

As a result, the five consumer clusters, classified based on the pre-regulation dataset, are as follows: superstore-oriented cluster, SSM-oriented cluster, SMS-oriented cluster, a traditional market-oriented cluster, and a traditional market and SMS-oriented cluster. A majority of households visited SMS more often than the other types of retailers, which was believed to be attributable to easy access (Table 4). The post-regulation dataset displayed similar spending patterns in all the clusters except for one. Before the regulation, both a traditional market-oriented cluster and an SMS-oriented cluster existed, but after the regulation, a new type of cluster emerged where consumers, although mainly purchasing at traditional markets, made fewer visits to traditional markets compared with the number of visits exhibited in the traditional market-oriented cluster. To distinguish two traditional market-oriented clusters, we assign names of two clusters TM-oriented (High intensity) and TM-oriented (Low intensity) respectively based on the percentage of visiting traditional markets per month (Table 5).

Table 4: Clustering results based on pre-regulation dataset (Retail Choice patterns)

*WCSS: the sum of squared average distance of all the points within each cluster to its cluster centroid

** \(\mathrm{R}^{2}=\frac{\text {Between } \mathrm{SS}}{\text { Total } \mathrm{SS}}\), where Between Sum of Squares(Between SS.) is the squared average distance between all centroids and Total Sum of Squares(Total SS.) is the squared average distance between all points to the centroid of all points.

Table 5: Clustering results based on post-regulation dataset (Retail Choice patterns)

 *WCSS: the sum of squared average distance of all the points within each cluster to its cluster centroid

** \(\mathrm{R}^{2}=\frac{\text {Between } \mathrm{SS}}{\text { Total } \mathrm{SS}}\), where Between Sum of Squares(Between SS.) is the squared average distance between all centroids and Total Sum of Squares(Total SS.) is the squared average distance between all points to the centroid of all points.

4.1.2. Post Clustering Analysis Results

In Table 6, each row and each column in the transition matrix are the household visiting patterns before and after the regulation, respectively. Out of the 515 households analyzed, 37 households were involved in the shift from superstore or SSM-oriented visiting patterns to SMS or traditional market-oriented ones. This is a positive change intended by the policy, representing 23.3% (17 out of 73) of superstore-oriented households and 33.3% (20 out of 60) of SSM-oriented households. The transition effects to SMS is larger than those to traditional markets.

Table 6: Transition Matrix (%)

However, during the same period, consumers who originally preferred small and medium sized supermarkets or traditional markets moved to the superstore- or SSM- oriented patterns. It accounted for 57 of the 515 (11%) households. This change is due to a change in the trend of the retail market. During the sample period, from 2010 to 2017, superstores and SSMs have steadily increased the number of stores, while small and medium sized supermarkets and traditional markets have reduced the number of stores (Yoo & Lee, 2011).

Despite these environmental changes in the retail industry, the promising result was that some of the consumer groups that used superstores and SSMs were moved to consumer groups, which favored smaller retailers after the regulation. That means the positive effect we estimated is likely to be lower bound of the actual effect. In other words, if we control the time trend shared by all consumer groups and estimate the actual effect, the positive transition effect is expected to be higher.

4.2. Changes in Consumers’ Shopping baskets

4.2.1. Clustering Results

All the purchase items were divided into five clusters for each pre-regulation and post-regulation dataset. Similar to the analysis in Section 4.1, the names that represent each cluster's trait were assigned depending on the location of a cluster centroid. Based on the results of the clustering analysis, consumers' purchased items were grouped into five most bought patterns of shopping basket. However, these patterns appeared unaffected by the implementation of the regulation. Before the regulation, processed food accounted for the largest percentage of shopping basket. The clustering results for the pre-regulation and post-regulation datasets are presented in Tables 7 and 8, respectively.

Table 7: Clustering results based on pre-regulation dataset (Shopping basket patterns)

 *WCSS: the sum of squared average distance of all the points within each cluster to its cluster centroid

** \(\mathrm{R}^{2}=\frac{\text {Between } \mathrm{SS}}{\text { Total } \mathrm{SS}}\), where Between Sum of Squares(Between SS.) is the squared average distance between all centroids and Total Sum of Squares(Total SS.) is the squared average distance between all points to the centroid of all points.

Table 8: Clustering results based on post-regulation dataset (Shopping basket patterns)

 *WCSS: the sum of squared average distance of all the points within each cluster to its cluster centroid

** \(\mathrm{R}^{2}=\frac{\text {Between } \mathrm{SS}}{\text { Total } \mathrm{SS}}\), where Between Sum of Squares(Between SS.) is the squared average distance between all centroids and Total Sum of Squares(Total SS.) is the squared average distance between all points to the centroid of all points.

4.2.2. Post Clustering Analysis Results

The results in Section 4.2.1 reveal that no changes occurred in overall basket patterns. Therefore, a subsequent study was carried out to identify whether any changes occurred in the basket pattern by retailer. In other words, an additional study delved into the effect of the regulation on the line of goods that consumers used to purchase at a specific type of store. For this study, spider charts (Figure 1) were prepared to display two sets of basket cluster distribution for each period.

Each chart in Figure 1 indicates the percentage distribution of five basket clusters at a specific type of store on five axes, linking them to create a pentagon. The nearer a vertex is to a centroid, the fewer consumer purchases are in that basket pattern. Consequently, a pentagon that is evenly spread outside means that consumers of diverse shopping patterns visited that retailer.

When examining the distribution of basket patterns by retailer, a similarity was witnessed between superstores and SSM, and SMS. The processed food-oriented basket pattern took the major share (approximately 60%), whereas the basket pattern for fresh food including vegetables and fruits accounted for relatively low proportion. By contrast, traditional markets differentiated themselves from the other three retailers in terms of basket pattern. The basket pattern of traditional markets is characterized by a low proportion of processed foods with the patterns of livestock products, vegetables, fruits, and fishery products displaying higher weights. This is due to the characteristics of traditional markets. A traditional market is composed of independent specialized stores for each product group, unlike other types of retail stores. However, the product groups sold by stores in traditional markets are not evenly distributed. In particular, the portion of stores selling processed foods in traditional markets is not high. According to the survey on management of traditional markets, shopping malls and stores conducted by the Small Enterprise and Market Service, in 2017, the number of stores selling processed foods in traditional markets in Seoul accounted for 5.9% of the total (10.9% in Gyeonggi-do and 7.1% in Incheon). This is low compared to the share of other food categories. Therefore, the reason why the purchase of processed foods from traditional markets is lower than that of other retailers is because there are not many stores selling processed foods in traditional markets.

This suggests that the policy may not sufficiently meet the consumer's diverse food purchase demand in the traditional market even if they move to the traditional market. This can be the reason why the transition effects to traditional markets is not significant in Section 4.1.2. Since the basket pattern of consumers has not changed since the implementation of the regulation, it can be inferred that the traditional market does not fully meet the needs of consumers who want to buy a variety of products. This implies that the regulations should not only include schemes to drives consumers to traditional markets, but also include incentives for consumers to buy a variety of product groups available in traditional markets.

5. Conclusions

This study examined the effect of the mandatory closing regulation on consumers' purchasing patterns through clustering methodology. By defining consumers' spending patterns as (1) the pattern of consumers' retailer choice and 2) the pattern of consumers' shopping basket by store type, this paper focused on confirming whether any changes in consumers’ consumption patterns occurred following the regulation. Thus, the effect of the mandatory closing regulation was identified. The analytical results revealed that the regulation partly changed the pattern of consumers' choice of retail shops. Specifically, it was confirmed that 20.3% and 33.3% of superstore-oriented households and SSM-oriented households, respectively, changed their shopping places to SMS or traditional markets. The transition effects to SMS is larger than the transition effects to traditional markets. The analytical results for the effect of the regulation on consumers’ basket patterns showed that the pattern of already established product categories of consumer purchase at a specific retailer is similar across all types of retailers (superstores, SSM, and SMS) except for traditional markets. This can be the reason why the transition effects to traditional markets is not significant. Additionally, the regulation caused transfers of some households to SMS and traditional markets, but it did not change consumers' basket patterns by retailer. Despite a slight increase in the proportion of the processed food-oriented basket pattern at traditional markets, the policy fell short in making traditional market-oriented consumers exhibit purchasing patterns similar to those shown at the other retailers. This is because the variety of stores located in the traditional market is insufficient for the traditional market to satisfy consumers' various purchase needs.

Figure 2: Distribution of shopping basket clusters by type of retailer

In line with the mandatory closing regulation's objective of protecting SMS and traditional markets, the analytical results of this study suggest that the authorities assist traditional markets in their efforts of diversifying product categories to enable superstore or SSM-oriented consumers to shop at traditional markets. The three type of retailers, in which consumers showed a different purchasing pattern than the traditional market, have the advantage of being able to do "one-stop shopping". In other words, consumers can purchase various categories of products at one store. If the diversity of the product groups sold by traditional markets does not improve, the effect of the regulation can be limited. On top of the effect of temporarily transferring consumers by restricting the business of superstores and SSM, future policies, to maximize their long-term effects, should be geared toward transforming traditional markets so that consumers can maintain their superstore and SSM-oriented spending patterns even when they visit traditional markets.

To this end, in order to diversify categories of products in the traditional market, policies such as initial cost support and tax incentives are needed so that various stores can be located in traditional markets. Choi and Suh (2017) suggests that government support for small business owners has positive effect on their business performance. In addition, product-specific discounts and voucher systems should be expanded to allow consumers to purchase more products such as processed foods in addition to vegetables and fruits that were previously purchased in traditional markets. Prashar et al. (2015a) suggests that discount offerings can impulse consumers’ buying behavior. Finally, traditional market merchants should strive to diversify their products in individual stores, and to modernize and improve the service environment to be competitive with other retail businesses. The price of commodities, quality of goods in a retail store can have positive effect on consumers’ revisit intention. (Kim, 2013; Kim, 2017).

This paper differs from previous studies in that it analyzed the effects of the superstore mandatory closing regulation, which had many discussions of effectiveness, in terms of changes in consumer behavior. This paper also contributes to this field by verifying the policy effect using machine learning technique. The findings presented in this study have academic and policy implications in that they have laid the foundation for future policy improvement and further policy development. However, there is a limitation that only households living in the metropolitan area are analyzed. It is expected that more accurate policy effects can be identified if data from the nationwide households and longer periods of data are obtained in the future.