Correlation Analysis of Atmospheric Pollutants and Meteorological Factors Based on Environmental Big Data

Abstract

With the acceleration of urbanization and industrialization, air pollution has become increasingly serious, and the pollution control situation is not optimistic. Climate change has become a major global challenge faced by mankind. To actively respond to climate change, China has proposed carbon peak and carbon neutral goals. However, atmospheric pollutants and meteorological factors that affect air quality are complex and changeable, and the complex relationship and correlation between them must be further clarified. This paper uses China's 2013-2018 high-resolution air pollution reanalysis open data set, as well as statistical methods of the Pearson Correlation Coefficient (PCC) to calculate and visualize the design and analysis of environmental monitoring big data, which is intuitive and it quickly demonstrated the correlation between pollutants and meteorological factors in the temporal and spatial sequence, and provided convenience for environmental management departments to use air quality routine monitoring data to enable dynamic decision-making, and promote global climate governance. The experimental results show that, apart from ozone, which is negatively correlated, the other pollutants are positively correlated; meteorological factors have a greater impact on pollutants, temperature and pollutants are negatively correlated, air pressure is positively correlated, and the correlation between humidity is insignificant. The wind speed has a significant negative correlation with the six pollutants, which has a greater impact on the diffusion of pollutants.

1. Introduction

Urban air pollution is an important part of national environmental protection. With the acceleration of my country’s urbanization process, urban environmental protection has been playing an increasingly important role in national environmental protection. The prevention and control of air pollution is not only a major livelihood issue, but also an important starting point for economic upgrading [1]. In 2021, the "Three-Year Action Plan for the Defense of the Blue Sky" issued by the State Council officially came to an end. The national air quality continued to improve, showing a steady and positive trend. However, the effectiveness of atmospheric governance is still unstable, especially since the autumn and winter. Haze weather appears again in some cities, and the increasingly prominent regional compound air pollution problem is formed over a long period of time. The concentration of air pollutants is also affected by many elements such as region, economy, meteorology, etc., There is a large gap in the distribution space of the same period [2, 3].

In recent years, domestic and foreign scholars have carried out a large number of studies on air pollution. Studies by Tan Lina, Fang Xiaoting and others have shown that there is a high correlation between the distribution of air pollutants and surface meteorological factors, and there is a clear relationship with seasonal changes. Summer air quality is obviously better than winter [4, 5]. Yang Liu et al. analyzed the variation characteristics of air pollutant concentration in Chengdu from 2014 to 2016 and the impact of precipitation on it [6]. Cheng Linjun used the Rotating Empirical Orthogonal Function (REOF) to analyze the temporal and spatial variation characteristics of the ozone concentration in Chinese cities in 2016 [7]. Liu Hui, Xiao Kai and others used PSCF and CWT analysis methods to simulate and analyze the pollution transmission path of the city, and found that the external pollution input has an important impact on the air quality of the city, and the air pollution similarity between the city and the surrounding area is relatively high [8, 9]. Jiang Qiqing and others also analyzed and studied the characteristics of atmospheric particulate pollution and the potential sources of PM2.5 in the urban area of Hangzhou [10].

The atmospheric pollutants and meteorological factors that affect air quality are complex and changeable. The compound relationship and correlation between each other need to be further clarified. As a statistical calculation method, the Pearson Correlation Coefficient (PCC) can be used to quantitatively measure the relationship between variables. Correlation, in recent years, has been applied to the field of environmental governance by many scholars, and has become one of the commonly used methods to study the correlation between air pollution indicators [11, 12]. Wang Geng et al. used the Pearson correlation coefficient to calculate the changes in pollution characteristics in Zhangqiu District and the correlation with meteorological factors [13]. Liu Qing et al. pointed out that the ozone concentration is negatively correlated with other pollutants, which is heavily affected by the topographical features [14]. Liu Zihao and others analyzed the temporal and spatial changes of air pollution in Hefei and the correlation with meteorological factors, and pointed out that air pollutants are negatively correlated with temperature and sunshine, positively correlated with air pressure, and have no obvious correlation with humidity and wind speed [15]. The above scholars' research is constrained by the limitation of the data source monitoring area and time interval, the incomplete pollution indicators, and the sample data volume is too small. This article will use Kong Lei, Tang Xiao and others' newly opened China 2013-2018 high-resolution air pollution reanalysis open data set [16], through the correlation calculation and correlation of environmental air pollutants and meteorological factors in cities of all levels and kinds in China. Statistical analysis, the use of big data visualization tools to visualize data on time-space series, to further explore the mechanism of the concentration of different pollutants, provide a reference for studying the influencing factors of air pollution indicators, and use air quality routines for environmental management departments Monitoring data facilitates dynamic decision-making and advances research on global climate governance.

2. Data source and system architecture

2.1 Data source and data format

This experiment uses China’s 2013-2018 high-resolution air pollution reanalysis open data set [14], which is jointly authoritatively released by the Institute of Atmospheric Physics, Chinese Academy of Sciences and other units. Air quality data with time series characteristics, including daily average data and hourly data, stored in .csv format, data information includes [PM_2.5, PM₁₀, SO₂, NO₂, CO, O₃, U, V, TEMP, RH, PSFC, lat, lon], the first six items are conventional pollutants, the middle five items are atmospheric and meteorological factors, and the last two items are latitude and longitude coordinates.

Secondly, select the geographic data provided by Alibaba Cloud DataV from the Gaode Maps Open Platform (http://datav.aliyun.com/tools/atlas) updated in December 2020, and use the DataV Atlas map selector to obtain 34 administrative offices across the country. The geographic data files of divisions and second-level administrative divisions are stored in. json format and used as the input of map geographic data in the system.

2.2 System architecture and design process

The system adopts front-end and back-end agile development, obtains 34 administrative divisions across the country through the inverse address coding of AutoNavi Map API, adds data dimensions such as AQI index, wind speed, wind direction, and AQI level. The front-end uses Vue, Echarts, and D3 to achieve visualization, and the back-end uses Node builds an Express server, and the database uses MongoDB for data storage. The system architecture and design process are shown in Figure 1.

Figure 1. System architecture and design process

3. Data pre-processing and algorithm model

3.1 Air Quality Index (AQI)

The Air Quality Index (AQI) simplifies the monitored air concentration into a single conceptual index value based on the proportions of various components in the air. It classifies the degree of air pollution and air quality, and is suitable for presenting the city's short-term air quality status and changing trends.

This experiment refers to the "Ambient Air Quality Standards of the People's Republic of China" (GB 3095-2012) for a unified measurement of pollutants, Combining the air quality sub-index in the standard and the corresponding pollutant item concentration limit, Calculated according to the measured concentration values of fine particulate matter (PM_2.5), inhalable particulate matter (PM₁₀), sulfur dioxide (SO₂), nitrogen dioxide (NO₂), carbon monoxide (CO), ozone (O₃) and other pollutants The air quality sub-index (IAQI for short) is calculated as follows:

\(I A Q I_{p}=\frac{I A Q I_{H i}-I A Q I_{L o}}{B P_{H i}-B P_{L O}}\left(C_{p}-B P_{L o}\right)+I A Q I_{L o}\)       (1)

where 𝐼𝐴𝑄𝐼𝑝 represents the air quality sub-index of the pollutant item p; 𝐶𝑝 represents the mass concentration value of the pollutant item p; 𝐵𝑃𝐻𝑖 represents the high limit value of the pollutant concentration; 𝐵𝑃𝐿𝑜 represents the low limit value of the pollutant concentration; 𝐼𝐴𝑄𝐼𝐻𝑖 is the air quality index corresponding to 𝐵𝑃𝐻𝑖; 𝐼𝐴𝑄𝐼𝐿𝑜 is the air quality index corresponding to 𝐵𝑃𝐿𝑜.

The AQI of all districts and counties, the smallest administrative unit divided by formula 1, is calculated, and then the average value of the IAQI of each pollutant of all districts and counties under the jurisdiction of a city is calculated as the IAQI of each pollutant of the city. Take the maximum value as the city’s AQI, and the calculation formula is as follows:

\(A Q I=\max \left\{I A Q I_{1}, I A Q I_{2}, \cdots, I A Q I_{n}\right\}\)       (2)

where 𝐼𝐴𝑄𝐼 represents the air quality sub-index; n represents the number of pollutant items.

For the solution to areas above the provincial level, due to the large geographical scope, the pollutants of each province and each city are quite different. Directly taking the average value of each pollutant will make the final value lose its original meaning. The maximum value of IAQI can no longer represent the AQI situation of the province. Therefore, the average value of the AQI of all cities under the jurisdiction of the province is taken as the AQI of the province for data statistics and analysis.

After calculating the AQI, according to the air quality index classification standard Table 1, all AQI are converted into corresponding levels, namely first-level excellent, second-level good, third-level light pollution, fourth-level light pollution, and fifth-level severe pollution. Six levels of serious pollution, a total of six levels. Therefore, the AQI and corresponding levels of all provinces, cities, districts and counties have been obtained, which is convenient for reporting the impact of air quality on health and the recommended measures.

Table 1. Air Quality Index classification standards

3.2 Wind speed and direction

Wind is a vector that has both speed and direction. In order to better analyze the influence of wind on the concentration of pollutants in the area, the daily average wind speed value is calculated by the following calculation formula.

\(\text { wind } y=\sqrt{u^{2}+v^{2}}\)       (3)

In addition, the wind direction is calculated by the components of wind speed in latitude and longitude, and the wind direction of each regional point is formatted as [N, NNE, NE, ENE, E, ESE, SE, SSE, S, SSW, SW, WSW, W, WNW, NW, NNW] means true north, north northeast, northeast, east northeast, true east, east southeast, southeast, south southeast, true south, south southwest, southwest, west southwest, and true west. Northwest, Northwest and Northwest. Finally, the wind direction of each regional point is formatted and stored in a structured database in JSON format as a visual mapping data source.

3.3 Pearson Correlation Coefficient

The Pearson Correlation Coefficient (PCC) is a statistical method that can quantitatively measure the correlation between variables. It is calculated by Gaussian normal distribution data, and its value is between -1 and 1. When the absolute value is larger, the correlation is stronger. This experiment will use the Pearson correlation coefficient to measure the linear correlation between the concentrations of different air pollutants.

First, calculate the correlation between two random variables through covariance:

\(\operatorname{cov}(X, Y)=\frac{\sum_{i}^{n}\left(X_{i}-\bar{X}\right)\left(Y_{i}-\bar{Y}\right)}{n-1}\)       (4)

The Pearson correlation coefficient calculation formula is as follows:

\(\rho(X, Y)=\frac{\operatorname{cov}(X, Y)}{\sigma_{X} \sigma_{Y}}\)       (5)

Where cov(X, Y) represents the covariance between two variables, σ_X and σ_Y represent the standard deviation of the variables respectively.

It can be seen that the value of Pearson's correlation coefficient is a value between -1 and 1. When the linear relationship between the two variables is enhanced, the correlation coefficient tends to 1 or -1. When one variable increases and the other variable also increases, it indicates that there is a positive correlation between them, and the correlation coefficient is greater than 0; if one variable increases, the other variable will decrease, indicating that there is a negative correlation between them. The coefficient is less than 0. If the correlation coefficient is equal to 0, it indicates that there is no linear correlation between them.

3.4 Pearson Correlation Coefficient

By performing data pre-processing on the obtained data source as described above, the processed data structure is shown in Table 2.

Table 2. Sample data structure after processing

4. Correlation analysis and visualization

In the process of urban air pollution, it is mainly affected by the following two factors. One is the distribution and emission of various pollution sources and the mutual influence of pollutants; the other is the influence of meteorological conditions on the diffusion, dilution and migration of atmospheric pollutants.

4.1 Pearson Correlation Coefficient

Calculate the Pearson correlation coefficients among the 6 pollutants (PM2.5, PM10, SO2, NO2, CO, O3) for each city with 6-year full-level data according to the monthly granularity, and store the calculation results in The MongoDB database visually maps its temporal and spatial evolution and then dynamically displays and analyzes it. In this experiment, the Pearson correlation coefficient statistical analysis was performed on the daily average data of Sichuan Province in March 2018. The calculation results are shown in Table 3:

Table 3. Correlation coefficients among pollutants in Sichuan Province in March 2018

The statistical results of the Pearson correlation coefficients among the six pollutants in Sichuan Province in March 2018 are shown in Figure 2 using chord diagrams for big data visualization.

Figure 2. Visualization of pollutant correlation (chord diagram)

Combined with the data analysis in Table 3 and Figure 2, it can be obtained that the ozone in the air pollution index of Sichuan Province in March 2018 and the other five types of pollutants all show different degrees of negative correlation, while the other two pollutants show different degrees of negative correlation. Significant positive correlation.

Between any two groups of pollutants, the largest absolute value of the correlation coefficient is fine particulate matter and inhalable particulate matter, and the Pearson correlation coefficient reached 0.99. The concentration of fine particles and sulfur dioxide, nitrogen dioxide, and carbon monoxide all show significant correlations, and the Pearson correlation coefficients are 0.81, 0.84, and 0.67, respectively. Based on practical experience, we know that the man-made source of sulfur dioxide in the atmosphere is the combustion of sulfur-containing coal, the source of nitrogen oxides in the urban atmosphere is mainly emitted from mobile combustion sources such as motor vehicles, and carbon monoxide mainly comes from incomplete combustion of fuels. The emission of these gaseous pollutants is often accompanied by the emission of other gaseous pollutants and particulate matter. Therefore, they have a positive correlation with each other and between fine particulate matter and inhalable particulate matter.

Ozone is a secondary pollutant with strong oxidizing ability and is a representative pollutant of photochemical smog. The concentration of ozone and other pollutants all show a significant negative correlation. When the concentration of particulate matter in the atmosphere rises to form aerosols, ultraviolet radiation can be reduced, thereby reducing the formation of ozone. Therefore, the concentration of fine particulate matter and inhalable particulate matter are both related to The ozone concentration is negatively correlated. In addition, carbon monoxide and nitrogen dioxide are precursors in the formation of low-altitude ozone, and the formation of ozone needs to consume these precursors. At the same time, the strong oxidizing ozone can oxidize sulfur dioxide and reduce the concentration of sulfur dioxide in the atmosphere.

Through experiments, the evolution of the correlation between air pollutants in the time-space sequence of each city can be calculated separately, and the factors that affect the weak correlation of the air pollution indicators in these cities can be spied to provide environmental protection departments for the formulation of prevention and control measures and the development of dynamic decision-making. convenient.

4.2 Correlation analysis between pollutants and meteorological factors

In the same way, in the experiment, the meteorological factors (TEMP, RH, PSFC, WINDY) of air pollution in Sichuan Province in March 2018 and the six pollutants in the same period were statistically analyzed by the Pearson correlation coefficient during the experiment, and the calculation results As shown in Table 4.

Table 4. Correlation coefficients between pollutants and meteorological factors in Sichuan Province in March 2018

The statistical data of Pearson correlation coefficient between pollutants and meteorological factors in Sichuan Province in March 2018 is shown in Figure 3 using radar charts for big data visualization. Among them, r=-1 is the center of the circle, which means a complete negative correlation, and r=1 is the corresponding point on the arc, which means a complete positive correlation.

Figure 3. Visualization of the correlation between pollutants and meteorological factors (radar chart)

In combination with the data analysis in Table 4 and Figure 3, we can get that there are varying degrees of correlation between the meteorological factors (temperature, humidity, pressure, wind speed) affecting air pollution in Sichuan Province in March 2018 and the six conventional pollutants.

As shown in the upper left of Figure 3, temperature has obvious effects on the six pollutants, and there is a linear relationship between the ozone concentration in the atmosphere and the temperature. The temperature and other pollutants are negatively correlated, so it can be concluded that the increase in temperature can improve pollution. This is because that the temperature is conducive to the vertical flow of hot air on the ground, as well as to the diffusion of pollutants. Due to the relatively good atmospheric diffusion conditions in summer and more rainfall, it has a certain effect on removing atmospheric pollutants. In winter, the frequency of adverse weather conditions such as temperature inversion is higher, which makes it difficult for pollutants to diffuse. This is also better than summer air quality. One reason for winter. It can be seen from the upper right of Figure 3 that the relative humidity has a small effect on atmospheric pollutants. Compared with PM2.5, PM10, CO, and O₃ are positively correlated, and slightly negatively correlated with SO₂ and NO₂, because the increase in humidity will dilute the pollution. The concentration of the substance. It can be seen from the lower left of Figure 3 that the atmospheric pressure has a weak positive correlation with the concentration of other pollutants. It can be seen from the lower left of Figure 3 that wind speed is negatively correlated with most pollutants, and slightly positively correlated with CO. It can be seen that wind speed has a close relationship with the diffusion of air pollution, and the direction and speed of propagation need to be further studied.

5. National AQI visualization system and correlation analysis examples

In order to further explore the evolution of the Air Quality Index (AQI) and various pollutants and meteorological factors in various time and space series across the country, to find the pollution types and causes of the most polluted cities in each period, and to analyze the effects of various air pollution indicators. Correlation, through a large number of complex calculations, the national air quality visual analysis system has been realized.

5.1 National air quality visual analysis system

After the data pre-processing in the third section, in the experiment, the national pollution situation was visualized and mapped to facilitate the analysis and comparison of the spatial and temporal evolution of air pollution in various cities. Among them, refer to my country's "Ambient Air Quality Standards" (GB 3095-2012). The air quality index (AQI) classification standard and color scheme in) realize the national air quality visual analysis system as shown in Figure 4.

Figure 4. National air quality visual analysis system

Through the visual analysis system and Figure 4, it can be seen that in the past 6 years in my country, January has been the most serious stage of air pollution, and the air quality in Southwest my country is better than that in Northeast China. In January 2017, air pollution in the central and eastern regions was severe. The Beijing-Tianjin-Hebei provinces in North China, the three northeastern provinces, central Shanxi, Henan, Hubei, Hunan, Anhui, and some cities in the eastern coastal provinces and cities all underwent moderate or severe pollution.

1) System visualization: the user can select the time/region module in the left control panel area, select the province, city and date that need to be queried, and then the air pollution and meteorological factor data overview of the city can be displayed on the lower left side; the middle view area dynamically displays the whole country The pollution distribution of AQI cities; the right side displays the most severely polluted cities with the air quality index in the form of TOP10 rankings; the lower side adds a timeline selection function to the map, and supports sliding to view the national AQI distribution on specific days of the month.

2) System interaction design: Each view is designed and implemented with mouse hovering and sliding to update the control panel data, mouse click on the view to drill down and return operations, and support scroll wheel zoom, drag and drop operations.

5.2 Examples of correlation analysis of heavily polluted cities

In order to further study the correlation among the various indicators affecting air pollution, the experiment once again showed the pollutants and the pollution of the top three most polluted cities (Tianjin, Shandong, Henan) in January 2017 in the country. Correlation analysis between objects and meteorology. Through the statistical analysis and visualization of the Pearson correlation coefficient on the spatiotemporal series, as shown in Figure 5.

Figure 5. Correlation comparison chart of TOP3 cities

Experimental data shows that most of the correlations among the six pollutants in Tianjin, Shandong and Henan provinces are positive correlations, and the three provinces all show negative correlations between O₃ pollutants and NO₂ pollutants. This is because that nitrogen oxides are the main gas causing photochemical smog and ozone layer depletion. In terms of the correlation between meteorological conditions and the concentration of pollutants, the three regions have similar correlations. From the geographical point of view, the three regions of Tianjin, Shandong and Henan are adjacent to each other in the order of above, middle and lower. They are located in northern China. Due to the dry climate, less wind and less rain, they are not conducive to the diffusion of pollutants. In the case of strong air currents, pollutants can stay in the air for a long time. According to Figure 4 and Figure 5, it can be seen that the wind direction in Shandong Province is southwest wind, and the wind speed on the day has a significant impact on pollutants. There is a possibility that the wind may cause pollutants to migrate. It can be seen that the spread of pollutants has a greater relationship with wind speed and wind direction, and the pollution between neighboring provinces and cities may be affected.

6. Conclusions

Through the 6-year full-level data of each city according to the monthly granularity, the daily average data is calculated between the 6 pollutants (PM2.5, PM10, SO₂, NO₂, CO, O₃), and the meteorological factors (TEMP, RH, PSFC, WINDY) and the Pearson correlation coefficient of pollutants are statistically analyzed. The experimental results show that the pollutants are positively correlated except for ozone, which is negative; meteorological factors have a greater impact on pollutants, and temperature is related to pollutants. It which is in negative correlation, air pressure is positively correlated, and humidity is not significant. Wind speed has significant negative correlation with the six pollutants, which has a greater impact on the diffusion of pollutants.