DOI QR코드

DOI QR Code

Sentiment Analysis on Indonesia Economic Growth using Deep Learning Neural Network Method

  • KRISMAWATI, Dewi (Directorate of Statistical Analysis and Development, BPS Statistics Indonesia) ;
  • MARIEL, Wahyu Calvin Frans (Directorate of Statistical Analysis and Development, BPS Statistics Indonesia) ;
  • ARSYI, Farhan Anshari (Directorate of Statistical Analysis and Development, BPS Statistics Indonesia) ;
  • PRAMANA, Setia (Department of Computational Statistics Politeknik Statistika STIS)
  • Received : 2022.03.16
  • Accepted : 2022.06.08
  • Published : 2022.06.30

Abstract

Purpose: The government around the world is still highlighting the effect of the new variant of Covid-19. The government continues to make efforts to restore the economy through several programs, one of them is National Economic Recovery. This program is expected to increase public and investor confidence in handling Covid-19. This study aims to capture public sentiment on the economic growth rate in Indonesia, especially during the third wave of the omicron variant of the covid-19 virus, that is at the time in the fourth quarter of 2021. Research design, data, and methodology: The approach used in this research is to collect crowdsourcing data from twitter, in the range of 1st to 10th October 2021. The analysis is done by building model using Deep Learning Neural Network method. Results: The result of the sentiment analysis is that most of the tweets have a neutral sentiment on the Economic Growth discussion. Several central figures who discussed were Minister of Coordinating for the Economy of Indonesia, Minister of State-Owned Enterprises. Conclusions: Data from social media can be used by the government to capture public responses, especially public sentiment regarding economic growth. This can be used by policy makers, for example entrepreneurs to anticipate economic movements under certain conditions.

Keywords

1. Introduction

It’s been two years after the discovery of the Covid-19 case at the end of 2019 in Wuhan, a new variant of the coronavirus, namely Omicron, has become a new concern for the world. The government around the world is still highlighting the effect of the new variant of Covid-19 Omicron on economic recovery. Nevertheless, the Indonesian government remains optimistic that Indonesia's economic recovery and growth. It is supported by Indonesia's success in passing the peak the delta variant of Covid-19 cases which is a threat to economic growth in the third quarter of 2021.

Indonesia's economic growth in Quarter II-2021 grew by 7.07% (year on year), the highest in the last 16 years (BPS, 2021). This also recorded the highest quarterly growth record since the last crisis in Indonesia. This growth was achieved at a time when Active Cases of Covid-19 in Indonesia were still very high. The COVID-19 pandemic that has occurred globally has had a huge impact on the lives of all people in the world (Boshkoska, 2020). The Indonesian government, through the Minister of Education and Culture, acknowledged that the Covid-19 pandemic was a difficult period where it had an impact on the health crisis, economic crisis, and education crisis (Dikti, 2020). The Indonesian government as the public sector issued several policies to mitigate the impact of COVID-19 faced by the community. Indonesia's economic growth figures in Quarter II-2021 are a signal that the economic recovery in Indonesia will continue.

The Indonesian economy based on the amount of Gross Domestic Product (GDP) based on current prices in the third quarter of 2021 reached IDR 4, 325.4 trillion or based on constant 2010 prices reaching IDR 2, 815.9 trillion. Indonesia's economy in the third quarter of 2021 compared to the third quarter of 2020 grew by 3.51 percent (y-on-y). In terms of production, the Health Services and Social Activities Business Field experienced the highest growth of 14.06 percent. From the expenditure side, the Export Component of Goods and Services experienced the highest growth of 29.16 percent. Up to quarter III-2021, the Indonesian economy grew by 3.24 percent (c-to-c). In terms of production, the largest growth occurred in the Health Services and Social Activities Business Field of 9.81 percent. Meanwhile, in terms of expenditure, all components grew, the highest growth occurred in the Export Component of Goods and Services by 22.23 percent. Economic growth (y-on-y) in the third quarter of 2021 experienced an increase in almost all regions, except for groups on the islands of Bali and Nusa Tenggara which experienced a growth contraction of 0.09 percent. However, Java Island with a contribution of 57.55 percent recorded a growth of 3.03 percent (BPS, 2021).

Spatially in quarter III-2021, the structure of the Indonesian economy is influenced by the group of provinces in Java, which contributes to GDP of 57.55 percent; Sumatra Island by 21.95 percent; Kalimantan Island by 8.32 percent; Sulawesi Island by 6.98 percent; the islands of Bali and Nusa Tenggara by 2.75 percent; and the islands of Maluku and Papua by 2.45 percent. The second wave of the COVID-19 pandemic triggered the Implementation of Community Activity Restrictions (PPKM) during the third quarter of 2021 which resulted in slowing economic growth in several island groups. Economic growth (y-on-y) for the group of provinces by island, respectively, namely Maluku and Papua, grew by 9.15 percent; Sulawesi Island 4.58 percent; Kalimantan Island 4.52 percent; Sumatra Island 3.78 percent; and Java Island 3.03 percent. Meanwhile, the groups of provinces in Bali and Nusa Tenggara experienced a growth contraction of 0.09 percent.

The government continues to make efforts to restore the economy through several programs, one of which is the National Economic Recovery (PEN). Through this program, it is hoped that it will increase public and investor confidence in handling COVID-19. Based on GDP data, this economic growth is supported by growth in terms of business (supply) and expenditure (demand). From the business sector, all sectors experienced growth. The sector that contributed the largest GDP growth was the Manufacturing Industry, which was 19.29%, and grew by 6.58% (y-on-y). Other major sectors also grew significantly, including the Transportation and Warehousing Sector which grew by 25.10% (y-on-y) and the Accommodation and Food and Beverage Sector which grew by 21.58% (y-on-y). From the demand side, the Government's commitment to PEN encourages government consumption to grow to 8.06% (y-on-y). Several sectors that support activities amid the Covid-19 pandemic have also continued to strengthen growth, such as the information and communication sector and health services. We also see that the agricultural sector is still consistently growing during the pandemic and plays an important role in Indonesia's food security.

Based on the explanation above, this study aims to capture public sentiment on the economic growth rate in Indonesia, especially during the third wave of the omicron variant of the covid-19 virus, namely in the fourth quarter of 2021. The approach used in this research is to collect crowdsourcing data from twitter.

2. Literature Review

Big Data is a trend in the analysis of large data. Broadly, Big Data can be categorized based on its source. First, Machine Generated Data (MGD) is the biggest source of Big Data. Sensors, cameras, and satellites are continuing to produce data from terabytes to petabytes per second constantly. Second, Organization Generated Data (OGD) is the data that the organization produces is highly structured and trustworthy. From sales transaction data to digitized product patterns, we will find out whether there is non-transparency or fraud. The last, People Generated Data (PGD) or human-generated data. Nowadays, almost everyone is active on social media. People generate huge amounts of data on social media like Facebook, Twitter, and LinkedIn. Data can also be retrieved when humans share posts via blog sites, e-mail, and text messages. It also happens when humans share photos on Instagram and Pinterest to share videos via YouTube and TikTok. PGD is referred as unstructured data. In this study, PGD big data sources are used, namely crowdsourcing data sourced from twitter.

2.1. Twitter

Twitter is a microblogging service and social networking service where the users can post a short message or images known as “tweets”. Users can also interact with these tweets with like, reply, or retweet these said tweets. Twitter is one of the most popular social networking services in Indonesia. (Firmansyah & Pramana, 2018) Twitter is more open by nature than other social networking platforms, where non-registered users can search, view and read the tweets posted by their users, that is why in this study we choose twitter as our main data sources (Simanjuntak & Pramana, 2021). Twitter provides a public API to access their data free of charge, but this free tier API comes with limitations on how many tweets that we can collect. To bypass this limitation and gather the data that we need, we use web scraping techniques that gather twitter data directly from the website. To perform the web scraping we use a python library called twint. Twint is one of the libraries used for scrapping data from twitter. Twint is a widely used library because it allows users to retrieve Twitter data without the need to register to get the API. Some of the inputs that can be used include searching for specific topics, trends and hashtags (Rachman & Pramana, 2021). With all of these features we can specify the query based on our needs.

2.2. Sentiment Analysis

One of the text analyses that is quite popular and is mostly done by researchers is Sentiment Analysis. The purpose of sentiment analysis is to find out subjective public perceptions or opinions related to certain topics/issues. Sentiment analysis is a method for classifying text data and dividing it into three opinions, namely positive, neutral, and negative. There are several methods of conducting sentiment analysis, namely Machine learning approach, Lexicon-based approach, Rule-based approach, and Statistical model approach (Rachman & Pramana, 2020). In this research we used Machine Learning approach, specify Deep Learning Neural Network.

3. Research Methods and Materials

3.1. Data Collection Method

The data was collected by extracting data from twitter with keywords related to macroeconomics indicator. From several macroeconomic indicators we choose economic growth, export, import, and inflation. We use these economic indicators that released monthly by credible government statistical agencies. In addition, these indicators can describe influence market sentiment on the economy in Indonesia. The keywords used to capture the responses & opinions of the Indonesian people to the release of economic indicator numbers in this study used several Indonesian keywords, namely “Pertumbuhan Ekonomi” (Economic Growth), and all sectors that make up GDP in terms of business fields and expenditures. The tweets data taken were tweets posted on Twitter social media in the range of 1st to 10th October 2021 due to limitations in data collection. Data collection using web scraping technique using the python programming language and utilizing the twint package. The flow of Research Method can be seen in Figure 1.

Figure 1: Flow of Research Method

3.2. Data Preparation Method

After the data can be collected, the next stage is data preparation activities. It aims to prepare the data to be ready for analysis. Data preprocessing activities include deleting duplicate data or the exact same data. The preprocessing stage in this study refers to the research that has been conducted by (Mariel, Mariyah, & Pramana, 2018). The preprocessing stages carried out are:

[1]. Case Folding is the process to reduce all characters into lowercase. For example, the phrase “Teknologi Informasi dan Komunikasi” after the informasi dan

case folding will be “teknologi komunikasi”.

[2]. Stop Word Removal, is a process of omitting the meaningless words that often appear in a text such as “yang”, “di”, “ke”, “atau”, etc.

[3]. Text Formalization is the conversion of an informal word into a standard-formal word. Thus, the words that are supposed to be one thing or the same (semantically) can be equalized when the text transformation stage.

3.3. Data Analysis Method

The activity of classifying tweets or sentiment analysis in this study was carried out using Deep Learning Neural Network method. The analysis is done by building your own model based on the available training and testing data sets. Detail Model can be seen in Figure 2.

Figure 2: Model Development for Data Analysis Sequential Model with 3 Dense layers with each

Sequential Model with 3 Dense layers with each activation of relu, relu, and sigmoid

model = Sequential ()

 model.add(Dense(90, input_dim=X_len, kernel_initializer="normal", activation='relu'))

 model.add(Dense(9, kernel_initializer="normal", activation='relu'))

 model.add(Dense(3, kernel_initializer="normal", activation='sigmoid')).

Model development is done by Splitting Train with data sharing Test: Training = 75:25. Measurement of Model Performance is measured by loss = 'categorical crossentropy', optimizer = 'adam', and metrics = Accuracy, precision, Recall and AUC. Model fitting is done with epochs = 20 and batch size = 5.

4. Results and Discussion

The number of tweets collected by day from 1 to 10 October 2022 can be seen in Figure 3. In Indonesia Press Conference, the release of economic growth figures was carried out on October 7th. Based on Figure 2, the trend in the number of twitters discussing economic growth began to increase after the 7th, namely after the release of economic growth by the Indonesian government statistics agency.

Figure 3: Numbers of Tweet By Date (2022, October 1st – 10th)

In general, the topic of discussion regarding economic growth can be seen in Figure 4. The most words from each tweet are presented in the form of a word cloud. The most discussed words are “Indonesia”, “development”, “year” and “percent”.

Figure 4: Wordcloud Tweet “Economic Growth”

Every quarter, the Indonesian government announces economic growth figures at both national and regional levels. At the beginning of the COVID-19 pandemic, the world's economic growth rate fell drastically. This illustrates that the COVID-19 pandemic is affecting the economy throughout the world, especially Indonesia. At the beginning of Covid-19, in the first quarter of 2020, the economic growth rate weakened at 2.97 percent. The figure decreased sharply in quarter 2-2020, namely at -5.32 percent. This is in line with the increasing spread of COVID-19 cases in Indonesia. At this time, the government also issued a policy of limiting community activities by issuing a work from home (WFH) policy. Many offices and industries were closed, causing the economy to weaken. The figures for economic growth in Indonesia by quarter can be seen in the figure 5. However, when viewed in quarter 2-2020, the economic growth rate increased sharply, namely at 7.07 percent. People's sentiments are quite diverse.

Figure 5: Economich Growth in Indonesia by Indonesia Statictical Office

On that occasion, the author also conducted research on the number of tweets with the keyword "economic growth" which aimed to capture the public's response to the government's release figures at that time. The second quarter 2021 economic growth figures are released on August 5, 2021.An illustration of how the community responded at that time can be seen in the figure 6.

Figure 6: Fig. 6. Number of Tweet By Date (2021, August 1st -7 th)

The Indonesian government has issued a National Economic Recovery Program (PEN) policy which aims to help the community's economy from the impact of the COVID-19 pandemic. The realization of the National Economic Recovery (PEN) program in Quarter 4-2021 increased by 55.2 percent (q-to-q) and decreased by 10.1 percent (y-on-y). (Ministry of Finance, 2022). The increase in this figure also has an impact on the improvement of the Indonesian economy. Based on the picture (the word cloud of economic growth above) the community responded positively and had high hopes for the "development" of the "community", "Indonesia".

In this study, several keywords were used which were collected based on the category of GRDP of the business field and GRDP of expenditure. Indonesia refers to the grouping of business categories obtained from the United Nation. Figure 7 shows the number of tweets based on the category of business field and expenses that are used as keywords in this study. The greatest number of tweets is about economic growth. Then the business sector that was most discussed was the electricity and gas procurement sector, while the next sector that was most discussed was the Health and Social Services sector. This is in line with the Covid-19 phenomenon faced by the people of Indonesia. Where the health service sector is an essential sector that gets the attention of the government and the people of Indonesia. The next sector that is most discussed on Twitter is the Education Services sector. This is in accordance with the government's focus on handling education during the COVID-19 pandemic.

Figure 7: Number of Tweet by GDP Category

In the electricity, water and gas supply sector, there are several words that are most often mentioned, namely "waste", "pdam" and "water", can be seen in Figure 8. The spread of the Covid-19 virus in Indonesia, not only has an impact on human health, but also has an impact on the slowing down of the economy. The poor and vulnerable groups are the community groups whose economy is most affected by the spread of the Covid-19 virus. In order to maintain people's purchasing power, especially the lower classes in the midst of the Covid-19 pandemic, the Government has prepared several social protection programs, one of which is the provision of free electricity for 450 VA customers and a 50% electricity discount for subsidized 900 VA customers. The target recipients of assistance in general are quite precise, namely the poor and vulnerable, which aims to help ease the expenditure of their household needs during the current Covid-19 pandemic. In terms of implementation, the Government together with PT PLN (Persero) have been quite active in disseminating information on the mechanism for distributing free tokens to prepaid customers, namely by involving the Regional Government, especially for people who do not have telecommunication/internet access. This is an anticipatory measure if there is a new addition to the number of poor and vulnerable households affected. (Kemenkeu, 2021).

Figure 8: Number of Tweet in Electricity dan Water Category

The government has decided to continue to provide electricity sector stimulus to the public and business actors due to the Covid-19 pandemic. More than 32 million customers have received electricity stimulus until the first semester of this year. And we always support by carrying out the Government's decision to provide electrical stimulus for small communities, and business actors affected by Covid-19. (ESDM, 2021). The health protocols (prokes) that were made to minimize the spread, create a 3M lifestyle (washing hands, wearing masks, and keeping a distance) have in fact created new impacts on the environment, namely piles of medical waste such as bottles of hand sanitizer and hand soap, masks, gloves. and Personal Protective Equipment (PPE) (Astuti, 2020). One important aspect that should not be forgotten in handling this outbreak is the handling of medical waste with an infectious character resulting from patients and medical staff exposed to the virus when handling patients. Handling this infectious waste is important because it is feared that this waste could become a medium for the spread of the virus if it is not handled properly. Worldwide, it is estimated that at least 5.2 million people, including 4 million children, die each year from diseases associated with poorly managed medical waste.

The principle of preventing the transmission of infectious diseases is through breaking the host/host/host chain. Therefore, in response to the Covid-19 outbreak, the Association of Indonesian Internal Medicine Specialists (PAPDI) recommends breaking the host/host/host chain in various ways. One of the ways to break the chain of spread of the virus is by properly managing infectious medical waste according to procedures. In particular, the management of medical waste is regulated in Permen LHK No. P.56/Menlhk-Setjen/2015 concerning Procedures and Technical Requirements for B3 Waste Management from Health Service Facilities (Nugraha, 2020). In line with the current state of the Covid-19 pandemic, the focus of this review is on policies for handling medical waste with an infectious character. Referring to Permen LHK No.P.56/Menlhk-Setjen/2015, infectious waste is waste contaminated with pathogenic organisms that are not routinely present in the environment and these organisms are in sufficient quantity and virulence to transmit disease to susceptible humans. In general, this infectious medical In addition to analyzing the most discussed topics in waste is categorized as Hazardous and Toxic Waste each of the categories that make up GDP, data from Twitter (Envihsa, 2021).

The second sector most widely discussed by the public on Twitter is the education sector. The most widely used words can be seen in Figure 9, in the form of a bar chart and word cloud. The three most used words are “child”, “people” and “school”. One of the most basic effects of the Covid pandemic is the crisis in the education sector. With the pandemic the education system has changed. Prior to the COVID-19 pandemic, teaching and learning activities were carried out in schools. The Covid-19 outbreak has forced teachers and students to do online learning, or more popularly known as School from Home (SFH). At the beginning of 2022, several schools have started teaching and learning activities in schools, but at half their capacity. The thing that must be done in this SFH is to maintain health protocols in schools, both by teachers and students.

Figure 9: Number of Tweet in Education Category

The third sector that is most discussed on social media Twitter is the Health Services sector. This is closely related to the conditions experienced by Indonesia and all countries in the world, namely the COVID-19 pandemic. The Health Services Sector is an essential sector that is at the core of community health services and the front line in dealing with this pandemic. The most widely used words in the health care sector are "rumah"(home), "sakit"(sick) and "puskesmas"(local hospital). Details of the number of words that often appear in public tweets in the Health sector can be seen in Figure 10, which is presented in the form of a bar chart and word cloud.

Figure 10: Number of Tweet in Health Services Category

In addition to analyzing the most discussed topics in each of the categories that make up GDP, data from Twitter can be used to build a network representing the interactions between Twitter users, this aims to identify interactions between Twitter users. To perform the analysis, we can use the Graph Data Structure. Graph is a data visualization that is used to describe the relationship between elements of the data to be analyzed. In this study, each point or node describes the account/username of the Twitter user. If there is a relationship between users, in this case a mutual reply, a line/connection will be formed between these points. The direct graph is used if the focus of the research is to find out if the usernames are retweeting each other. However, because the focus of this research is to determine the interaction between users in the form of a one-way reply, we will use an Undirected Graph. (Euge, 2022).

4.1. Reply Spring Network Graph

There was a large discussion cluster involving figures to the conditions experienced by Indonesia and all from prabowo, the political party Gerindra, baligerindra, countries in the world, namely the COVID-19 pandemic. sultra_gerindra, and the media sympathisantimur, and The Health Services Sector is an essential sector that is at kd_kepri. In addition, there is also a discussion cluster the core of community health services and the front line in involving KompasTV, Erickthohir (Minister of State dealing with this pandemic. The most widely used words in Owned Enterprises), Airlangga_hrt (Minister of the health care sector are "rumah"(home), "sakit"(sick) and Coordinating for the Economy of Indonesia), and "puskesmas"(local hospital). Details of the number of democracymedia. The Result of Reply Spring Network words that often appear in public tweets in the Health Graph can be seen in Figure 11.

Figure 11: Reply Spring Network Graph

4.2. Reply Circular Network Graph

The more lines leading to a character, organization, or media, the higher the degree of the username. It can be seen that Gerindra and Prabowo are in the spotlight in discussions related to economic growth, followed by KompasTV, Erickthohir, and Airlangga_hrt (Minister of). The Result of Reply Circular Network Graph can be seen in Figure 12.

Figure 12: Reply Circular Network Graph

4.3. Reply Random Network Graph

The more lines leading to a figure, organization, or media, the higher the degree of involvement of the username. Gerindra and Prabowo are in the spotlight in discussions related to economic growth, followed by kompastv, erickthohir, and airlangga_hrt. The Result of Reply Random Network Graph can be seen in Figure 13.

Figure 13: Reply Random Network Graph

Based on the three graph methods above, there are three central figures in the discussion related to economic growth in Indonesia, namely kompastv, airlanggahr, and erickthohir. Kompas TV is one of the mass media that most often displays news related to economic growth. Meanwhile, Airlangga Hartarto and Erick Thohir are ministers currently serving in the Indonesian government. Airlangga Hartarto is the coordinating minister for Economic Affairs of the Republic of Indonesia, while Erick Thohir is the Minister of State-Owned Enterprises of the Republic of Indonesia

4.4. Sentiment Analysis

Based on the results of the analysis, most of the tweets have a neutral sentiment on the discussion of Economic Growth. This indicates that the discussion is only news or information about issues of economic growth in Indonesia. The percentage value of economic growth can be seen on the pie chart of Figure 14.

Figure 14: Sentiment Analysis of Economic Growth

As for the discussion Most of the tweets with positive sentiment on the discussion on Economic Growth can be seen in Figure 15. Some of the most frequently used words are related to development, people. The interesting thing is that there are many political figures and political parties whose names are mentioned, namely Subianto and Gerindra. In addition, it can be concluded that based on economic growth, it is hoped that the spirit of people's development can realize a Prosperous Indonesia.

Figure 15: Wordcloud of Tweet with Positive Sentiment of Economic Growth

The results of the word cloud analysis regarding the most frequently mentioned words in tweets with negative sentiments can be seen in Figure 16. The state, development, people, and prosperity were the most mentioned words. This indicates that the public still negatively views the development of the people to increase the rate of economic growth in Indonesia.

Figure 16: Wordcloud of Tweet with Negative Sentiment of Economic Growth

Finally, the most frequently mentioned words in tweets with neutral sentiments can be seen on word cloud according to figure 17. The most frequently mentioned words are percent, year, and Indonesia. This indicates that there are many tweets that bring up news related to economic growth figures in Indonesia.

Figure 17: Wordcloud of Tweet with Neutral Sentiment of Economic Growth

5. Conclusions

Data from social media can be used by the government to capture public responses, especially public sentiment regarding economic growth figures released by the government. This can be used by policy makers, for example entrepreneurs to anticipate economic movements under certain conditions. The number of public comments through social media channels, in this case Twitter is divided into several sentiments, namely positive, negative, and neutral.

One of the keys to success as well as limitations in text analysis is the determination of search keywords. Research from social media sources, or news portals is very dependent on the use of search keywords. Likewise, in this study, the limit of the research is the use of keywords, the research time is only limited to ten days from before the news release until after the official release from the government statistical agency. The limitation of the last research is that the neural network algorithm used still uses three dense layers.

Therefore, the next research proposal is to perform a deep learning algorithm with more layers and nodes to produce more accurate analysis results. Suggestions for further research is the need to conduct a time series analysis sentiment study, so that it can make comparisons between time series on the issue of economic growth in Indonesia.

References

  1. Alamsyah, A., & Syawiluna, M. (2021). Mapping Organization Knowledge Network and Social Media Based Reputation Management. arXiv preprint arXiv:2102.12337.
  2. Blazquez, D., & Domenech, J. (2018). Big Data sources and methods for social and economic analyses. Technological Forecasting and Social Change, 130, 99-113. https://doi.org/10.1016/j.techfore.2017.07.027
  3. BPS. (2021). Berita Resmi Statistik Pertumbuhan Ekonomi (Produk Domestik Bruto).
  4. Boshkoska, M., & Jankulovski, N. (2020). Coronavirus Impact on Global Economy. Annals of the „Constantin Brancusi" University of Targu Jiu, Economy Series,, 4, 18-23.
  5. Buntoro, G. A. (2017). Analisis Sentimen Calon Gubernur DKI Jakarta 2017 Di Twitter. INTEGER: Journal of Information Technology, 2(1).
  6. Paramartha, D. Y., Deli, N. F., Fitriyani, A. L., & Pramana, S. (2021). Tourism Resilience Process During Pandemic with Big Data Insight. Jurnal Ekonomi Indonesia, 10(3), 239-255. https://doi.org/10.52813/jei.v10i3.184
  7. Dikti. Direktorat Jenderal Pendidikan Tinggi, Riset, dan Teknologi. (2020). Tantangan Dunia Pendidikan di Masa Pandemi. Oktober 2020.
  8. Enviha. (2021). Limbah Medis Saat Covid-19. Fakultas Kesehatan Masyarakat Universitas Indonesia. 2021.
  9. Euge Inzaugarat. (2019). Visualizing Twitter interactions with NetworkX. 2019.
  10. Firmansyah, A. F. B., & Pramana, S. (2018). Ensemble based gustafson kessel fuzzy clustering. Journal of Data Science and Its Applications, 1(1), 1-9. https://doi.org/10.21108/jdsa.2018.1.6
  11. Rachman, F. F., & Pramana, S. (2021). Analysis of Indonesian People's Sentiments About the Side Effects of the COVID-19 Vaccine on Twitter. Journal of Data Science and Its Applications, 4(1), 1-10.
  12. Rachman, F. F., & Pramana, S. (2020). Analisis Sentimen Pro dan Kontra Masyarakat Indonesia tentang Vaksin COVID-19 pada Media Sosial Twitter. Indonesian of Health Information Management Journal (INOHIM), 8(2), 100-109.
  13. Hadiana, A. (2018). Designing interface of mobile parental information system based on users' perception using Kansei engingeering. Journal of Data Science and Its Applications, 1(1), 10-19. https://doi.org/10.21108/jdsa.2018.1.7
  14. Manuel, B., & Tricahyono, D. (2018). Classifying electronic word of mouth and competitive position in online game industry. Journal of Data Science and Its Applications, 1(1), 20-27. https://doi.org/10.21108/jdsa.2018.1.9
  15. Iskandar, A. R., & Purno, A. (2018). Transition Strategies of Change Management For the Succesful Implementation of Data Warehouse of Higher Education in Indonesia. Journal of Data Science and Its Applications, 1(1), 28-38. https://doi.org/10.21108/jdsa.2018.1.2
  16. Kemenko. (2021). Pertumbuhan Ekonomi Triwulan II 2021 Menembus Zona Ekspansif. Kementrian Koordinator Bidang Ekonomi. 2021.
  17. Kemenkeu. (2022). Stimulus Covid-19 Diskon Listrik Diperpanjang. Kementian Keuangan. 2022.
  18. Pramana, S., Paramartha, D. Y., Ermawan, G. Y., Deli, N. F., & Srimulyani, W. (2021). Impact of COVID-19 pandemic on tourism in Indonesia. Current Issues in Tourism, 1-21.
  19. Simanjuntak, T. N., & Pramana, S. (2021). Sentiment Analysis on Overseas Tweets on the Impact of COVID-19 in Indonesia. Indonesian Journal of Statistics and Its Applications, 5(2), 304-313. https://doi.org/10.29244/ijsa.v5i2p304-313
  20. Wahid, D. H., & Azhari, S. N. (2016). Peringkasan sentimen esktraktif di twitter menggunakan hybrid TF-IDF dan cosine similarity. IJCCS (Indonesian Journal of Computing and Cybernetics Systems), 10(2), 207-218. https://doi.org/10.22146/ijccs.16625
  21. Mariel, W. C. F., Mariyah, S., & Pramana, S. (2018, March). Sentiment analysis: a comparison of deep learning neural network algorithm with SVM and naive Bayes for Indonesian text. In Journal of Physics: Conference Series (Vol. 971, No. 1, p. 012049). IOP Publishing. https://doi.org/10.1088/1742-6596/971/1/012049