A Computational Model to Detect Affective Response Based on Narrative Agent's Knowledge

Abstract

Narratives arouse diverse and rich affective responses to recipients, and this is one of the reasons why narratives are universal and popular. Computational studies on narratives have established a formal model or system of the affective response based on the theory in psychology or media research, and have analyzed or generated a narrative that can evoke a specific affective response. In this paper, we propose a new computational model that can detect the affective response expected to appear in the narrative based on the narrative agent's knowledge. First, we designed a narrative representation model that can elaborately express the event structure and the agent's knowledge as well. Additionally, an analysis method was proposed to detect the three affective responses and the related situational information. Then, we validated the model through a case study about an actual movie narrative. Through the case study, we confirmed that the model captures the affective responses of the audience. The proposed model can be effectively used for the narrative analysis and the creation that must consider the affective responses of the recipient.

1. Introduction

People respond to diverse and rich affects while experiencing narrative-based contents such as novels, movies, and dramas. It is one of the characteristics of the narrative as a genre of discourse that evokes the diverse and rich affective responses, and this is why the narrative is universal and popular [1].

There have been many studies in various fields of affective response. In studies from a computational perspective on narratives, the affective response has been also one of the key themes. These studies have established a formal model or system of the affective response based on the theory in psychology or media research, and analyzed or generated a narrative that can evoke a specific affective response. These studies have used different approaches depending on the definition of the affective response and the explanation of the mechanism to evoke it [2].

In this paper, we propose a new computational model that can detect the affective response expected to appear in the narrative based on the narrative agent’s knowledge. The narrative agent is a subject responsible for making the meaning of the narrative, and is divided into the character agent living in the story-world and the pragmatic agent participating in storytelling. The interaction between the character and recipient, one of the pragmatic agents, makes the narrative richer and more complex [3]. Branigan argued that the disparity of knowledge between the character and the recipient had a decisive effect on the affective responses, which are divided into three categories; suspense, mystery (or curiosity), and surprise [4]. Our model is based on Branigan’s theory, and has the advantage in that it has an expressive representation model and can detect the affective responses with situational information.

The proposed model is composed of three parts. The first part is a narrative representation model that can express the event structure and agent’s knowledge together. Second part is a method to calculate the knowledge state of each narrative agent using a probabilistic reasoning model, and to visualize the change of the knowledge state according to the event development. And third part is a method to automatically detect the affective responses based on fuzzy rules derived from the patterns of change in the knowledge state.

This paper consists of the following sections. In section 2, we examine several studies on computational models or systems of the affective response, especially suspense. In section 3, we explain our model by dividing into three parts. In section 4, we validate the model through a case study about an actual movie narrative. Lastly, in section 6, we draw a conclusion.

2. Related Works

Many computational studies of the affective response have focused on suspense. The reason may be that suspense directly related to interest, immersion, and enjoyment, therefore it has been actively studied in psychology or aesthetics [5].

Cheong et al. [6] proposed Suspenser, a story generation system for suspense and O'Neill et al. [7] proposed Dramatis, a computational model of suspense. Both of these studies were theoretically based on the concept of suspense by Gerrig and Bernardo [8]. They argued that readers' reports on suspense were moderated by their perceptions of the range of solutions available to a textual dilemma. In this study, the readers identified themselves with the problem-solving protagonist, and the fewer the number of paths to a solution the more increased the suspense level of the readers. The Suspenser automatically selected and rearranged a set of given events to construct a desirable discourse structure that maximized the suspense rating, which was the ratio of failed plans to successful plans out of all possible plans that a protagonist might have. In the Dramatis model, the reader traversed a search space on behalf of the protagonist, searching for an escape from some impending negative outcome. The escape plan was evaluated by the likelihood of success perceived by the reader in a representation model that reflected the narrative understanding based on situation model [9]. These studies had the advantage to represent a narrative rigorously and to evaluate the suspense level quantitatively. They also captured the suspense that the readers feel when they identify with the main character at risk. However, there is a limitation in that they cannot capture other affective responses related to suspense. And it is needed to supplement the explanation of the relationship between the planning-related suspense and the knowledge-related suspense.

There are also computational studies of affective response based on the narrative agent’s knowledge. Bae et al. [10,11] proposed a surprise-evoking narrative generation system called Prevoyant based on structural affect theory (SAT). The SAT proposed by Brewer et al. [12,13] is a representative study on suspense in terms of the discourse structure and the agent’s knowledge. The SAT argued that the order of events made a disparity of knowledge between the agents and this could evoke different affective responses. The Prevoyant took a series of chronologically ordered events as input and reordered the events by using flashbacks and foreshadowing to make the disparity of knowledge. The amount of surprise was evaluated by checking the ‘unexpectedness’ and ‘postdictability’ of events. The Prevoyant reconstructed a narrative with surprise quite effectively, but had some limitations. First, the discourse structure focusing on the order of events was too simple to represent a complex narrative. In addition, there was a lack of situational information about who the character the readers are focusing on, and what knowledge they have in mind. We made up for the weak points by designing the representation model to express the narrative agent’s knowledge and the analysis method to detect three affective responses with situational information.

3. Proposed Model

The proposed model is composed of three parts, which are functionally divided into a narrative representation model and an analysis method, as shown in Figure 1. In the following, we explain the model in detail for each part. by dividing in three parts.

Figure 1. Overview of the proposed model

3.1 Knowledge Structure

3.1.1 Components and process

The Knowledge Structure (KS) as a narrative representation model is designed on the basis of the previous works [14,15]. The KS expands the discourse structure focused on the event to the level of information and knowledge. Figure 2 shows the components of the KS and the underlying cognitive processes of the agent.

Figure 2. The components and process of the Knowledge Structure

The KS models the semantic structure with information and knowledge components orthogonal to the discourse structure with events, while the functional relations between components are established as follows. Each agent experiences the events and acquires the pieces of information included in the events. The level of acquisition of information varies among the agents depending on the perception link, which indicates how much an agent who experiences the event notices the information. Each agent reasons the set of knowledge in the narrative based on the accumulated information so far. The result of reasoning varies depending on the impact link, which indicated how much each piece of information supports or opposes the linked knowledge. In summary, each agent obtains information by experiencing the event and updates the state of a specific knowledge (i.e. knowledge state) through reasoning based on the accumulated information.

3.1.2 Configuration pattern using information link

We devise an additional component, ‘information link’ that indicates the relations between information to deal with more complex narrative. In many situations of narrative, the exact meaning of a piece of information becomes clear when several related pieces of information are combined. We propose four configuration patterns in the KS using the information link as shown in Figure 3. Each of the patterns forms a kind of if-then-else rules that changes the interpretation of a piece of information by other information given as the condition.

Figure 3. Four configuration patterns in the Knowledge Structure

(a) Combination pattern: In Figure 3a, the impact link 1 is not activated when an agent acquires I1 by experiencing E1. It is activated after I2 is acquired from E2, and enables updating of the knowledge state about K1. This is similar to ‘foreshadowing’, which is the plot device the author provides hints about what happens by placing signs of upcoming events [16]. It is used to raise questions, to predict future events, or to consistently explain unexpected events that will occur later.

(b) Disregard pattern: In Figure 3b, the impact link 1 is not activated when an agent acquires both I1 and I2 from experiencing E1 and E2 in order. However, if an agent acquires only I1 without I2 by not experiencing E1, the agent updates the knowledge state about K1 by activating 1 . This pattern typically occurs when there is a deceptive character and K1 is the false knowledge. This is similar to ‘red herring’ in plot device that leads recipients toward a false condition [16]. The deceptive character who experiences E1 can keep the knowledge state by disregarding information as a red herring, however, the other characters who have not acquired the condition information update their knowledge states toward an incorrect direction.

(c) Recurrence pattern: In Figure 3c, the impact link 1 is activated after an agent acquires I1 by experiencing E1 while the link 2 is activated when the agent reacquires I1 from E3 after acquiring I2 from E2. This pattern is similar to ‘frequency’ or ‘repetition’ in narration, which Genette identified as one of the main aspects of narrative temporality [17]. It is the concept of how many times an event that occurred once in story time is presented in discourse time.

(d) Reversal pattern: In Figure 3d, the impact link 1 is activated after an agent acquires I1 by experiencing E1. The supporting direction of 1 is revered when the agent acquires I2 from E2, so the knowledge state about K1 is drastically changed. In the reversal pattern, the agent first misunderstands the meaning of the information that has been acquired. After going through another event, the agent adjusts their previous belief.

3.1.3 Formal notation

The KS serves as a representation framework modeled with several sets of narrative components and relations among the sets. Extending the formalism introduced in [15], the following definition describes a structural representation of narrative captured by the KS.

Definition 1. A narrative Σ is a tuple 〈C, E, p, I, J, K, π, ι〉, where C is a set of characters and the recipient R, E = (e₁,⋯,e_n) is an ordered set of events sorted by the story time, p : {1, ⋯,m} ↦ {1, ⋯,n} is a plot function of the discourse composed of m events, by which the i-th event in the discourse is identified as e_p(i) ∈E, I is a set of information, J = {(τ,I_j,i_j)} is a set of composition pattern among information where τ ∈ {com, dis, rec, rev} is a composition type indicating one of the four configuration patterns, respectively, I_j ⊂ I is a subset of information appeared in the pattern’s condition part, and i_j ∈ I is the information being conditioned in the pattern, K is a set of knowledge, π : E ⨉C ⨉ I ↦ [0,1] is a function that represents the perception value of the information perceived in an event by an agent, and ι : 2^I ⨉ I ⨉ K ↦ [-1,1] is a function of the impact value of a set of information with respect to the knowledge. Without loss of generality, the values of π and ι are both defined as zero if the corresponding links are not defined in the KS.

Extending the vocabulary with composition patterns J, the domain of ι has been changed from (I ⨉ K) to (2^I ⨉ I ⨉ K). The first part, a set of information, can be interpreted as the condition of the second information towards the knowledge. Let 𝑖 # 𝑘 denotes the impact link from information or combination pattern i to knowledge k with the impact value of v, which is defined in the KS. The function ι can be defined over the whole domain in the constructive manner as follows.

\(\iota\left(I_{c}, i, k\right)=\left\{ \begin{aligned} v_{1} &\text { if } \begin{array}{l} \left(\forall\left(\tau, I_{c}^{\prime}, i^{\prime}\right) \in J: \quad i^{\prime} \neq i\right) \wedge \\ \left(\forall\left(\operatorname{com}, I_{c}^{\prime}, i^{\prime}\right) \in J: i \notin I_{c}^{\prime}\right) \wedge i \stackrel{v_{1}}{\rightarrow} k \end{array} \\ v_{2} &\text { if } \begin{array}{l} \exists j=\left(\operatorname{com}, I_{c}^{\prime}, i^{\prime}\right) \in J \wedge i \in I_{c}^{\prime} \cup\left\{i^{\prime}\right\} \wedge \\ I_{c}^{\prime} \cup\left\{i^{\prime}\right\} \subseteq I_{c} \cup\{i\} \wedge j \stackrel{v_{2}}{\rightarrow} k \end{array} \\ v_{3} & \text { if } \exists\left(\neg c o m, I_{c}^{\prime}, i\right) \in J: I_{c}^{\prime} \nsubseteq I_{c} \wedge i \stackrel{(1) v_{3}}{\longrightarrow} k \\ v_{4} & \text { if } \exists\left(r e c, I_{c}^{\prime}, i\right) \in J: I_{c}^{\prime} \subseteq I_{c} \wedge i \stackrel{\text { (2) } v_{4}}{\longrightarrow} k \\ -v_{5} & \text { if } \exists\left(r e v, I_{c}^{\prime}, i\right) \in J: I_{c}^{\prime} \subseteq I_{c} \wedge i \stackrel{\text { (1) } v_{5}}{\longrightarrow} k \\ 0 & \quad otherwise. \end{aligned} \right.\)       (1)

The first case describes that the information i is not part of any configuration pattern. The second case explains a combination pattern j is activated by a subset IC and the information i. The third case describes the presumable link is activated due to the lack of information while the secondary interpretation is activated with the enough information in the fourth and fifth cases. If the condition of a disregard patters is satisfied, it falls the last case and has no effect as in unspecified cases.

Extending the notion of agent, which includes both the characters and the recipient, we introduce the character judged by the recipient, c|R (c∈C∖{R}), observed and reconstructed by the recipient through the narrative in discourse order.

Given an agent A and the discourse time point t, the event sequence, ℰ(A,t), that the agent has been engaged and the set of information, ℐ(A,t), that have been gathered are defined as follows.

\(\begin{array}{c} \mathcal{E}(A, t)=\left\{\begin{array}{ll} \left(\left(\left(e_{i} \mid A \text { is in } e_{i}, 1 \leq i \leq p(t)\right)\right.\right. & \text { if } A \in C \backslash\{R\} \\ \left(\left(e_{p(i)} \mid A \text { is in } e_{p(i)}, 1 \leq i \leq t\right)\right. & \text { if } A=R \text { or } c \mid R(c \in C \backslash\{R\}) \\ \mathcal{J}(A, t)= \bigcup_{e \in \mathcal{E}(A, t)}\{i \mid \pi(e, A, i) \neq 0\} \end{array}\right. \end{array}\)       (2)

Note that characters have experienced the events in the story time while the recipient and the characters judged by the recipient has experienced in the discourse time.

To interpret a piece of information, we need all the information gathered so far to examine the condition of the composition patterns. Given the set of information, ℐ(A,t), gathered so far, the effective impact of information i towards knowledge k that perceived and acquired in event e_p(t), is given as the following function:

ϕ(A, e_p(t), ℐ(A,t), i, k) = π (A, e_p(t), I ) ⨉ ι (ℐ(A,t), i, k)       (3)

3.2 Knowledge Flow

3.2.1 Analysis and representation of the knowledge state

The knowledge state of an agent about a piece of knowledge indicates how confident the agent is about the knowledge. Because the knowledge is a form of proposition in the model, the knowledge state is expressed as degree of belief (DoB) in the truth of the corresponding knowledge. It can be estimated through the probabilistic reasoning on the basis of the accumulated information that has been acquired by experiencing events until a given time point. Table 1 shows the detailed interpretation of the knowledge state.

Table 1. Interpretation of knowledge state

※ DoB (Degree of Belief) value = 0 can be interpreted in two ways:

1) Ignorant: No information is linked to the knowledge.

2) Indecisive: Information linked to the knowledge contradicts, thus cannot be decided.

The Knowledge Flow (KF) is a representation of the trajectory of the knowledge state that changes as events develop. It is expressed in the form of a line graph in which the x-axis represents the order of events and the y-axis represents the knowledge state after the agent has experienced events so far. Each KF is defined with respect to an agent and a piece of knowledge. There are three types of KF: KF_C, KF_R and KF_C|R. KF_C and KF_R, respectively, denote the KF of the character C and the KF of the recipient. Characters experience events in the story-world in chronological order while the recipient only experiences the presented events in the discourse along the plotted order. Therefore, the KF_C is evaluated based on the story time, whereas KF_R is evaluated based on the discourse time. Unlike KF_C or KF_R, which represents the actual KF of the agent, KF_C|R reflects the knowledge flow of the character judged by the recipient. Because the recipient estimates the knowledge state of other character based on their own experience [18], KF_C|R is evaluated from the previous events in discourse time in which the character has been engaged.

3.2.2 Probabilistic reasoning about the knowledge state

Given a narrative Σ, the knowledge state at discourse time t is evaluated from the agent’s experience by the reasoning model M, as outlined by the following formula in [15].

\(K S_{\Sigma}(A, t)-M_{\Sigma}\left(\delta_{M}^{0}[A], \mathcal{E}(A, t)\right)\)       (4)

where KS_Σ(A,t) is a vector whose elements correspond to the knowledge state about each piece of knowledge, δ_M^t is a model parameter for internal use in the model M indexed by agents and pieces of knowledge at time t, and 𝛿_M⁰ [A] is an initial value of the parameter for the initial knowledge states. Note that M takes Σ so that M internally evaluates the KS via the functions in Σ. For the clarity, we will omit the subscript Σ in the obvious context.

The KF defined by the series of the knowledge states is conveniently denoted by the syntactic construct:

∀𝑘 ∈ 𝐾, 𝐾𝐹_R(𝑘) = (𝐾𝑆(𝑅, 𝑡)[𝑘]|𝑡 = 1, ⋯ , 𝑚)       (5)

where the subscript denotes the agent and parameter enclosed by parenthesis denotes the specific piece of knowledge. The KF is presented as a line graph in which the above series is plotted to provide the visual cues of trends and changes in the knowledge state.

The knowledge state as a DoB can be defined in terms of probability: a subjective probability of the truth of a piece of knowledge given the evidences throughout the experience. In this sense, the effective impact of the information towards the knowledge is interpreted as the proving power of evidence given a set of information acquired, which is independent to the prior knowledge state of the agent. A reasoning model M maintains the knowledge states of an agent in terms of posterior probability updated from the prior state using new evidences. We have proposed a formal probabilistic reasoning method based on the Dempster-Shafer theory of evidence (DST) within the proposed modeling framework [15]. It is valid to adopt the former previously proposed reasoning method in this paper because the reasoning model only takes the effective impact value function out of the framework elements and the function is suitably extended by Equation 1.

The following is a brief summary of the reasoning model. In DST, four kinds of hypotheses for each piece of knowledge are used: {H_∅, H_T, H_F, H_U}. H_T or H_F means that the proposition of the knowledge is true or false, respectively. The null hypothesis H_∅ means that it is neither true nor false while H_U means that it is either true of false but cannot be decided by anyone. The effective impact of the information is used as the mass for either H_T or H_F according to its type – positive or negative. Because the masses sum up to one, the remaining mass is assigned to H_U, rather than the opposite. This allows DST to treat the either role of evidence, support or denial, independently, in a sense that a piece of information supporting any knowledge with probability p may not mean that the information denies the knowledge with probability 1-p. For the effective impact v = ϕ(A, e_p(t), ℐ(A,t), i, k), the mass function is defined as follows.

\(\begin{array}{c} m_{i \rightarrow k}^{t}\left(H_{\emptyset}\right)=0 \\ m_{i \rightarrow k}^{t}\left(H_{U}\right)=1-|v| \\ \text { if } \quad v \geq 0\left\{\begin{array}{l} m_{i \rightarrow k}^{t}\left(H_{T}\right)=v, \text { if } \quad v<0\left\{\begin{array}{c} m_{i \rightarrow k}^{t}\left(H_{T}\right)=0 \\ m_{i \rightarrow k}^{t}\left(H_{F}\right)=|v| \end{array}\right. \end{array}\right. \end{array}\)       (6)

For the new piece of information i to be considered at time t, the mass function \(m_{i \rightarrow k}^{t}\) towards each related knowledge k is set by Equation 6 and is subject to combine the previous mass function towards the knowledge k, \(m_{k}^{t-1}\), that the agent have kept and updated to \(m_{k}^{t}=m_{k}^{t-1}\) \(\oplus m_{i \rightarrow k}^{t}\) by the traditional Dempster’s rule of combination.

The agent’s belief about the proposition (i.e. the subjective probability) is bounded by the two values: belief (bel) and plausibility (pl). These are computed from the mass function as follows.

\(\text { bel }_{k}^{t}\left(H_{i}\right)=\sum_{\left\{H_{j} \mid H_{j} \subseteq H_{i}\right\}} m_{k}^{t}\left(H_{j}\right), \quad p l_{k}^{t}\left(H_{i}\right)=\sum_{\left\{H_{j} \mid H_{j} \cap H_{i} \neq \emptyset\right\}} m_{k}^{t}\left(H_{j}\right)\)       (7)

Applying the DST method to the proposed modeling framework, the knowledge state about knowledge k of an agent at time t is defined by the difference between \(\text { bel }_{k}^{t}\left(H_{T}\right)\) and \(\operatorname{bel}_{k}^{t}\left(H_{F}\right)\) evaluated from the mass function of the agent \(m_{k}^{t}\), and the value varies in [-1, 1]. The proposed DST-based model M^DST is defined in the form of Equation 4 as follows.

\(K S_{\Sigma}(A, t)[k]=M_{\Sigma}^{D S T}\left(\delta_{M^{D S T}}^{0}, \mathcal{E}(A, t)\right)[k]=b e l_{k}^{t}\left(H_{T}\right)-b e l_{k}^{t}\left(H_{F}\right)=m_{k}^{t}\left(H_{T}\right)-m_{k}^{t}\left(H_{F}\right)\)       (8)

The model parameter, \(\delta_{M^{D S T}}^{t}[A][k]=m_{k}^{t}\), is the mass function of knowledge k combined so far with respect to agent A.

In the computational aspect, it is notable that if the mass of either H_T or H_F becomes one or zero, it would never be changed through the update process. It means that once an agent reaches the definite belief about either truth of a proposition, they never change their mind regardless of the future crucial evidences. Authors may intentionally mislead their recipients throughout the narrative and give an unexpected consequence. To address this problem, the definite belief value, either 0 or 1, is adjusted by a small value ε > 0 to allow the marginal uncertainty.

3.3 Affective Response

3.3.1 Pattern of Knowledge Flow

Since the agent's knowledge is elaborately expressed in the KF, it is possible to identify the affective responses suggested by Branigan through the pattern of KF. Table 2 shows the pattern of KF for each affective response.

Table 2. Affective response and pattern in the Knowledge Flow

※ (when R realizes) KF_R and KF_C|R drastically shift towards KF_C and become identical.

3.3.2 Rule-based detection

Comparing the KFs of two agents is to compare the knowledge states of the respect agents about the certain piece of knowledge at every moment in the development of narrative. Focusing on the affective response in Table 2, this comparison can be simplified by interpreting the quantitative knowledge state as the qualitative categories: negative (F), neutral (N) or positive (T). However, the criteria for classifying affective responses likely to be associated with emotions is somewhat ambiguous. As an example of this interpretation, we propose the simple fuzzy sets to represent each of these categories with membership functions, μ_F, μ_N and μ_T, as defined in Figure 4.

Figure 4. Fuzzy sets of qualitative categories in knowledge state

A pairwise comparison of two knowledge states is a binary relation among the set Q = {F, N, T}, which is extended to the fuzzy relation as follows.

ℛ(𝑥₁, 𝑥₂) = ∑_{(𝑞1, 𝑞2)∈Q×Q}(q₁, q₂)/𝜇_q1×q2(𝑥₁, 𝑥₂)       (9)

where 𝜇_q1×q2(𝑥₁, 𝑥₂) = min{𝜇_q1(𝑥₁), 𝜇_q2(𝑥₂)}.

Let a time interval is the type about knowledge k and character c. Certainly, the recipient should take the knowledge into account before the beginning of the interval. If not, it must be the case that the knowledge k is ignorant due to the lack of relative information yet. Thus, it should be satisfied the following basic filtering rule for the interval [t₁, t₂] to be of affective response.

Rule I: ∃𝑡´ ∈ [1, t₁]∧ 𝐾𝑆(𝑅, 𝑡´)[𝑘] ≠ 0       (10)

If it is in type A at any time point, the relation between KF_R and KF_C|R is among a set SA = {(F,N), (F,T), (T,F), (T,N)} while the relation is among a set SB = {(F,F), (N,N), (T,T)} in type B. Both the affective responses last or alter along the line of the discourse. For a time t ∈ [t₁, t₂] satisfying rule I and a certain α> 0, type A is detected by the following rule.

Rule A: 𝑆_A ∩ ℛ(𝐾𝑆(𝑅, 𝑡)[𝑘], 𝐾𝑆(𝑐|𝑅, 𝑡)[𝑘])_αA ≠ ∅       (11)

where {∙}_α is a crisp set of the elements whose member functions are not less than α, called α-level set.

Since the recipient and characters share the uncertainty in type B, the interval of type B should be before a time point t_d at which the recipient acquires the true knowledge. Because the author actually knows the time point, they can set and pass t_d to the rule-based analyzer to eliminate late intervals within which the significant knowledge has been so uncovered that KF_R coincides in KF_C|R. Given t_d and the interval [t₁, t₂] (t₂d), type B is detected during the interval by the following rule.

Rule B: 𝑆_B ∩ ℛ(𝐾𝑆(𝑅, 𝑡)[𝑘], 𝐾𝑆(𝑐|𝑅, 𝑡)[𝑘])_αB ≠ ∅       (12)

On the contrary, type C is different from the above two situations; it requires a kind of retrospective analysis. The situation of type C is recognized only when KF_R that has been apart from KF_C drastically changes and approaches to KF_C, and then, the recipient realizes that they have misunderstood by reflecting the past events. Detecting this type can be done by the following rule. For SC = {(F,F), (T,T)} and any t ∈ [t₁, t₂),

Rule C: 𝑆_A ∩ ℛ(𝐾𝑆(𝑐, 𝑡)[𝑘], 𝐾𝑆(𝑅, 𝑡)[𝑘])_αC ≠∅∧       (13)

𝑆_A ∩ ℛ(𝐾𝑆(𝑅, t₂)[𝑘], 𝐾𝑆(𝑅, t₂ −1)[𝑘])_αC ≠∅∧

𝑆_C ∩ ℛ(𝐾𝑆(𝑐, t₂)[𝑘], 𝐾𝑆(𝑅, t₂)[𝑘])_αC ≠ ∅

It is remarkable that the recipients have no idea of the actual KFs of characters at all. Type C is recognized by the author with a global perspective, and the author utilizes these situations to maintain the narrative tension and interest via narrative techniques such as foreshadowing and bias. On the recipient's side, they regard type C as type A or B, or even insignificant situation according to the respect KF_C|R unless they experience a sudden change in their own KF detected by the second line of rule C. What interesting is how the recipient realizes this surprise situation without reference to KF_C. One possible and plausible way is for them to realize by the similar sudden change of KF_C|R, by which they hypothesize about their misunderstanding of the character; therefore, the following supplementary rule can be added.

Rule C′: 𝑆_A ∩ ℛ(𝐾𝑆(𝑐|𝑅, t₂)[𝑘], 𝐾𝑆(𝑐|𝑅, t₂ − 1)[𝑘])_αC ≠∅∧       (14)

𝑆_C ∩ ℛ(𝐾𝑆(𝑐|𝑅, t₂)[𝑘] 𝐾𝑆(𝑅, t₂)[𝑘])_αC ≠ ∅

The analysis process of affective response consists of the exhaustive combinatorial pairwise comparison between KF_R and both KF_C and KF_C|R for each agent and each piece of knowledge at every time point during the discourse. The α-level set helps the subtle control of detecting and interpreting the situations. We empirically suggest a relatively high 𝛼C to capture the dramatic situations only. It is also noted that, at a certain time point in the discourse, the type of situation may not be identified uniquely even though one single piece of knowledge and a single character are in account due to the fuzziness.

The affective response detected by the rules can be presented to the author as a list of the result record: {[type, relevant knowledge, relevant character, event interval]}, which are four factors of the affective response. The author can select each item in the list and visually inspect the corresponding affective response with KFs. The author can also modify the KS and analyze the KF iteratively.

4. Case Study

In this section, we validate the model through a case study about an actual movie narrative. The target work is Vertigo (Alfred Hitchcock, 1958), which is selected for the following reasons. a) This work has a twistplot which arouses surprise with an unexpected consequence in the narrative. The main feature of the twist-plot is that the recipient’s knowledge state suddenly changes, and the contents and timing of the change are clearly presented. b) Various focalization techniques [17] which are to change the viewpoint of narration are used. Because of this, the disparity of knowledge between the agents appears in abundance. c) The discordance between the story time and the discourse time plays a critical role in the configuration of events. Therefore, various narratives that have different features and effects can be created by changing the order of events.

After representing and analyzing the movie narrative according to the model, we compared the affective responses detected by the model with the affective responses reported by the audience.

4.1 Representation and Analysis

4.1.1 Knowledge Structure

We divided the narrative into the events and identified the main characters and actions in each event. The narrative was simplified focused on the agent’s knowledge. Table 3 shows the order and description of each event, and Figure 5 depicts the KS constructed based on these listed events.

Table 3. Events of Vertigo

Figure 5. Knowledge Structure of Vertigo

4.1.2 Knowledge Flow

We analyzed the KF based on the KS as shown in Figure 6.

Figure 6. Knowledge Flows of Vertigo

The main feature of the KF is the rapid change of KF_R and KF_C|R at E14 and E15 in discourse time. That means the recipient experiences the key events at that time and knows the secret of the narrative. These events presented as flashback reveal the secret of the narrative as the conspiracy by Judy and Gavin. As shown in (c), Judy's KF remains unchanged in the starting setting, which means that Judy knew all the secrets from the beginning. As the audience experience E14 and E15, they become aware of Judy's identity, and this change of knowledge state appears in (f). As shown in (b), the KF of Scottie remains unchanged at E14 and E15, because only the audience experience these events. Scottie knows the secret only at the end of the narrative as shown in (b) and (e).

4.1.3 Affective response

We detected the affective responses based on the fuzzy-rules with α_A = 0.3, α_B = 0.2 and α_C = 0.7, as shown in Table 4.

Table 4. Affective responses of Vertigo

4.2 Experiment

4.2.1 Experimental design

We recruited 15 subjects who are college students in their 20s, majoring in engineering, 8 males and 7 females, and have not seen Vertigo. After watching the movies together, each of them completed a questionnaire as shown in Table 5, and there was no time limit for completing the questionnaire, and it took 25 minutes on average.

Table 5. Questionnaire about each affective response

※ The subjects have to fill in all questions on each affect.

Among the affective responses, only those that were reported by more than five subjects were selected. Then, we compared them with the affective responses presented in Table 4. The hypothesis set in this experiment is that the affective responses detected by the model will be consistent with the affective responses reported by the audience.

4.2.2 Result and discussion

The result of the experiment is shown in Table 6.

Table 6. Result of experiment

The result shows that the recall is significant but the precision is quite low. However, it is needed to examine each result in detail to grasp the characteristics, limitations, and supplementary points of the model.

In the aspect of recall, the model did not detect three affective responses reported by the audience. Two were about surprise or horror for the sudden fall of Judy and the appearance of sister like a ghost in the last scene. These could not be detected because they are related to the action, not the agent’s knowledge. The other was about tension or suspense when Scottie noticed the secret and took Judy to the monastery. Scottie had begun to suspect Judy as the criminal, but Judy was still unaware of the change of Scottie’s knowledge. The audience identified with Judy and felt tension. This affective response is related to the agent’s knowledge about other agent’s knowledge, which is a kind of meta-cognition [19]. The knowledge state in this situation can be formally expressed as KF_C|C’ or KF_(C|C’)|R which can be calculated by the probabilistic reasoning method proposed in this paper. We did not use these nested KFs because we were worried that the analysis result would be overly complicated. However, these KFs used in actual narratives need to be included in the model.

The model detected much more affective responses that were not in the audience’s reports. Most of them (about 68%) were the type B affective responses, which are less salient compared to type A or C. The reason why these were not reported by the audience is presumably due to the nature of the remembrance process. When the recipients remember a narrative, they reconstruct the situation model of the narrative focusing on outstanding event, affect and motive, so details are often ignored or omitted [20]. Another reason is that the audience tends to report only one affect in one event interval, while the model detects more detailed affective responses subdivided by the knowledge and character in the same event interval. The detailed analysis results unreported by the audience, however, can be helpful for analysis and creation of narratives.

5. Conclusions

We proposed a computational model that can detect the affective response expected to appear in the narrative based on the narrative agent’s knowledge. The model consisted of the narrative representation model and the analysis method; the KS and KF to represent the event structure and agent’s knowledge, and the method to detect the affective response based on fuzzy-rules derived from the pattern of KF. Through the case study, we confirmed that the model captures the affective responses of the recipient.

The propose model has three advantages over other computational models based on the SAT. First, by expanding the narrative representation model from discourse structure to the KS, more complex narratives can be analyzed. Second, it is possible to elaborately express the change of agent’s knowledge through four configuration patterns that imitate the plot devices used by narrative authors. Third, it is possible to detect not only the affective response of the recipient, but also situational information such as who the character the recipient is focusing on, and what knowledge she has in mind.

There are several limitations with this study, which will be complemented by future works. First, the limitations presented in the case study is that one does not detect the affective response which are not related to the agent’s knowledge, and the other does not cover the agent’s knowledge about another agent’s knowledge. The former is an intrinsic limitation of this study, and the latter can be solved by making nested types of KF (i.e. KF_C|C’ or KF_(C|C’)R) and diversifying the comparison between KFs. Second, the proposed model cannot analyze the strength or level of the affective response. We plan to calculate the level of affective response using the degree of change in the knowledge state. Many case studies about actual narratives are needed to examine the relationship between these two factors. Third, it is needed to pattern the relationship between the three components of the model; KS, KF, and affective response. The pattern should include how to manipulate the KS to evoke a specific affective response, which is related to a reverse operation that if the intended affective response is preferentially set, the KS is changed accordingly. If the model is complemented through these follow-up studies, it can be practically used for analysis and creation of narratives.