An Oligopoly Spectrum Pricing with Behavior of Primary Users for Cognitive Radio Networks

Abstract

Dynamic spectrum sharing is a key technology to improve spectrum utilization in wireless networks. The elastic spectrum management provides a new opportunity for licensed primary users and unlicensed secondary users to efficiently utilize the scarce wireless resource. In this paper, we present a game-theoretic framework for dynamic spectrum allocation where the primary users rent the unutilized spectrum to the secondary users for a monetary profit. In reality, due to the ON-OFF behavior of the primary user, the quantity of spectrum that can be opportunistically shared by the secondary users is limited. We model this situation with the renewal theory and formulate the spectrum pricing scheme with the Bertrand game, taking into account the scarcity of the spectrum. By the Nash-equilibrium pricing scheme, each player in the game continually converges to a strategy that maximizes its own profit. We also investigate the impact of several properties, including channel quality and spectrum substitutability. Based on the equilibrium analysis, we finally propose a decentralized algorithm that leads the primary users to the Nash-equilibrium, called DST. The stability of the proposed algorithm in terms of convergence to the Nash equilibrium is also studied.

1. Introduction

The rapid advance of mobile telecommunication has made wireless spectrum the scarcest resource in the last decade. However, the recent reports by the Federal Communications Commission (FCC), ironically, show that traditional fixed spectrum allocation results in very low efficiency in spectrum utilization. The increasing spectrum demand, together with under-utilization in currently allocated spectrum, raises a need for a new technology, called cognitive radio. Cognitive radio networks (CRNs) [1] enable dynamic spectrum access in which licensed (primary) users (PUs) share their unused spectrum with unlicensed (secondary) users (SUs). When the spectrum assigned to the primary user is not fully utilized, the primary user has an opportunity to sell the excessive spectrum to secondary users for monetary payoff [2]. In this model, PUs are opportunistic spectrum providers while SUs are the buyers of this spectrum. Therefore, it is natual to analyze the spectrum allocation in the persepective of economic models and market stratagies such as an auction [3] or game-theory [4].

In such an emerging network scenario, multiple primary users can co-exist in the same geographical site and compete for the purchase of secondary users equipped with cognitive radios. Obviously, a meaningful problem for the spectrum trading is how the primary users set prices of per-unit spectrum in a competitive spectrum market. For example, when a primary user sets a low price, the primary user may lose possible revenue (by increasing price). In contrast, if the price is too expensive, secondary users may be willing to purchase other spectrum from other primary users. Niyato et al [5] firstly presented the spectrum pricing game within the Bertrand model in which prices offered by primary users are competitively determined. On the other hands, our analytical model further takes into account the behavioral patterns of PUs. For example, two 20MHz channels, say CA and CB, can be utilized for twenty minutes. If CA is available for continuous 20 minutes while CB is not (e.g., often interrupted by PU itself), the model needs a mechnism that incorporates this criterion. Indeed, this behavior of PUs is a commonly considered feature [6] in the research field of CRNs.

In this paper, we model dynamic spectrum pricing in CRNs with an oligopoly spectrum trading game [7]. Unlike previous work [5][8], we focus our attention to the PUs’ behavioral patterns and incorporate them in our analytical framework. In order to quantify the relative superiority of one channel to another, we adopt a renewal process [9] to model the ON-OFF behavior of PUs. In the oligopoly spectrum market, we employ a commonly used quadratic utility function to quantify the spectrum demand of secondary users, and the strategy of each primary user is analyzed under the utility function. To incorporate the behavioral model, a common principle borrowed from the law of supply and demand is adopted. As the price of a good goes up, consumers demand less of it. If the demand from the secondary users exceeds the contiguous available spectrum, the primary user can increase the price so that the demand falls off. We extend the demand function of the secondary users by dividing the case into two counterparts; (i) when the PU can afford the spectrum and (ii) when the PU cannot afford the spectrum. We analyze the strategies under the two cases and find a static solution that makes the spectrum trading reach Nash-equilibrium. Assuming “Bounded Rationality”, we finally propose a decentralized algorithm that leads the primary users to the Nash-equilibrium, called DST (Decentralized Spectrum Trading). Also, the stability of the proposed algorithm is investigated using the Jacovian matrix method. Our decentralized algorithm, which is proven to be stable by the analysis as well as the simulation, converges to Nash equilibrium quickly.

Our major contributions are summarized as follows:

The rest of this paper is organized as follows. Section 2 reviews the related work. We then give our priliminaries in Section 3. We present the game-theoretic formulation in section 4 and Section 5 examines our analysis via the MATLAB numerical simulations. Finally, Section 6 concludes the paper.

 

2. Related Work

Recently, a lot of work targeted toward solving the scarcity problem of wireless spectrum has been made. [1] introduces cognitive radio (CR) as a next generation technology to address this problem. The authors well-summarized the fundamemtal tasks and agile chracteristics of cognitive radio devices.

Major issues on the CR research are categorized into 4-groups; (i) spectrum sensing, (ii) spectrum sharing, (iii) spectrum management, and (iv) spectrum mobility [10]. The spectrum sensing involves to a subject including how to explore the spectrum opportunities in the licensed primary band. Kim et al found an optimal spectrum sensing schedule, taking into account the behavior of a primary user. An another important issue on the spectrum sensing involves a physical method of sensing [11]. [11] gives a clear technological description on two kinds of spectrum sensing; the feature and the energy detection.

Researchers have started to study how to effectively share the discovered spectrum opportunities. Game theory is one of the resource allocation techniques. It perfectly fits to reflect the selfish behavior of current wireless devices. The current devices compete for their maximal use of wireless spectrum, even they jam each other to hinder other communication on shared frequencies. There are many work on network research, applying the game theoretic approach (e.g., admission control, rate control [12], power control [13][14][15]). Particularly in the area of multiple radio and multiple channel (MRMC) research, adaptive channel allocation schemes [16][17] were proposed.

In game theory, price is the fundamental and important criterion to model the value of wireless spectrum. In [18], a price-based transmission rate control scheme was proposed for wireless ad-hoc networks. An auction model [3] and an oligopoly market [7] are the examples of such pricing scheme, a specialized version of the game-framework. Specifically, The oligopoly market model is one of the popular price and quantity competition model in cognitive radio network [5][8]. The authors of [5][8] modeled the spectrum trading within the oligopoly market framework. The trading price of spectrum is determined based on the secondary user’s preference for spectrum, in which the preference includes the channel quality, substitutability among the primary users, and QoS reguirements of primary users.

Our work lies its basis on [7] and is similar with [5][8], however, the entended framework (with multiple, i.e., more than two users) and the analysis on spectrum scarcity based on the PU’s behavioral pattern ensure to position our work into the state-of-the art research field of cognitive radios.

 

3. Preliminaries

3.1 System Model

We consider a cognitive radio network where N licensed (primary) users or wireless spectrum providers (WSPs) compete for a shared pool of secondary users. The secondary users can be any static/mobile devices equipped with cognitive transceivers. The primary users are the spectrum brokers that rent the unused spectrum frequency to the secondary users for monetary payoff. We depict the spectrum market in Fig. 1. The primary users treat the set of secondary users as a spectrum consumer. Each primary user sells an unused portion of its spectrum (e.g., time slots in TDMA based wireless system) to the market at price pi (i=1,2,…,N). In this market, the demand of secondary users depends on the prices of per-unit spectrum. Each primary user chooses its own strategy pi to induce the subscription of the secondary users.

Fig. 1.Spectrum Market Trading Model

3.2 PU’s Channel Usage Model

Channel usage of a primary user can be modeled as an ON-OFF state semi-Markov chain, alternating between ON (busy) and OFF (idle) periods as depicted in Fig. 2. We assume a periodic sensing mechanism [6]. We also assume that the secondary users co-operate in spectrum sensing. Channel trading is taken place at each sensing time.

Fig. 2.The alternating semi-Markov chain

For channel i (i=1,2,…,N), let denote a random variable for an OFF period of channel i, and its probability density function (p.d.f.) is Similarly, let be a random variable for an ON-period of channel i. The ON-OFF periods are assumed to be independent and identically distributed (i.i.d.). We do not assume that the ON-OFF periods follow any particular distribution. Because the behavior of primary users alternates between the ON-OFF periods, we can analyze it with alternating renewal theory [9].

3.3 Bertrand Game Model

In economics, an oligopoly is a market in which a number of firms compete each other non-cooperatively to maximize their profit. The profit (revenue) is generated by selling a product to the market. The demand (i.e., the trading quantity of the product) from buyers depends on the prices set by the competing firms. Thus we have two options to control the market competition. In the Bertrand game, the sellers handle the market by setting prices. In contrast, in the Cournot game, the market is controlled in the view point of buyers. We apply the Bertrand game model to analyze the behavior of wireless spectrum provider (WSP) and to propose a distributed scheme for a primary user in cognitive radio networks.

 

4. Price Competition and Nash Equilibrium

4.1 A Utility Function of Secondary Users

We characterize the spectrum demands of secondary users in the oligopoly market. Let qi be the quantity of spectrum that secondary users buy from primary user i at price pi. In order to model the utility of an average secondary user, we employ the following commonly used quadratic utility function [7]:

where αi and βi are positive constants for all i ∈N. Here, αi denotes the spectral efficiency of channel i when a secondary user uses it as a channel access medium. For example, channel variations such as path loss, fading, and Doppler effect can be properly taken into account. In wireless communications, the above wireless characteristics may determine the instantaneous signal-to-noise ratio (SNR). In addition, the data transmission rate is basically controlled by the instantaneous SNR. Therefore, we incorporate the instantaneous SNR into our utility function. The spectral efficiency αi of a transmission by a secondary user using channel i can be obtained from [5]:

where σ is the SNR at the receiver and BERtar is the target bit-error-rate (BER).

Similar to previous work [5][8], we also consider the spectrum substitutability via the parameter μ. Suppose γ=0, then a secondary user cannot switch among the primary users. However, 0<γ<βi, a secondary user can switch among the primary users based on the spectrum preferences such as the spectral efficiency, the price of per-unit spectrum. For example, if one primary user increases its price, some of the secondary users may move to other channels instead of using the channel consistently. In particular, when αi= α and βi= γ for all i ∈N, the spectra of primary users are perfectly substitutive for secondary users. The most important property of the utility function is its concavity. Thus, we can incorporate the basic principle of the law of supply and demand with this utility function.

To obtain the purchase price (pi) of secondary users for channel i, we take the first-order derivative of the utility functions with respect to qi and let it be 0:

We now derive the demand of secondary users using eq. (3):

By extending eq. (3)′ for all i ∈N, we obtain:

By multiplying the inverse matrix of the constants, we get:

where p is the price vector and q is the spectrum demand vector. Given the price vector, the demand of secondary users for channel i is rewritten by:

where ai, bi, and cij are positive constants derived from equ. (5). Obviously, they represents the spectral efficiency and the subsitituability.

4.2 Analysis of PU’s Behavior

In this section, we explain how to compute the sojourn-time for secondary users using the alternating renewal theory. We describe an example of the spectrum sensing in Fig. 3. Note that we assume proactive and periodic sensing. Let Tp be the sensing period of the secondary users. Since spectrum sensing is nothing but a sampling process, it is possible to identify each state only (ON or OFF state) at each sensing time (Ts,Ts+Tp,Ts+2Tp,… ). In addition, secondary users cannot detect the ON→OFF state transition. Thus the time between the start time of the OFF period and the discovery time (sensing occurs Ts+Tp in Fig. 3) of the OFF period cannot be utilized (marked as unavailable in Fig. 3). In this way, some OFF periods may remain totally undiscovered at all if sensing is infrequent. On the other hand, secondary users can figure out the return of primary users (the OFF→ON state transition). It can be taken place by the ‘listen-before-talk’ policy of the secondary users. This is particularly important because primary communication must not be interrupted by secondary users. Thus the secondary users can only utilize the time from the sensing time of the OFF period to the OFF→ON transition time or the next sensing time if the primary user does not return (marked as available in Fig. 3).

Fig. 3.The channel usage model and available sojourn-time for secondary users.

In this situation, secondary users may buy the spectrum that has the longest remaining time until the primary user’s return. This is particularly important as secondary user must switch to other channel when the current channel is not available. This may introduce a significant and frequent interruption of a secondary service, and even it imposes large spectrum sensing overhead because it needs frequent spectrum sensing. Therefore, we incorporate this behavioral feature of primary users into the spectrum pricing model.

To analyze PU’s behavior, we use the alternating renewal theory. According to the renewal theory, for an alternating renewal process that has been started a long time ago, the remaining time in the current state (ON or OFF state) from the sensing time Ts has its probability distribution function [9]:

where x denotes the time for OFF state, (or ON state), F(x) is its cumulative distribution function, and E[X] is its expected value. With eq. (7), we rewrite the available opportunity Li between two consecutive spectrum sensing for channel i, i∈N:

We refer to Li as the “sojourn-time” of channel i because secondary users can stay the channel for the duration of Li.

The sojourn-time can be considered as a maximal capacity of channel i. That is, if the demand from secondary users exceeds the sojourn-time, primary user i cannot offer the spectrum to secondary users. In this case, the rational decision for the primary user is to increase the price pi for reducing the demand.

4.3 Static Bertrand Game

With aforementioned components of the game, we can formulate a Bertrand game as follows. The game players in the spectrum trading are primary users. The strategy of the player is to set the price of per-unit spectrum. The payoff is a profit of selling spectrum to secondary users. The spectrum demand of secondary users depends on the spectral efficiency and the channel substitutability, and the prices quoted by primary users. Also, it cannot exceed the sojourn-time. Note, since primary channels are heterogeneous, i.e., have different spectral efficiency, a smaller sojourn-time does not necessarily mean the spectrum limitation compared with a larger one (or unlimited one). The solution of the game is to set a pure strategy to reach Nash equilibrium. The profit of each primary user is given as the product of price pi and trading quantity qi of spectrum.

The Nash equilibrium of the game is defined as a set of all players’ strategies with the property that no player can increase his payoff without changing other players’ strategies [19]. The best response function is defined as the best strategy of one player given others’ strategies. In particular, when other players choose the Nash equilibrium strategies best response against them is the Nash Equilibrium strategy.

In order to mathematically derive Nash equilibrium of the game with the sojourn-time constraint, we have to find all capacity-insufficient primary users. Here, a primary user is capacity-insufficient if the sojourn-time is less than the best demand under no sojourn-time constraint. However, we may not be able to find the capacity-insufficient primary users once for all. The reason is that when capacity-insufficient primary users increase their prices of spectrum, some secondary users may buy the spectrum of other primary users, potentially leading to the lack of capacity in those primary users. Therefore, we need to search several time recursively to find all the capacity-insufficient primary users in the Nash equilibrium [20].

Firstly, we compute the best response strategies of all primary users under no sojourn-time constraint. To compute those strategies, we have to solve the set of marginal profit function. The marginal profit function is the first-order partial derivative of the profit function with respect to pi. Due to its concavity, we find the Nash equilibrium strategy of player i, by letting the derivative be 0.

Note that eq. (10) does not consider the sojourn-time constraint. Denote Mk to be the number of capacity-insufficient primary users in the kth search. Assuming no sojourn-time constraints, we can obtain M1 by solving (10). Next, we take the capacities of M1 primary users into consideration. Because M1 primary users cannot afford the spectrum, they will increase their prices until the demand reaches their affordable spectrum, i.e., equal to the capacities of their spectrum. Thus we obtain the Nash equilibrium under the sojourn-time constraint by letting the demand and the sojourn-time constraint is equal.

The remaining primary users increase the prices correspondingly. The best responses of primary users are obtained through the eq. (10) and eq. (11) for capacity-sufficient primary users and capacity-insufficient ones, respectively. We next define a new matrix with M1 as

and a vector

Before advancing to the next iteration, we have to know whether Q(M1) is invertible or not. According to [8], the matrix Q(Mk) is positive definite if βi>γ>0 for all i∈N in the utility function, which implies that the matrix Q(Mk) is invertible. Hence, we can rewrite each iterative search (i.e., kth iteration) as follows

An important question here is whether the iterative search method eventually ends or not within the finite number of iteration. Y. Xu et al [8] proved that eq. (14) leads the problem to the Nash equilibrium at most N iterative steps.

4.4 Dynamic Bertrand Game

In reality, a primary user may not be able to observe the profit obtained by other primary users. Also, the current strategies adopted by other primary users may be unknown. Therefore, each primary user must learn the policies of other primary users from the history. We derive a dynamic solution to find the Nash equilibrium with the assumption that the past strategies of players are observable each other.

Let be the price offered by primary user i at iteration n. Given the strategies adopted by other players at iteration n, the price offered by primary user i can be obtained iteratively from:

The current strategies used by other primary users are actually unknown. Therefore, each primary user can use historical information and the spectrum demand from secondary users to adjust its strategy. This self-gradual learning under local information is a concept of “Bounded Rationality”. We propose a decentralized algorithm, called DST (Decentralized Spectrum Trading). The algorithm forces each player to reach the Nash equilibrium in a distributed manner. The algorithm incorporates two basic concepts; (i) the gradual-learning of a price increase based on local information and (ii) the stopping rule when the demand of spectrum reaches the maximal capacity (the sojourn-time). Algorithm I shows the pseudo code of our DST algorithm.

In algorithm I, to estimate the partial derivation of the profit function, a primary user can observe the marginal spectrum demand for small variation in price ε as shown in eq. (16).

4.5 Stability Analysis

Stability investigation of the dynamic adjustment is an important task to prove the completeness of the algorithm in terms of convergence to the Nash equilibrium. We analyze the stability of our DST algorithm with the eigenvalue method of the Jacobian matrix consists of the self-mapping functions, i.e., eq. (15).

By definition, self-mapping function is stable if and only if eigenvalues of the Jacobian matrix (denoted by λi) are all inside unit circle in the Euclidian hyperspace (i.e., |λi|<1). From the definition, the Jacobian matrix of this game is given by:

The analysis of the Jacobian matrix eigenvalues shows unconditional convergence to the Nash equilibrium with the assumption (i.e., past strategies of opponents are observable). However the convergence property of DST algorithm is not guaranteed. The property depends on the learning rate δ and the sojourn-time constraint. Proper choice of these variables results in the stability of DST algorithm.

 

5. Performance Evaluation

5.1 Evaluation Setup

We consider a cognitive radio network with several primary users. Each primary user has its own 20MHz channel to lease (i.e., ai=20). We also consider a market consists of many spectrum consumers (secondary users). We assume that the behavior of primary users follows a certain distribution. Note that it could be any distribution which can be expressed with a formal definition. In the numerical simulations, without loss of generality, we used the exponential distribution. The channel quality for the secondary users varies between 5 to 20 dB, which is incorporated by the value αi, i∈N. We assume that the market has diverse freedom for spectrum purchasing. The substitutability values (βi,γ) represent the freedom and we used βi=1,γ=[0.5,1). The positive constants bi are computed from αi, βi, and γ. We choose 0.15 as the constant value for cij. For DST algorithm, the initial prices are set as follows: i∈N. Note that some of these parameters will be varied according to the evaluation scenarios.]undefined

5.2 Numerical Results

Fig. 4 and Fig. 5 show the dynamics of prices and revenues, respectively. In this scenario, we do not incorporate any behavioral constraint. As a result, the prices can only be determined by the spectral efficiency and the substitutability among primary users. Since a primary user can observe only the spectrum demand from the secondary service, and the price is adjusted based on local information. Therefore, the speed of convergence depends largely on the learning rate. We depict one player’s price adjustment under varying learning rate values to show the impact of the learning rate in Fig. 6. As anticipated, if this learning rate is properly set, the algorithm converges to the equilibrium price as fast as that for the case when the strategies of the other players are observable. However, if the learning rate is set to too large values, we see the fluctuations in the price adaptation, and the algorithm may require a larger number of iterations to reach the equilibrium. Though we only used a particular set of the system parameters (i.e., αi, βi, and γ) in this paper, we confirmed that the price and the revenue are converged to Nash equilibrium in every single case. Numerical results show that our DST algorithm makes the game players achieve the Nash equilibrium in the decentralized manner with a proper choice of the learning rate.

Fig. 4.Dynamics of prices under no sojourn-time constraint.

Fig. 5.Dynamics of revenues under no sojourn-time constraint

Fig. 6.Dynamics of one player’s prices under varying learning rate

Next, we intentionally impose the sojourn-time constraint on player 4 to observe the impact of the primary user’s behavior (Other parameters remain unchanged.). For the player 4, the demand from the secondary service (which is determined by set of the offered prices) exceeds the spectrum owner’s capacity; he will increase its price to decrease the demand. This adjustment will last until the spectrum demand reaches the capacity. That is the DST’s feature against in-sufficient capacity condition. Fig. 7 and Fig. 8 show the dynamics of prices and revenues, respectively under the primary users’ behavioral constraint. We observe that the player 4 (having in-sufficient spectrum to sell) increased its price to decrease the spectrum demand from the secondary service. In addition, we do not see any fluctuation. The previous version of our algorithm leads the change of strategy when reaching the capacity limit. In contrast, DST algorithm examines the arrival of its own capacity limit all the time. Hence, we do not see any fluctuation and need any further arbitration. In conclusion, numerical results show that our DST algorithm makes the game players achieve the Nash equilibrium given in the PU’s behavioral constraint as well.

Fig. 7.Dynamics of prices under the primary user’s behavioral constraint

Fig. 8.Dynamics of revenues under the primary user’s behavioral constraint

 

6. Conclusion

We have presented a game-theoretic spectrum pricing scheme to obtain Nash equilibrium. Unlike the state-of-the-art studies, we consider a behavior pattern of primary users and we provide criteria for spectrum sharing in cognitive radio networks.We proposed a decentralized price (revenue) dynamic adaptation algorithm, which forces the price adaption to Nash equilibrium without fluctuations. We also investigated the convergence property of our DST algorithm.

The design of cognitive radio technology requires the attention on the current trend of the selfish networks devices. We leave the impact of the collusion of selfish players to our future work.