Evaluation of a Solar Flare Forecast Model with Value Score

There are probabilistic forecast models for solar flare occurrence, which can be evaluated by various skill scores (e.g. accuracy, critical success index, heidek skill score, and true skill score). Since these skill scores assume that two types of forecast errors (i.e. false alarm and miss) are equal or constant, which does not take into account different situations of users, they may be unrealistic. In this study, we make an evaluation of a probabilistic flare forecast model [Lee et al., 2012] which use sunspot groups and its area changes as a proxy of flux emergence. We calculate daily solar flare probabilities from 2011 to 2014 using this model. The skill scores are computed through contingency tables as a function of forecast probability, which corresponds to the maximum skill score depending on flare class and type of a skill score. We use a value score with cost/loss ratio, relative importance between the two types of forecast errors. The forecast probability (y) is linearly changed with the cost/loss ratio (x) in the form of y=ax+b: a=0.88; b=0 (C), a=1.2; b=-0.05(M), a=1.29; b=-0.02(X). We find that the forecast model has an effective range of cost/loss ratio for each class flare: 0.536-0.853(C), 0.147-0.334(M), and 0.023-0.072(X). We expect that this study would provide a guideline to determine the probability threshold and the cost/loss ratio for space weather forecast.