Influenced by various mining activities, fractures in rock masses have different densities, set numbers and lengths, which induce different mechanical properties and failure modes of rock masses. Therefore, precisely expressing the failure criterion of the fractured rock influenced by coal mining is significant for the support design, safety assessment and disaster prevention of underground mining engineering subjected to multiple mining activities. By adopting PFC2D particle flow simulation software, this study investigated the propagation and fractal evolution laws of the micro cracks occurring in two typical kinds of rocks under uniaxial compressive condition. Furthermore, it calculated compressive strengths of the rocks with different confining pressures and box-counting dimensions. Moreover, the quantitative relation between the box-counting dimension of the rocks and the empirical parameters m and s in Hoek-Brown strength criterion was established. Results showed that with the increase of the strain, the box-counting dimension of the rocks first increased slowly at the beginning and then exhibited an exponential increase approximately. In the case of small strains of same value, the box-counting dimensions of hard rocks were smaller than those of weak rocks, while the former increased rapidly and were larger than the latter under large strain. The results also presented that there was a negative correlation between the parameters m and s in Hoek-Brown strength criterion and the box-counting dimension of the rocks suffering from variable mining activities. In other words, as the box-counting dimensions increased, the parameters m and s decreased linearly, and their relationship could be described using first order polynomial function.