Bridge hangers, such as those in suspension and cable-stayed bridges, suffer from cumulative fatigue damage caused by dynamic loads (e.g., cyclic traffic and wind loads) in their service condition. Thus, the identification of damage to hangers is important in preserving the service life of the bridge structure. This study develops a new method for condition assessment of bridge hangers. The tension force of the bridge and the damages in the element level can be identified using the Bayesian optimization method. To improve the number of observed data, the additional mass method is combined the Bayesian optimization method. Numerical studies are presented to verify the accuracy and efficiency of the proposed method. The influence of different acquisition functions, which include expected improvement (EI), probability-of-improvement (PI), lower confidence bound (LCB), and expected improvement per second (EIPC), on the identification of damage to the bridge hanger is studied. Results show that the errors identified by the EI acquisition function are smaller than those identified by the other acquisition functions. The identification of the damage to the bridge hanger with various types of boundary conditions and different levels of measurement noise are also studied. Results show that both the severity of the damage and the tension force can be identified via the proposed method, thereby verifying the robustness of the proposed method. Compared to the genetic algorithm (GA), particle swarm optimization (PSO), and nonlinear least-square method (NLS), the Bayesian optimization (BO) performs best in identifying the structural damage and tension force.