# RECENT ADVANCES IN DOMAIN DECOMPOSITION METHODS FOR TOTAL VARIATION MINIMIZATION

• Accepted : 2020.06.01
• Published : 2020.06.25

#### Abstract

Total variation minimization is standard in mathematical imaging and there have been numerous researches over the last decades. In order to process large-scale images in real-time, it is essential to design parallel algorithms that utilize distributed memory computers efficiently. The aim of this paper is to illustrate recent advances of domain decomposition methods for total variation minimization as parallel algorithms. Domain decomposition methods are suitable for parallel computation since they solve a large-scale problem by dividing it into smaller problems and treating them in parallel, and they already have been widely used in structural mechanics. Differently from problems arising in structural mechanics, energy functionals of total variation minimization problems are in general nonlinear, nonsmooth, and nonseparable. Hence, designing efficient domain decomposition methods for total variation minimization is a quite challenging issue. We describe various existing approaches on domain decomposition methods for total variation minimization in a unified view. We address how the direction of research on the subject has changed over the past few years, and suggest several interesting topics for further research.

#### References

1. A. TOSELLI AND O. WIDLUND, Domain Decomposition Methods-Algorithms and Theory, vol. 34, Springer, Berlin, 2005.
2. A. QUARTERONI AND A. VALLI, Domain Decomposition Methods for Partial Differential Equations, Oxford University Press, New York, 1999.
3. J. XU, Iterative methods by space decomposition and subspace correction, SIAM Rev., 34 (1992), pp. 581-613.
4. C. R. DOHRMANN, A preconditioner for substructuring based on constrained energy minimization, SIAM J. Sci. Comput., 25 (2003), pp. 246-258.
5. J. MANDEL, Balancing domain decomposition, Commun. Numer. Methods Engrg., 9 (1993), pp. 233-241.
6. C. FARHAT, M. LESOINNE, AND K. PIERSON, A scalable dual-primal domain decomposition method, Numer. Linear Algebra Appl., 7 (2000), pp. 687-714.
7. C. FARHAT AND F.-X. ROUX, A method of finite element tearing and interconnecting and its parallel solution algorithm, Internat. J. Numer. Methods Engrg., 32 (1991), pp. 1205-1227.
8. C.-O. LEE AND E.-H. PARK, A dual iterative substructuring method with a penalty term, Numer. Math., 112 (2009), pp. 89-113.
9. L. I. RUDIN, S. OSHER, AND E. FATEMI, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259-268.
10. D. STRONG AND T. CHAN, Edge-preserving and scale-dependent properties of total variation regularization, Inverse Problems, 19 (2003), pp. S165-S187.
11. A. CHAMBOLLE, An algorithm for total variation minimization and applications, J. Math. Imaging Vision, 20 (2004), pp. 89-97.
12. Y. WANG, J. YANG, W. YIN, AND Y. ZHANG, A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sci., 1 (2008), pp. 248-272.
13. A. BECK AND M. TEBOULLE, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202.
14. T. GOLDSTEIN AND S. OSHER, The split Bregman method for L1-regularized problems, SIAM J. Imaging Sci., 2 (2009), pp. 323-343.
15. A. CHAMBOLLE AND T. POCK, A first-order primal-dual algorithm for convex problems with applications to imaging, J. Math. Imaging Vision, 40 (2011), pp. 120-145.
16. E. ESSER, X. ZHANG, AND T. F. CHAN, A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science, SIAM J. Imaging Sci., 3 (2010), pp. 1015-1046.
17. L. BADEA, Convergence rate of a Schwarz multilevel method for the constrained minimization of nonquadratic functionals, SIAM J. Numer. Anal., 44 (2006), pp. 449-477.
18. L. BADEA AND R. KRAUSE, One-and two-level Schwarz methods for variational inequalities of the second kind and their application to frictional contact, Numer. Math., 120 (2012), pp. 573-599.
19. X.-C. TAI AND J. XU, Global and uniform convergence of subspace correction methods for some convex optimization problems, Math. Comp., 71 (2001), pp. 105-124.
20. M. FORNASIER, A. LANGER, AND C.-B. SCHONLIEB, A convergent overlapping domain decomposition method for total variation minimization, Numer. Math., 116 (2010), pp. 645-685.
21. M. FORNASIER AND C.-B. SCHONLIEB, Subspace correction methods for total variation and l1-minimization, SIAM J. Numer. Anal., 47 (2009), pp. 3397-3428.
22. M. HINTERMULLER AND A. LANGER, Subspace correction methods for a class of nonsmooth and nonadditive convex variational problems with mixed $L^1$/$L^2$ data-fidelity in image processing, SIAM J. Imaging Sci., 6 (2013), pp. 2134-2173.
23. Y. DUAN AND X.-C. TAI, Domain decomposition methods with graph cuts algorithms for total variation minimization, Adv. Comput. Math., 36 (2012), pp. 175-199.
24. A. LANGER, S. OSHER, AND C.-B. SCHONLIEB, Bregmanized domain decomposition for image restoration, J. Sci. Comput., 54 (2013), pp. 549-576.
25. C.-O. LEE, J. H. LEE, H. WOO, AND S. YUN, Block decomposition methods for total variation by primal-dual stitching, J. Sci. Comput., 68 (2016), pp. 273-302.
26. C.-O. LEE AND C. NAM, Primal domain decomposition methods for the total variation minimization, based on dual decomposition, SIAM J. Sci. Comput., 39 (2017), pp. B403-B423.
27. H. CHANG, X.-C. TAI, L.-L. WANG, AND D. YANG, Convergence rate of overlapping domain decomposition methods for the Rudin-Osher-Fatemi model based on a dual formulation, SIAM J. Imaging Sci., 8 (2015), pp. 564-591.
28. M. HINTERMULLER AND A. LANGER, Non-overlapping domain decomposition methods for dual total variation based image denoising, J. Sci. Comput., 62 (2015), pp. 456-481.
29. C.-O. LEE AND J. PARK, Fast nonoverlapping block Jacobi method for the dual Rudin-Osher-Fatemi model, SIAM J. Imaging Sci., 12 (2019), pp. 2009-2034.
30. C.-O. LEE, E.-H. PARK, AND J. PARK, A finite element approach for the dual Rudin-Osher-Fatemi model and its nonoverlapping domain decomposition methods, SIAM J. Sci. Comput., 41 (2019), pp. B205-B228.
31. C.-O. LEE AND J. PARK A finite element nonoverlapping domain decomposition method with Lagrange multipliers for the dual total variation minimizations, J. Sci. Comput., 81 (2019), pp. 2331-2355.
32. Y. DUAN, H. CHANG, AND X.-C. TAI, Convergent non-overlapping domain decomposition methods for variational image segmentation, J. Sci. Comput., 69 (2016), pp. 532-555.
33. T. F. CHAN, S. ESEDOGLU, AND M. NIKOLOVA, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM J. Appl. Math., 66 (2006), pp. 1632-1648.
34. T. F. CHAN AND L. A. VESE, Active contours without edges, IEEE Trans. Image Process., 10 (2001), pp. 266- 277.
35. J. PARK, An overlapping domain decomposition framework without dual formulation for variational imaging problems. arXiv:2002.10070 [math.NA], 2019. To appear in Adv. Comput. Math.
36. C.-O. LEE, C. NAM, AND J. PARK, Domain decomposition methods using dual conversion for the total variation minimization with $L^1$ fidelity term, J. Sci. Comput., 78 (2019), pp. 951-970.
37. T. F. CHAN AND S. ESEDOGLU, Aspects of total variation regularized $L^1$ function approximation, SIAM J. Appl. Math., 65 (2005), pp. 1817-1837.
38. A. TIKHONOV, Solution of incorrectly formulated problems and the regularization method, Soviet Math. Dokl., 4 (1963), pp. 1035-1038.
39. G. AUBERT AND P. KORNPROBST, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, Springer, New York, 2006.
40. E. GIUSTI, Minimal Surfaces and Functions of Bounded Variation, Birkhauser, Boston, 1984.
41. X. FENG AND A. PROHL, Analysis of total variation flow and its finite element approximations, ESAIM Math. Model. Numer. Anal., 37 (2003), pp. 533-556.
42. A. CHAMBOLLE AND T. POCK, An introduction to continuous optimization for imaging, Acta Numer., 25 (2016), pp. 161-319.
43. Y. MEYER, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: the Fifteenth Dean Jacqueline B. Lewis Memorial Lectures, vol. 22, American Mathematical Society, Providence, 2001.
44. M. NIKOLOVA, A variational approach to remove outliers and impulse noise, J. Math. Imaging Vision, 20 (2004), pp. 99-120.
45. R. T. ROCKAFELLAR, Convex Analysis, Princeton University Press, New Jersey, 2015.
46. I. EKELAND AND R. TEMAM, Convex Analysis and Variational Problems, vol. 28, SIAM, Philadelphia, 1999.
47. Y. DONG, M. HINTERMULLER, AND M. NERI, An efficient primal-dual method for $L^1$TV image restoration, SIAM J. Imaging Sci., 2 (2009), pp. 1168-1189.
48. K. KUNISCH AND M. HINTERMULLER, Total bounded variation regularization as a bilaterally constrained optimization problem, SIAM J. Appl. Math., 64 (2004), pp. 1311-1333.
49. L. BADEA, X.-C. TAI, AND J. WANG, Convergence rate analysis of a multiplicative Schwarz method for variational inequalities, SIAM J. Numer. Anal., 41 (2003), pp. 1052-1073.
50. X.-C. TAI, Rate of convergence for some constraint decomposition methods for nonlinear variational inequalities, Numer. Math., 93 (2003), pp. 755-786.
51. J. BOLTE, S. SABACH, AND M. TEBOULLE, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., Ser. A, 146 (2014), pp. 459-494.
52. A. CHAMBOLLE, AND T POCK, A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions, SMAI J. Comp. Math., 1 (2015), pp. 29-54.
53. R. SHEFI AND M. TEBOULLE, On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems, EURO J. Comput. Optim., 4 (2016), pp. 27-46.
54. A. LANGER AND F. GASPOZ, Overlapping domain decomposition methods for total variation denoising, SIAM J. Numer. Anal., 57 (2019), pp. 1411-1444.
55. S. BARTELS, Total variation minimization with finite elements: convergence and iterative solution, SIAM J. Numer. Anal., 50 (2012), pp. 1162-1180.
56. M. HERRMANN, R. HERZOG, S. SCHMIDT, J. VIDAL-NUNEZ, AND G. WACHSMUTH, Discrete total variation with finite elements and applications to imaging, J. Math. Imaging Vision, 61 (2019), pp. 411-431.
57. P.-A. RAVIART AND J.-M. THOMAS, A mixed finite element method for 2-nd order elliptic problems, in Mathematical Aspects of Finite Element Methods, Springer, 1977, pp. 292-315.
58. I. GOODFELLOW, Y. BENGIO, AND A. COURVILLE, Deep Learning, MIT Press, Cambridge, 2016.
59. S. J. WRIGHT, Coordinate descent algorithms, Math. Program., 151 (2015), pp. 3-34.
60. P. L. COMBETTES AND V. R. WAJS, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
61. Y. E. NESTEROV, A method for solving the convex programming problem with convergence rate O(1=$k^2$), Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547.
62. B. HE, Y. YOU, AND X. YUAN, On the convergence of primal-dual hybrid gradient algorithm, SIAM J. Imaging Sci., 7 (2014), pp. 2526-2537.
63. B. HE AND X. YUAN, Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective, SIAM J. Imaging Sci., 5 (2012), pp. 119-149.
64. R. T. ROCKAFELLAR, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), pp. 877-898.
65. A. CHAMBOLLE, AND T POCK, On the ergodic convergence rates of a first-order primal-dual algorithm, Math. Program., Ser. A, 159 (2016), pp. 253-287.