A controlled low strength material (CLSM) is a highly flowable cementitious material used for trench backfilling. However, when applying vertical loads to backfilled trenches, shear failure or differential settlement may occur at the interface between the CLSM and natural soil. Hence, this study aims to evaluate the characteristics of the interface friction between the CLSM and soils based on curing time, gradation, and normal stress. The CLSM is composed of fly ash, calcium sulfoaluminate cement, sand, silt, water, and an accelerator. To investigate the engineering properties of the CLSM, flow and unconfined compressive strength tests are carried out. Poorly graded and well-graded sands are selected as the in-situ soil adjacent to the CLSM. The direct shear tests of the CLSM and soils are carried out under three normal stresses for four different curing times. The test results show that the shear strengths obtained within 1 day are higher than those obtained after 1 day. As the curing time increases, the maximum dilation of the poorly graded sand-CLSM specimens under lower normal stresses also generally increases. The maximum contraction increases with increasing normal stress, but it decreases with increasing curing time. The shear strengths of the well-graded sand-CLSM interface are greater than those of the poorly graded sand-CLSM interface. Moreover, the friction angle for the CLSM-soil interface decreases with increasing curing time, and the friction angles of the well-graded sand-CLSM interface are greater than those of the poorly graded sand-CLSM interface. The results suggest that the CLSM may be effectively used for trench backfilling owing to a better understanding of the interface shear strength and behavior between the CLSM and soils.