Sliding ferroelectric materials are a type of recently disclosed two-dimensional van der Waals layers, where each layer is not electrically polarized and the interfacial interaction gives observable out-of-plane ferroelectricity. Since quite a few theoretical predictions (courtesy of Prof. M Wu at HUST), recent experiments have firmly demonstrated the existance of this unusual ferroelectric systems. In the current work, we use theoretical analysis to show photomagnetization can be flipped under the interfacial ferroelectricity switches. This technique provides a potential all-optical method to detect and manipulate the sliding ferroelectricity. See https://www.nature.com/articles/s41699-022-00297-6.