All that being understood, saying "rest in peace" isn't necessarily incompatible with Judaism. As the Jewish Theological Seminary explains, there are plenty of precedents in Jewish texts that illustrate that it's totally fine to talk about peace — shalom — in a variety of life contexts. Genesis 37:4, for instance, describes a family conflict centered around Joseph that made saying "l'shalom" (said when saying goodbye) in the house difficult.

Rather than saying "rest in peace," though, Rabbi Julie Zupan on Reform Judaism describes a variety of more traditionally Jewish phrases to say when speaking of the dead. "Zichrona livracha" (feminine) and "zichrono livracha" (masculine) mean, "May their memory be for a blessing." "Aleha hashalom" (feminine) and "hashalom alav" (masculine) mean, "May peace be upon them." "Baruch dayan ha'emet," commonly said shortly after someone dies, means "Blessed is the Judge of Truth." And if you're visiting a family that is sitting shiva, you can say (it's a long one), "Hamakom yenakhem etekhem betokh shaar avelay tziyon viyrushalayim," which means, "May God comfort you among the other mourners of Zion and Jerusalem" (via Chabad). 

The closest to "rest in peace" we have in Judaism comes at the end of a couple of different prayers, the El Maleh Rachamim and Yizkor: "vatanuach b'shalom" (feminine) or "vayanuach b'shalom" (masculine). The "b'shalom" is important here, rather than "l'shalom," because as the North Western Reform Synagogue says, it implies finality. It should be used if you never expect to see someone again.

ncG1vNJzZmivp6x7qL7Up56eZpOkunB9j2loa3FkZMSpxYyypq6knGK%2For7EpbBmoJWWv27Ax55kqaCilsCmedGeqq1lmaN6sbHAnJxmoZ5it7awwKKqpmc%3D