Did the US Postal Service ever deliver mail on Sunday?

In Jon Meacham's book on Andrew Jackson, American Lion Chapter 6 describes how Ezra Stiles Ely, a Presbyterian Pastor, petitioned Jackson to stop the federal delivery of mail on Sundays. "We have always viewed it as a national evil of great magnitude, and one which calls for national repentance oand reformation , that the mails are carried and post-offices kept open, on that holy day in every part of the countyry." This would be in the 1830s and clearly priore to that time the mails were delivered on Sunday. In 1827 Ely's sermon on a "Christian Party in Politics" which should join forces to keep "Pagans" and Mohammedans" from office as well as deists like George Washingon or Thomas Jefferson or Unitarians Like John Quincy Adams.