Is Maine the only state that borders Canada and the Atlantic ocean?

No, New Hampshire and New York also border both Canada and the Atlantic Ocean.

However, only New York actually touches the Atlantic Ocean: New Hampshire and Maine border the Gulf of Maine.