The first pennies were English coins. An exact date for their first use isn't known but some numismatic historians trace the denomination back to Anglo-Saxon coins called pæningas, issued during the 7th century CE.
The denomination continued in various forms until it was standardized in the 18th century. It was used in most British colonies as well, and variations of it existed in other countries such as Germany where they were known as pfennige.
After independence the US experimented with a quasi-decimal coinage system. The equivalent of the penny was the 1-cent denomination, worth 1/100 of a dollar. Among the first coins was the so-called
"Fugio" cent, issued in 1787 with a design suggested by Benjamin Franklin. When formal coinage began in 1793 the 1-cent coin carried a picture of Miss Liberty. It was similar in size and composition to its British ancestor, so the colloquial term "penny" continued to be used and persists to today.
Canada used coinage based on the British system until 1858 when it too adopted a quasi-decimal coinage system. Canadian 1-cent coins were also similar in size and composition to British pennies so the slang term persists among Anglophone Canadians as well.