What is the statute of limitations on music piracy if the music collection is kept?

There are actually two statutes of limitations for music piracy. One is the civil statute of limitations, which is how long you can be sued by the music company, which is 3 years. The other is the criminal statute of limitations, which is how long you can face criminal prosecution for the crime of music piracy (copyright infringement), which is 5 years. These two statutes of limitation are found in 17 USC ยง507.

Note that the fact that you kept the music collection is irrelevant. Courts have ruled that piracy is not a "continuing tort;" what this means is that the statute of limitations starts at the time you commit the theft. Therefore, you can be sued for 3 years or prosecuted for 5 years after you have stolen music, even if you HAD deleted the music.