A primary index is based on the primary key of a table and determines the physical order of the table data, allowing for faster retrieval of rows. A secondary index is created on a non-primary key column to provide an alternative access path to the data, enabling faster retrieval based on that specific column.