How were the scientists able to study the earth's interior?

They study the layers by seeing how seismic waves travel through the earth. By the time of the arrival, and how the frequencies of the waves are arranged within each set they can tell things about the density and other things about the layers.