Why did they cancel the Dreamcast?

Because the Sega Saturn was defeated by PS1 and N64, and once PS2 came out it defeated the Dreamcast. Sega realized it was time to abandon the console market, so they cancelled the Dreamcast and started making games for other systems.