Delete completely gets rid of a column. Columns to the right of it all move over. So if you deleted column D, column E would become column D.
Hiding a column makes it disappear from view, but it is still part of the spreadsheet and can be hidden. If you hid column D, you would still see column C and to the right of it would be column E. You can still use cells in columns that are hidden in formulas.