When letting off the brake pedal why would there be a knock or clunking noise?

Front brake shoes are sometimes attached to the calipers by "spring clips" and these can rust and break causing your pads to "float". Sometimes calipers get rusty or pitted on the hydraulic shaft and don't retrack smoothly when you let go of the brakes. This is more noticeable if the pads are well worn. The same happens with the rear shoes. If they are well worn, there is a lot of dust in the vicinity and it can foul your automatic adjusters so the shoes aren't close to the drums and you hear noises. If the noise can be localized to one wheel and the pads are OK, then you might need a new caliper.