Can you connect a receiver speaker jack using banana connectors to wireless headphones using audio cable with rca jacks?

No. The wireless transmitter is expecting a line level input. The banana plugs are power amplifier level, and you will damage your transmitter. If you add the model of your receiver/amplifier into the discussion tab above, we can tell you where to connect it.