Walmart does tend to have better prices, especially since that will match prices, and take coupons. So technically, yeah they could sell cheap rechargeable batteries, but to make it even cheaper, look for coupons and add-matches.
Your Playstation Wallet is rechargeable at the Playstation Network. The card does not have to be an actual card and can just be a number to recharge your wallet. Gamestop will sell a $20 Digital card for $19.99
A rechargeable mouse can be found practically at any store. The best place to find one will be at an electronics store like Best Buy or Radio Shack. Walmart, Target, Kmart also sell them. They can also be found online at sites such as Amazon or eBay.