According to polling, the most common way to listen to podcasts is using a smart phone, tablet or similar. As such, when I'm editing podcasts I check all of my mixes on my laptop speakers as a proxy for the poor audio quality of a smart phone speaker. This helps, but my laptop speakers are not quite as bad as smart phone speakers. The other day I was struggling to get a mix right for the smart phone. Rather than repeatedly bouncing the file and listening back on my phone, I had the idea of routing all of the audio from my computer straight to my smart phone using Airfoil. This meant I could mix for a smart phone in real time. I also set my phone to play mono audio, because many smart phones only have one speaker. There was some lag, but it was much more streamlined. I was pretty pleased with the resulting audio quality.
Streamlining the process
You can't control the quality of the equipment that your listeners are using, but you can control the recording and editing process.