Phone <=> Car Bluetooth connection

My 2005 Toyota Prius has Bluetooth for connecting my phone to the audio system. When I receive a call I press a button on the steering wheel and it cuts out the radio and I can have a conversation. At least, that’s how it’s supposed to work.

The MDF has a linear scale when a call comes in. The bars to the left and right of the middle marker appear to be signal strength for audio input and output. Sometimes on my previous phone, and it seems every time on my current phone, there are only bars to the left of the middle mark. If I talk, the caller can hear me, but I cannot hear them over the car’s speakers.

I’m guessing that since this problem had happened with my old phone, that the problem lies within the car. Your best guesses: Is this a software problem in the car, a hardware problem in the car; or does the problem lie within the phone? How can the connection be fixed?