Right, it will harass you a bit to review the driver, but you can bypass it.
Evidently in Uber’s early days, a driver could immediately see a passenger’s rating for them, but not any more. According to the Quora answers here, all the drivers can see are the average ratings over various time ranges. Drivers can kind of game it by looking after the first ride of the day or calculating the rise/drop in the average after a ride. But otherwise, they can’t link a specific rating to a rider. Just wait a few hours after the ride to rate them, and you’re fine.