iDrive and bluetooth pairing preferences

This has been one core frustration for me with our 2013 X3 (ok, so the nav is even more annoying, but that's for another post).

My partner and I both have iPhone's that are paired with iDrive. It seems to pick at random between our phones when we both get in the car and drive off. Most of the time this results in the car pairing with his phone, but I'm the one driving. So, the phone rings and I can't answer the damn thing until I get to a stopping point. I know I can use voice and get to BT options to make the switch, but on the next drive it always seems to pair with his phone first.

I was hoping the car would associate my phone with my key fob which is unlocking and starting the car. I assume that is not the case? Are pairing preferences completely at random, or is there a way to give preference to the person who is actually driving?