The ride quality is NOT that bad, while you cannot expect smooth ride like Lexus RX330. The problem with X3 I think it 1) Interior, 2) price.
I test drove Mysti/gray combo. The interior feels very small (compare to X5) and low quality (even compare to Toyota). Altought for the low quality part, people says it is because of the gray interior color, I cannot comment on this because I have not seen other color interior. But I would guess with black interior, X3 will feel even smaller.
Price, X3 3.0 base MSRP is $36,300 while X5 3.0 base MSRP is $40,800. I don't see a reason choosing X3 over X5