Well first off welcome. You're not going to get a V8 with a manual transmission. The V8 also didn't come in the X3. You'll also have a better shot finding a manual X3 than you would a manual X5. I'm not sure how well the six's tow, but my X has towed some stuff for me quite well. There's been some tranny issues for me, but after 181,000 miles its doing great right now. You will also not find a manual E70 nor a manual F25 (US). The 540i has the same engine as the E53 V8, you knew that already. The difference is the AWD which is not bad. CV boot changes is an addition to what maintenance is on the 5. 5 touring has the same single axle air suspension as some X's like mine which has been too much trouble. Some X's and all 4.8is have double axle air suspension which is controllable. Not so in the single setup. 4.6is only up to 2003 has the same engine as the 00-03 4.4 with different intake and cylinders bored. 2004 was the facelift (LCI) which sent the 4.4i away for a new N62. The 4.8is from 2004-06 has the same engine except tuned for 360HP.
They are great vehicles if kept maintained. A nice ride height above the traffic, but not too high and great handling makes it great for daily driving.
If you know your way around a box of tools the X shouldn't be an issue. Don't know too much about X3's, but one thing I do know is that I walk all over those from a dig. (The 2 that tried anyway)