Here's my take: some people love the diesel, some people don't.
Lot's of folks will highlight the diesel's strengths (and there are quite a few). I averaged 20+mpg with mostly city and some long highway trips factored in. This was awesome for such a heavy car. The car has a ton of torque, which gives the impression (and smile on your face feeling) of speed, especially in DS, even if the actual numbers are not any better than the gas. It's fun to push the pedal and go. On the highway it is mostly effortless to pass. The gas is no slouch here either but the diesel just feels solid in this respect.
It's a solid engine that has fewer moving parts so should theoretically require fewer service visits. Then again, many people have complained about having to have the DEF filled early (I did too), and that's a service visit that is unexpected. They are now covering the cost of the filling (they apparently did not for a while) the DEF under the service plan but it's still time and energy. Because of this, I had my diesel in more than my X3 during the same amount of time.
It may hold it's value better, but I'm pretty sure this will depend almost entirely on the cost of diesel when you sell, which you can't predict. I am leasing, so this wasn't an issue for me.
And as others have said it's not difficult to find diesel, usually. But you should drive around your neighborhood and plot out which stations you usually go to and which you'll have to go to with the diesel. It wasn't a huge issue for me, but it definitely took me out of my way a mile (each way) or so each time I wanted to fill up. Diesel prices seem to vary much more at stations as well, so plan to get Fuel finder. Use it before you buy the car to see what's available in your area. You may also want to pack some nitrile gloves in the car. When you get diesel on your hands, it does not go away easily. It dissipates much more slowly than gas, so the smell stays longer.
The noise is a factor some and not others. People will tell you you can't hear the engine in the cabin at speed. I can. Always. Radio blasting or not. At idle or acceleration, the tone is cringe-worthy...to me. Not to everyone. My wife doesn't notice it at all. It may not bother you at all. It may be music to your ears. Only you can tell. To me it sounds like I'm driving a box truck. That may float your boat. It may make you want to do a Tim Allen grunt. It bothered me constantly. I want my BMW to sound like what I expect in a BMW. That's me. It may be you too. If you're considering the diesel, do yourself a favor and take it on a long test drive. Sit in it while it's idle for a while. Drive it in as many different road conditions as you can. Take it through a drive thru. Drive it in the rain if you can (the engine noise is amplified in the rain). Play with the acceleration in different scenarios. Ply the radio loud and soft. Try talking on the phone. In my short test drives, I didn't think it'd bother me. I does. It may bother you. It may not.
Lots of other folks have mentioned the heavy steering in the X5. Do yourself a favor and try to test drive one with and without active steering. It's not available on the diesel. Starting from a dead stop in 2nd gear and with the heavy steering, the diesel can feel absolutely piggish to me (DS mode solves most of this, and I would switch back and forth constantly). The 8-speed in the 35i is more accurate to my mind.
The DEF warning is really annoying. It starts at 1000 miles from the point where your car won't stop. Unlike the iDrive warning that you get when you start the car, the DEF warning does not go away until you click it away. And a warning symbol remains on the instrument panel at all times. I get the need for the warning, but I like the idea of not having to deal with that again. That to me has to be considered in the service issue.
As I said, it was not difficult for me to find diesel in my local area, even if I had to drive out of my way to do so. But when it is more difficult to find, it's a bit nerve wracking. Traveling out to more rural areas, for example, was a challenge. Finding a station with Fuel finder could put you easily 5-10 miles out of your way. Not a big deal except when you're on your 500th mile, have two cranky kids in the car, and you just want to get to your destination. I was also in plenty of places where the only diesel was a truck stop and it required the pump adapter. This is a pain in the neck, if you ask me. It may not bother you at all but think about having to take the adapter out, fill your tank, and then figure out what to do with this diesel soaked plastic thing when you're on a long trip. Plus truck stop diesel pumps almost inevitably have diesel all over the ground. You'll get it on your shoes. Between your shoes and the adapter, your car will smell like diesel. Again, this may or may not bother you.
Anyway, my point is try the diesel before you buy. Try it as extensively as you can. You may be one of the many who love it. It's still a BMW and it's a beautiful car. But, you may not and when you don't it definitely takes away from the enjoyment of a very, very expensive car.