When comparing you have to factor in maintenance cost. I just got out of an e550 for my '13 550. Over that 3 years/50k miles i probably spent on the order of 2500 in maintenance. I know it doesnt close the gap completely (ok or even half in your case) but something not to ignore, unless mb is throwing in their maintenance package too ( which you might also get them to do)
Which car do you really want? Assuming you can just as easily afford both which does not sound like an issue- think about whether getting into the e350 every morning is going to feel like a compromise?
The only other thing i would say, just coming out of an E, is that the transmission shifts are very, very slow compared to the 5 series (at least my '10 e550 vs '13 550xi sport trans). If you test drive again, go simulate a three point turn somewhere that there is no risk of traffic, and you will see what i mean. Other negative is the comand software (even the updated one) is on a small screen and not as intuitive as bmw in my opinion.
Otherwise the benz was a very solid, very reliable and overall great car to drive. I also think it is discounted so much because it is due for a significant refresh next year (i think c&d has an article).
Current: '15 650i xdrive Gran Coupe, '15 X5 xdrive 50i, '16 M4 convertible