To get the handling, you will end up stiffer. That's life.
To get the speed, you end up with a Frankencar that will always be worth LESS than you put into it, and in many cases, less than the base car. Just because you love it doesn't mean you find ANYONE who will.
And no, you will not end up with an M3, as you can't easily swap out the rear fenders. The fonders, hood, side skirts, and diffuser you can do. But again, a lot of money for a Frankcar.
Oh, and let's not even consider warranty on ANY of it, or the fact that a lot of shops won't work on it, or if they do, it will be more costly since they are dealing with hacked together electronics.
Basically, if you want an M3, get an M3.
'95 Alpine M3 LTW
'00 Dakar M Roadster
'02 Topaz M3
Red/White SRF #4 (Chassis 561)