Thanks for the info guys.
I think I may have figured at least one issue out. I was fiddling with the car, check the TPS (PITA, couldn't really tell if it was ok or not) and checking for vac leaks. I hear a slight hissing noise near the firewall all the time but I can't seem to pin point it. Anyway I was moving wires around, and the fuel return line when the idle started changing. So I moved the return line and sure enough, changed the idle, if I move it a certain way, it will die. If I move it another way, it will idle right at 700 rpm. Part of the line is almost rock solid.
So tomorrow I'll replace that whole line and see what happens.