My philosophy on dirt: If its dirty even slighty(not "dusty" but "dirty") I wash it first. Why risk imbedding swirl marks with any product and cloth? The product itself may not cause swirls, but the application and removal certainly will if you have a dirty car.
Now, I *have* used a detail spray after about 3-4 days of driving in good sunny weather, after using a California Duster on the finish first...but there was no "dirt" visible, and I was extremely cautious not to apply too much pressure with my cotton towelettes....
Washing takes so little time, better safe than sorry!