It's called bump steer or roll steer.
You might need to have your alignment checked, or perhaps there are worn rear trailing arm bushings, or some other suspension anomaly. If you have 80k miles or more, your shocks and struts may be in need of replacement. Your front control arms and/or bushings may be worn too.
How's your tire pressure? Too high and the tire may be jumping over bumps rather than gliding over them.