One question:
When you guys let off of the throttle, do you have some form of
Cruise control or throttle/friction lock to keep the rev's stable?
If not then, imo, the issue is less likely the wheel offset and more the rotational force of the engine.
I say this because as long as my RPM's are stable she runs pretty straight. If the throttle is accelerating she goes one way and goes the other on deceleration.
Later,
Freelancer