Thanks, that is some good info. But after sleeping on it, I think I know what I did wrong. I grounded it to the "ground" side of the starter relay. I don't think that is a true ground, but only acts like one until the starter it activated. It then becomes the "hot" wire that powers the starter motor. Basically, I've got it grounded through the starter motor, until the starter relay switches on, then its "grounded" to a hot wire and the fuse blows. Probably lucky it didn't fry my GPS with it wired this way.
Over the weekend I'll ground it straight to the frame like I should have before, and see if that solves the problem.