I've found that using Fels Naptha works best. It can be purchased at your local Walmart next to the laundry detergent, or on Amazon.
In my laundry sink, I get the pants wet and then rinse the loose dirt off. Then you can rub the Fels Naptha bar on the pants (like you would a bar of soap and a wash cloth) until the pants are full of soap foam.
Then rub the pants together and work the stains/dirt out. This can be used on your top jersey as well.
Throw them in the washer on heavy wash and voila, back to looking brand new - takes about 10 minutes.
For Hats, I use the Ball Cap Buddy
We use Fels Naptha but work the soap in with my hand. Lock the hat in the Ball Cap Buddy, using a clip to ensure the form stays. Depending on how dirty it is, I'll use the permanent press/delicate, or normal (if really dirty) in the washing machine.
I wash it with other clothes, and then hang dry, as soon as the cycle is completed. I don't use the dishwasher.