Our main resource for physical education, (other than just running around playing) is the YMCA. It is a very economical choice - a membership for the whole family is a small amount per month compared to what a gym membership costs for a single person at a lot of gyms. Our Y has exercise equipment for adults and teens, various classes, a swimming pool, babysitting (for when I want to work out or swim), and a lot of drop-in classes for the kids. For people whose funds are really tight, they can reduce your fees to what is possible for your family.
Our children have gone to drop-in gymnastics for pre-schoolers (not on a regular basis), and have taken swim lessons since they could walk. My oldest son may start drop-in tae kwan doe lessons this fall, if our schedule allows it.
Check out the Y in your neighbourhood if you don't already belong!