Stamp Collecting is a fantastic hobby, and is very educational as well. I feel it makes a fantastic hobby for the homeschooling family. It is fairly inexpensive, can be worked on individually or as a family, and can be done by the very young right up to senior citizenship age.
I have sat around the table with Delta and Echo when they were 4 and 2, with them carefully sorting through stamps. They love looking for ones they don't yet have. At this point, they just store them in a stock-book - but at some point they will start making stamp pages, and probably even decide on specialties.
But do you want to know the great thing? The resource I found gives away free starter kits, including free stamps! I know I found a couple of other great resources too - but I think they are all available on the blog that I am going to give you. The blog also has information on how to actually do the stamp collecting. (I think they also have free starter kits for stamp clubs too, or for groups like scouting or guiding to work on collectors badges.)
Another great source of stamps can be families with missionaries servingin in other countries, or people with family in other countries that still mail letters.