The Atlantic had an interesting article about the wealth of those who have served as US Presidents.
They computed the net worth in 2010 terms and determined George Washington to be the richest President ever reaching $525 million. This was from a study from 24/7 Wall Street President Washington had vast amounts of land. He had little prior to marriage but his wife Martha had married a much older and very wealth man who died young and she inherited his fortune which George then controlled and built upon.
James Madison owned 5,000 acres of land.
President Washington received a salary of 2% of the US Budget in 1789.
President John F. Kennedy became the second richest president ever through inheritance from his parents.
Other interesting facts from their study Richard Nixon made more money for the interviews with David Frost which later became a movie than he had during his entire term in office.
The first dozen Presidents tended to be very well off but tied to commodity speculation and real estate and were not entirely stable figures with James Madison for example reaching $100 million than seeing significant depreciation.
Then there was a depression in 1837 for 6 years
For the full article see The Atlantic May 18 2010 Derek Thompson
Who was the richest President?