Gold vs S&P 500 Since 1971: Returns and the Tax You Pay

Gold vs S&P 500 Since 1971: What the Data Actually Shows

From August 1971 through end of 2023, the S&P 500 compounded at roughly 10.7–11.0% annually (total return, dividends reinvested) versus gold's approximately 7.7–8.0% nominal CAGR. On a $1 million starting investment, that gap compounds to approximately $180 million for equities versus $65 million for gold over 52 years. The spread is real, persistent, and larger than most gold advocates acknowledge.

But the headline number obscures what actually matters for portfolio construction at the $5M+ level: when gold works, why it works, and what it costs you after taxes to hold it.

The Nixon Shock Set the Starting Line

On August 15, 1971, President Nixon suspended dollar convertibility to gold, ending the Bretton Woods system and freeing gold from its fixed $35 per ounce price. That single policy decision created the comparison we're analyzing. Before 1971, gold wasn't an investable asset in any meaningful sense for U.S. investors, its price was administratively set.

The immediate aftermath was dramatic. Gold moved from $35 to roughly $183 per ounce by 1974, then continued climbing through the decade. According to Federal Reserve Bank of St. Louis (FRED) historical price data, gold reached approximately $850 per ounce by January 1980 before collapsing. That 1970s surge is not a footnote. It defines gold's entire 52-year track record in ways the aggregate CAGR conceals.

The S&P 500, meanwhile, spent most of the 1970s struggling against stagflation. Stocks returned roughly 75% nominally from 1971 to 1980 (including dividends), while gold returned approximately 1,300% over the same period. Understanding this single decade is prerequisite to understanding everything else in this comparison.

Has Gold or the S&P 500 Performed Better Since 1971?

The S&P 500 wins on total return, and it isn't particularly close. But the decade-by-decade breakdown tells a more useful story than the aggregate figure.

The 1970s were gold's decade by a historic margin. Every other decade since 1980 has favored equities, often substantially. According to Morningstar's Ibbotson SBBI data, large-cap U.S. stocks have compounded at approximately 10% annually over multi-decade periods, a benchmark gold has not matched outside of inflationary crisis environments.

Sources: FRED gold price data, S&P Dow Jones Indices total return data, Morningstar Ibbotson SBBI. Returns are approximate and rounded.

The critical insight: if you exclude the 1970s, gold's CAGR from 1980 through 2023 falls to approximately 3.5–4.0% nominal. The S&P 500 returned roughly 11% over the same period. Gold's 52-year track record is largely an artifact of one extraordinary inflationary episode tied to the unique monetary transition of 1971–1980. Whether that episode repeats is the actual question worth debating.

For long-term S&P 500 returns context, the equity compounding story is well-documented across multiple market cycles.

What Is the Average Annual Return of Gold vs the S&P 500 Over 50 Years?

The aggregate numbers, stated plainly:

• Gold CAGR (1971–2023): approximately 7.7–8.0% nominal
• S&P 500 CAGR (1971–2023): approximately 10.7–11.0% nominal (total return with dividends)
• Cumulative CPI inflation (1971–2023): approximately 650%, per Bureau of Labor Statistics data

In real terms, both assets have preserved and grown purchasing power since 1971. BLS CPI data shows that $1 in August 1971 required approximately $7.50 in 2023 to maintain equivalent purchasing power. Deflating nominal returns against that baseline, the S&P 500's real CAGR runs approximately 6.5–7.0% while gold's real CAGR comes in around 3.5–4.0%.

For a more granular look at inflation-adjusted historical performance, the gap between nominal and real returns matters significantly for long-horizon wealth planning.

The historical average annual returns for the S&P 500 are well-documented, but the relevant comparison for FATFIRE investors is after-tax real returns, which is where the analysis gets more consequential.

What Are the Tax Implications of Holding Physical Gold vs Gold ETFs?

This is where the comparison shifts most sharply for investors in the top federal bracket, and it's the section most financial articles skip entirely.

The IRS classifies physical gold, gold coins, and gold ETFs structured as grantor trusts (GLD is the primary example) as collectibles under IRC Section 1(h)(4). Long-term capital gains on collectibles are taxed at a maximum 28% federal rate. By contrast, equity index funds like VOO or SPY held over one year are taxed at 20% plus the 3.8% Net Investment Income Tax, an effective 23.8% rate for investors subject to NIIT.

Source: IRS Publication 544, IRC Section 1(h)(4). State taxes apply additionally and vary by jurisdiction.

The 4.2 percentage point rate difference (28% vs 23.8%) compounds meaningfully over decades on a $5M+ position. On a $500,000 gold allocation generating 7.5% annually over 20 years, the additional tax drag versus an equivalent equity position runs to several hundred thousand dollars in after-tax wealth.

Holding gold in a tax-advantaged account eliminates this disadvantage but introduces separate constraints. Under IRC Section 408(m), IRAs cannot hold collectibles directly. Physical gold is permitted in an IRA only through an approved custodian using IRS-qualified storage, not the home safe or bank deposit box. Gold ETFs held inside an IRA avoid the collectibles rate entirely, which is the most tax-efficient structure for most FATFIRE investors who want gold exposure without the custody complexity.

How Has Gold Performed During S&P 500 Bear Markets Since 1971?

Gold's defensive reputation is partially earned and partially overstated. The nuance matters.

According to NBER recession dating, the U.S. has experienced 8 recessions since 1971. Gold's behavior across those episodes has not been uniform. During the stagflationary recessions of the 1970s, gold was the clear winner. During the 2008 financial crisis, gold initially sold off alongside equities in Q4 2008 as investors raised cash, then recovered sharply and finished the year roughly flat while the S&P 500 fell approximately 37%.

Sources: FRED gold price data, S&P Dow Jones Indices, NBER recession dates. Returns are approximate.

The March 2020 data point is the one most institutional investors flag. Gold and equities sold off simultaneously as investors liquidated everything to raise cash. Gold's correlation with the S&P 500 has averaged approximately -0.05 to +0.10 over rolling 10-year periods since 1971, making it one of the few liquid assets with near-zero equity correlation in normal markets. But during acute liquidity crises, that correlation has spiked above 0.5 temporarily, precisely when diversification is most needed.

The World Gold Council's 2024 strategic asset research acknowledges this dynamic while still recommending gold for tail-risk hedging, noting that the correlation spike during liquidity events is typically short-lived and followed by divergence as gold recovers faster than equities.

Reviewing worst 10-year return periods for the S&P 500 illustrates the specific windows where gold's defensive properties were most valuable.

Is Gold a Good Inflation Hedge Compared to Equities?

The evidence is mixed, and anyone telling you otherwise is selling something.

Gold's inflation-hedging reputation derives almost entirely from the 1970s. During that decade, CPI inflation averaged over 7% annually, and gold returned approximately 1,300% nominally. The correlation between gold and inflation was high and the narrative was clean.

Outside of that episode, the relationship is considerably weaker. Gold lost roughly 28% in real terms during the 1980s despite positive inflation throughout the decade. During the 2021–2022 inflation surge, gold returned approximately -2% in 2022 while CPI ran at 40-year highs. Equities also struggled in 2022, but the point stands: gold does not reliably track near-term inflation.

The more accurate framing is that gold hedges against monetary system stress and currency debasement fears, not against routine CPI increases. When investors believe the monetary system itself is under threat, gold performs. When inflation is high but the system appears stable, the correlation is unreliable.

Equities, by contrast, have a stronger long-run inflation-hedging record through earnings growth and pricing power, though they can underperform inflation over shorter windows. For S&P 500 performance against inflation, the long-run real return data supports equities as the more consistent inflation beater over 20-plus year horizons.

What Percentage of a High-Net-Worth Portfolio Should Be Allocated to Gold?

The World Gold Council's 2024 research and independent academic work generally converge on a 4–15% range for institutional and high-net-worth investors, with the Sharpe ratio improvement peaking around 5–10% allocation in a traditional 60/40 portfolio. Allocations above 15% begin to meaningfully drag long-term returns without proportionate risk reduction.

For a $5M portfolio, a 5% gold allocation ($250,000) is a reasonable starting point. The implementation decision matters as much as the allocation percentage.

Physical gold carries storage costs of approximately 0.5–1.0% annually for insured vault storage, plus insurance, plus the 28% collectibles tax rate on gains. For a $250,000 position, annual carrying costs run $1,250–$2,500 before any tax consideration.

iShares Gold Trust (IAU) charges a 0.25% expense ratio with no storage or insurance burden. The tax treatment is identical to physical gold (collectibles rate), but the operational simplicity and cost differential are significant over a 10–20 year holding period.

Gold mining equities (GDX, GDXJ, or individual miners) are taxed as ordinary equities at 23.8% effective rate for FATFIRE investors, but they introduce company-specific risk, operational leverage to gold prices, and correlation to broader equity markets that reduces the diversification benefit.

Vanguard's portfolio construction research has found that commodities including gold add limited long-term return but can reduce portfolio volatility and drawdown severity at modest allocations, particularly during stagflationary environments resembling the 1970s.

Rebalancing frequency matters. Annual rebalancing back to target allocation captures some of gold's mean-reversion tendency and prevents a strong gold run from pushing allocation above the 15% threshold where return drag becomes material.

How Do Gold and S&P 500 Correlations Change During Recessions?

Gold's near-zero average correlation with equities is one of its most cited portfolio benefits, and the data supports it in normal market conditions. Over rolling 10-year periods since 1971, the correlation between gold and the S&P 500 has averaged approximately -0.05 to +0.10, per analysis of FRED price data.

The complication is that correlations are not stable. During the 2008 financial crisis and the March 2020 COVID crash, gold briefly sold off alongside equities as institutional investors raised cash across all liquid assets. Correlation spiked above 0.5 during those acute phases before diverging as gold recovered faster.

This pattern has two practical implications for portfolio construction. First, gold's diversification benefit is most reliable during slow-moving bear markets driven by valuation compression or earnings deterioration, not during sudden liquidity crises. Second, the correlation spike during crises is typically short-lived. Investors who held gold through March 2020 saw it recover to new highs within months while equities took longer to recover.

For rolling 10-year returns analysis on the equity side, the variability in S&P 500 returns across different starting points reinforces why low-correlation assets have portfolio value even when their standalone returns are lower.

The 20-year rolling returns for long-term investors show that equity return variability compresses significantly at longer horizons, which affects how much diversification value gold provides depending on your investment time horizon.

Implementation Considerations for $5M+ Portfolios

Beyond the allocation percentage, the structural decisions around gold ownership have meaningful consequences at FATFIRE scale.

Custody and storage for physical gold at $250,000+ typically means allocated storage at a recognized vault (Brinks, Malca-Amit, or through a private bank's commodity desk). Annual costs at this scale run 0.4–0.8% including insurance. Some private banks offer allocated gold accounts that simplify custody but add counterparty exposure. Unallocated gold accounts at banks are unsecured creditor claims, not physical ownership, a distinction that matters in a stress scenario.

ETF selection at this scale warrants attention to the GLD vs. IAU distinction. GLD (SPDR Gold Shares) has a 0.40% expense ratio versus IAU's 0.25%. On a $500,000 position, that 0.15% difference is $750 annually, compounding over decades. Both are structured as grantor trusts and receive collectibles tax treatment.

Gold in estate planning receives a step-up in cost basis at death under current law, which partially offsets the 28% collectibles rate for buy-and-hold positions. For investors with significant unrealized gains in gold positions, this is a relevant consideration in deciding whether to rebalance or hold.

Mining equities deserve mention as a distinct category. They provide equity tax treatment (23.8% effective rate vs. 28%), but their correlation to the S&P 500 is substantially higher than physical gold, reducing the diversification benefit. They also carry operational risk, currency exposure, and management risk that physical gold does not. They are not a substitute for gold in a portfolio construction context.

For context on how alternative assets compare, private equity returns compared to equities illustrates the broader alternatives landscape that FATFIRE investors typically consider alongside gold.

The Structural Case for Gold at This Level

The honest summary: the S&P 500 has outperformed gold by a wide margin since 1971, and that outperformance is likely to persist over most future multi-decade periods. Gold's 52-year CAGR is substantially driven by a single inflationary episode that may not repeat.

The case for holding gold at 5–10% of a large portfolio is not a return argument. It is a correlation and tail-risk argument. Gold is one of the few liquid assets with near-zero long-run equity correlation that also tends to perform during the specific scenarios, currency debasement, monetary system stress, geopolitical shock, that are hardest to hedge through conventional equity diversification.

At $5M+, the question is not whether gold beats the S&P 500. It almost certainly won't over a 20-year horizon. The question is whether a $250,000–$500,000 gold position improves portfolio-level risk-adjusted returns and provides insurance against the tail scenarios your equity portfolio cannot hedge. For most FATFIRE investors, the answer is yes, at modest allocation, held in the most tax-efficient structure available.

The 10-year market performance trends and historical valuation metrics for equities provide useful context for calibrating how much tail-risk hedging is warranted given current market conditions.

References

• Federal Reserve Bank of St. Louis (FRED), "Gold Fixing Price 10:30 A.M. (London time) in London Bullion Market, based in U.S. Dollars" (2024). - Federal Reserve Bank of St. Louis (FRED), "S&P 500 Index (SP500)" (2024). - IRS, "Publication 544: Sales and Other Dispositions of Assets" (2023).

• IRS, "IRC Section 1(h)(4): Maximum Capital Gains Rate for Individuals / Collectibles" (current). - Morningstar, "Ibbotson SBBI 2023 Classic Yearbook: Market Results for Stocks, Bonds, Bills, and Inflation" (2023). - World Gold Council, "Gold as a Strategic Asset: 2024 Edition" (2024). - Vanguard, "Vanguard's Framework for Constructing Diversified Portfolios" (2023). - Journal of Financial Planning, "The Role of Gold in a Modern Portfolio: Evidence from 50 Years of Data."

• Bureau of Labor Statistics, "CPI Inflation Calculator / Consumer Price Index Historical Data" (2024). - NBER (National Bureau of Economic Research), "US Business Cycle Expansions and Contractions" (2024). - S&P Dow Jones Indices, "S&P 500 Index" (2024).