Semi-Retirement Is a Tax Strategy, Not Just a Lifestyle Choice

Semi-Retirement Is a Tax Strategy, Not Just a Lifestyle Choice

For anyone with a $5M+ portfolio, semi-retirement deserves serious financial analysis before it gets filed under "nice idea for later." Reducing work to 10-20 hours per week creates a narrow window to execute Roth conversions, shelter income in tax-advantaged accounts, and manage sequence-of-returns risk during the most vulnerable years of a drawdown portfolio. Done right, semi-retirement doesn't just feel better than grinding full-time. It compounds better, too.

How Much Do You Need to Semi-Retire Comfortably?

The standard 4% rule doesn't map cleanly onto a 40-year retirement horizon. At a 3.5% safe withdrawal rate, which the Journal of Financial Planning considers more appropriate for early retirees given sequence-of-returns risk, a $5M portfolio supports roughly $175,000 per year in inflation-adjusted spending without meaningfully depleting principal.

Add $50,000-$80,000 annually from consulting, board work, or advisory roles, and your sustainable lifestyle budget rises to $225,000-$255,000 per year. Alternatively, that earned income allows your portfolio to compound for an additional 5-7 years before significant drawdown begins. That's not a trivial difference. A $5M portfolio growing at 6% real for five extra years becomes approximately $6.7M before you touch it.

The math changes the framing entirely. Semi-retirement for a FATFIRE individual isn't about needing the paycheck. It's about the 4 percent rule for retirement planning and recognizing that delaying full drawdown is one of the most powerful levers available.

What Is the Best Age to Semi-Retire with $5 Million?

The honest answer: it depends less on age than on tax bracket, healthcare access, and Social Security timing. But the age windows carry real financial consequences.

Semi-retiring before 59½ requires careful planning around distributions. The IRS allows substantially equal periodic payments (Rule 72(t)) from retirement accounts before age 59½ without the 10% early withdrawal penalty, but once you start a 72(t) schedule, you're locked in for five years or until age 59½, whichever is longer. Modifying the schedule triggers retroactive penalties on all prior distributions.

Between 59½ and 65, you have penalty-free access to retirement accounts but no Medicare. Private health insurance for a couple can run $20,000-$30,000 annually on the open market, according to the Centers for Medicare and Medicaid Services. That's a real cost that belongs in your semi-retirement budget, not a footnote.

From 62 to your full retirement age (67 for those born after 1960), Social Security is technically available but comes with a significant catch. The Social Security Administration's rules are clear: earning income before reaching full retirement age reduces benefits by $1 for every $2 earned above the annual exempt amount ($22,320 in 2024). If you're earning $80,000 in consulting income and claim Social Security at 62, you could forfeit a meaningful portion of your benefit.

Delaying Social Security to 70 increases your benefit by approximately 8% per year past full retirement age. For someone with a $5M portfolio, the portfolio can easily bridge the gap. The lifetime value of that delay frequently exceeds $200,000 in present value terms for a healthy individual.

Tax Optimization Windows During Semi-Retirement

This is where semi-retirement earns its keep for high-net-worth individuals. The years between leaving full-time employment and drawing full Social Security or required minimum distributions represent a limited window of lower taxable income. Most people waste it.

The IRMAA surcharges alone justify aggressive planning. For 2024, Medicare IRMAA surcharges begin at $103,000 MAGI for individuals and $206,000 for couples, adding up to $5,000 or more per person annually in Medicare Part B and D premiums. If your full-retirement MAGI routinely exceeds $300,000 from RMDs and Social Security, the semi-retirement years at lower income are your best opportunity to execute Roth conversions at 22-24% marginal rates rather than 32-37% rates later.

A semi-retiree with $3M in traditional IRA assets converting $150,000-$200,000 per year during a 5-year semi-retirement window could shift a substantial portion of that balance to Roth before RMDs begin at age 73, potentially saving six figures in lifetime tax liability. The strategies for withdrawing from retirement accounts matter enormously here, and the sequencing decisions made during semi-retirement often have more impact than any single investment decision.

Semi-retired individuals who structure consulting or freelance work as a pass-through entity may also qualify for the 20% qualified business income deduction under IRC Section 199A, substantially reducing effective tax rates on that earned income. A $100,000 consulting income with a QBI deduction effectively becomes $80,000 of taxable income, worth roughly $5,000-$8,000 in annual tax savings depending on your bracket.

How Semi-Retirement Income Affects Social Security Benefits

The mechanics here are misunderstood even by sophisticated investors. The earnings test only applies before you reach full retirement age. Once you hit FRA, you can earn unlimited income without any reduction in Social Security benefits.

If you semi-retire at 60 and earn $80,000 annually in consulting income, you'd lose approximately $28,840 per year in Social Security benefits if you claimed at 62 (the $80,000 minus the $22,320 exempt amount, divided by 2). The SSA does credit those withheld amounts back to you once you reach FRA by recalculating your benefit upward, but the recoupment math is complex and the break-even point typically falls in your mid-70s.

For most FATFIRE individuals with significant portfolio assets, the cleaner strategy is to delay claiming entirely and let the 8% annual delayed retirement credits accumulate to age 70. Semi-retirement earned income supports living expenses during that delay without forcing portfolio drawdown. Working past retirement age in a reduced capacity is one of the few strategies that simultaneously improves Social Security outcomes, reduces portfolio drawdown, and maintains the cognitive engagement that NBER research links to better health outcomes in later life.

Solo 401(k) Contributions: The Overlooked Semi-Retirement Advantage

Here's the counterintuitive part. Most people assume that at $5M+ net worth, retirement account contributions are irrelevant. They're wrong.

Consulting or board director income during semi-retirement can fund a Solo 401(k) contribution of up to $69,000 in 2024 (combined employee and employer contributions for those under 50, with a $7,500 catch-up for those 50 and older). A semi-retired individual earning $100,000 in consulting income could shelter up to $69,000 in a Solo 401(k), reducing taxable income substantially while continuing to build tax-advantaged assets.

The math: $69,000 sheltered at a 32% marginal rate saves $22,080 in federal taxes in a single year. Over a 5-year semi-retirement period, that's over $110,000 in tax savings, assuming consistent consulting income. That's real money even at this wealth level, and it's entirely unavailable to someone who has stopped earning income entirely.

This strategy pairs well with Roth conversion planning. Contribute to a traditional Solo 401(k) to reduce current-year taxable income, then convert prior-year traditional IRA balances to Roth in the same year to fill lower brackets. The net effect is tax arbitrage across both current and future liabilities.

Concentrated Positions and Semi-Retirement Timing

Many FATFIRE individuals built their wealth through a concentrated equity position, whether a company they founded, stock compensation, or a single investment that ran. Generic retirement articles never address this. Semi-retirement planning should.

A charitable remainder trust (CRT) allows you to contribute appreciated stock, avoid immediate capital gains tax, receive an income stream for a defined period or life, and take a partial charitable deduction. The income stream from a CRT can effectively fund semi-retirement living expenses while the underlying asset diversifies out of the concentrated position over time.

A donor-advised fund (DAF) works differently but serves a complementary purpose. Contributing appreciated stock to a DAF in a high-income year generates an immediate charitable deduction, reduces MAGI, and allows you to distribute grants to charities over time. For someone transitioning out of a high-earning role, the year of departure often creates a spike in taxable income from equity vesting or business sale proceeds. Timing a large DAF contribution to that year can offset a significant portion of that income.

These strategies connect semi-retirement planning directly to estate and philanthropic goals, which is where the real planning leverage exists for achieving FatFIRE with luxury at this wealth level.

Work Structures for High-Net-Worth Semi-Retirees

The Transamerica Center for Retirement Studies' 2024 survey found that phased retirement and part-time consulting rank among the most cited strategies for workers planning to remain professionally engaged in retirement. For FATFIRE individuals, the structure of that engagement matters as much as the hours.

Research from the Stanford Center on Longevity suggests that maintaining even minimal professional engagement, as few as 10 hours per week, produces significantly higher life satisfaction and purpose scores than full retirement. The effect plateaus quickly: working more than 20 hours per week in retirement eliminates most of the leisure benefit. That 10-20 hour window is the quantifiable target.

Structuring consulting income through an S-Corp rather than a sole proprietorship can reduce self-employment taxes by splitting income between salary (subject to payroll taxes) and distributions (not subject to payroll taxes). On $100,000 of net consulting income, the self-employment tax savings from an S-Corp election can reach $5,000-$10,000 annually, depending on the salary-to-distribution ratio your accountant establishes.

Health Insurance Before Medicare: The Real Cost

Private health insurance is the most frequently underestimated cost in pre-65 semi-retirement planning. The Centers for Medicare and Medicaid Services confirm that Medicare eligibility begins at 65 regardless of employment status, meaning any semi-retiree who exits employer coverage before then is on their own.

For a couple in their late 50s or early 60s, comprehensive private coverage on the open market runs $20,000-$30,000 annually. ACA marketplace plans are available, but FATFIRE-level income typically disqualifies you from subsidies. The ACA subsidy cliff at 400% of the federal poverty level phases out entirely, and at $5M+ net worth with any meaningful portfolio income, you're well above it.

The practical options for health insurance options for early retirees at this wealth level are narrower than most articles suggest. COBRA continuation coverage from a prior employer lasts 18 months and preserves existing networks, but premiums are often high. A high-deductible health plan paired with an HSA allows you to contribute up to $8,300 for a family in 2024, with triple tax advantages: deductible contributions, tax-free growth, and tax-free withdrawals for qualified medical expenses. After 65, HSA funds can be withdrawn for any purpose and taxed as ordinary income, functioning essentially as a traditional IRA.

Long-term care insurance, if you're going to purchase it at all, is most cost-effective when acquired in your mid-50s. Premiums roughly double between age 55 and 65. At $5M+ net worth, self-insuring is a legitimate option, but the decision should be explicit rather than default.

Building a Semi-Retirement Income Portfolio

The portfolio architecture for semi-retirement differs from both accumulation and full-retirement structures. You need enough liquidity to cover 2-3 years of expenses without touching equities during a downturn, enough income generation to reduce drawdown pressure, and enough growth exposure to sustain a potentially 40-year horizon.

A practical structure for a $5M portfolio in semi-retirement might hold 18-24 months of living expenses in short-term Treasuries or money market funds, a dividend-oriented equity sleeve generating 2-3% yield on the equity portion, and a real assets allocation (REITs, infrastructure, or direct real estate) providing inflation-linked income. The goal is to build a retirement income portfolio that doesn't force you to sell growth assets during the first decade of drawdown, which is precisely when sequence-of-returns risk is most acute.

The Journal of Financial Planning's research on sequence-of-returns risk confirms that the first 10 years of portfolio drawdown determine outcomes more than any other variable. A 20-30% market decline in year two of full retirement can permanently impair a portfolio's longevity. Semi-retirement's earned income acts as a buffer against exactly this scenario, allowing equities to recover before meaningful drawdown begins.

For those with real estate holdings, semi-retirement is also a natural time to evaluate whether active management of rental properties still fits the lifestyle goals. Transitioning to passive real estate exposure through DSTs (Delaware Statutory Trusts) or private REITs can preserve the income stream while eliminating the operational burden.

The Non-Financial Side of Semi-Retirement

The NBER's research on retirement and health is worth taking seriously: abrupt full retirement is associated with accelerated cognitive decline, while gradual phased retirement preserves mental acuity. This isn't motivational content. It's a longevity variable that belongs in the planning model.

The non-financial aspects of retirement planning matter at this wealth level precisely because money is no longer the binding constraint. Identity, structure, and social connection become the variables that determine whether the transition works. People who build the post-career life before they leave the career tend to transition better than those who assume the free time will fill itself.

Semi-retirement provides a natural testing ground. Working 10-15 hours per week while building out the activities, relationships, and projects that will anchor full retirement is a lower-risk approach than a hard stop. You calculate your financial independence timeline and then actually test the lifestyle before committing to it fully.

From Semi-Retirement to Full Retirement: The Transition Triggers

Rather than setting a target age for full retirement, FATFIRE individuals are better served by defining financial and personal triggers. The financial triggers worth tracking:

Portfolio has reached a level where the 3.5% SWR covers full spending without earned income. Roth conversion window is substantially complete. Social Security has been claimed or the delay-to-70 strategy is locked in. Healthcare transition to Medicare is within 12 months.

The personal triggers are harder to systematize but equally real. The consulting work has stopped being engaging and started feeling like obligation. The 10-20 hour week has drifted to 30 hours. The activities that were supposed to fill retirement are actually filling it.

Working past retirement age in a reduced capacity is worth continuing as long as it's genuinely optional and genuinely enjoyable. The moment it becomes neither, the financial math almost certainly supports stopping. At $5M+, you did the work. The semi-retirement phase is for optimizing the transition, not for proving you still can.

Use a partial retirement calculator to model the specific scenarios relevant to your portfolio size, tax situation, and target lifestyle. The numbers will be more instructive than any rule of thumb.

References

• Social Security Administration -- "How Work Affects Your Benefits (Publication No. 05-10069)" (2024).
• IRS -- "Publication 590-B: Distributions from Individual Retirement Arrangements (IRAs)" (2024).
• IRS -- "IRC Section 199A: Qualified Business Income Deduction"
• Centers for Medicare and Medicaid Services -- "Medicare & You 2025 Handbook" (2025).
• Transamerica Center for Retirement Studies -- "24th Annual Transamerica Retirement Survey" (2024).
• Journal of Financial Planning -- "Sequence-of-Returns Risk and Safe Withdrawal Rates in Early Retirement" (2023).
• Vanguard -- "How America Saves 2024" (2024).
• National Bureau of Economic Research -- "The Long-Run Effects of Retirement on Health (Working Paper)" (2022).