Why we're so bad at predicting what will happen to us in the future

daniel-von-appen-tb4heMa-ZRo-unsplash.jpg

We all think the way things are now is the way things will continue to be. If you’re flying high, that’s not so bad, but if you’re falling, flailing or treading water like many of us right now, then this is a dangerous tendency.

Here’s how to counteract it.

See the failure you’re going through as a step up an invisible staircase toward a Future You in a Future Life you can’t even imagine yet.

The staircase represents your life so far. And you can’t see up the invisible staircase.

Look down behind you. That part is visible. You can see where you came from. All the steps you already walked up.

Look. There’s the time you moved in fifth grade and got bullied by that goon Adam every day after school.

Remember? That’s when you first picked up a basketball and started practicing with Coach Williams every night.

There’s Francesco, the tattooed chef who chewed you out every shift you showed up late to wash dishes at the seafood place as a teen. It was painful but you learned to be on time.

Prom — remember that disaster? I guess that night helped you realize you were gay.

So many steps up to today. Big steps. Hard steps. But steps all the same.

And what’s next on the staircase?

Well, that’s the problem.

No one knows.

It’s invisible. We can’t see the future. And maybe if that were the only problem, that would be okay. But it isn’t. It gets worse.

Why?

Because according to the research, we actually think we can see up that staircase.

Our brains think, “Oh yeah, sure, I know what’s next in my life.” In reality, we suck at it. Let me explain.

In 2013, Science published a fascinating study conducted by the researchers Jordi Quoidbach, Daniel T. Gilbert and Timothy D. Wilson. They teamed up to measure the personalities, values and preferences of more than 19,000 people ages 18 to 68. In a series of tests, they asked the subjects about two pretty simple things: how much they thought they had changed in the past decade and how much they would change in the next decade.

They used a lot of scientific methods to make sure the data were legit, then they published their results. Academic circles started buzzing. Media outlets clamored to share the results.

Why?

Because the results were mind-blowing.

It turned out that no matter how old the respondents were, they uniformly believed that they had changed a ton in the past but would change little in the future.

What?

Imagine a 30-year-old guy telling the tempestuous story of his last 10 years but figuring his next 10 years would be smooth sailing. Imagine a 50-year-old woman talking about how everything had flip-flopped after she turned 40 but then assuming that at 60, she’d be the same person she was now. That was the case for everybody regardless of age, gender or personality.

We all do it.

We all think that the way things are now is the way things will continue to be.

If you’re flying high, that’s maybe not a bad thing, but if you’re falling, if you’re busted, if you’re heartbroken, if you’re lonely, then this is a dangerous psychological tendency. And we all share it.

When we’re at rock bottom, we are certain that there’s no way up. We think we’ll never get out of our parents’ basement. We think our divorce means we’ll never meet someone new. If we’ve lost our jobs, we think we’ll be scrolling online postings forever.

The researchers called this the “end of history illusion.” We think everything will remain unchanged from here on out.

Why did those researchers study go to the effort of 19,000 people? Gilbert went on NPR’s Hidden Brain and explained, “You know, like everybody, I suffer the slings and arrows of outrageous fortune. You know, we have divorces. We have surgeries. We have breakups with women we love and friends we enjoy. So it was sort of ordinary events that befell me all in one year. And I realized that, had you asked me a year earlier how I would be faring, the answer would have been, ‘Oh, my gosh, I’ll be devastated.’ But I wasn’t devastated…. And it made me wonder if I was the only person who was just too stupid to be able to look ahead into his future and figure out how he’d feel if really bad or maybe really good things happened.”

There it is. The invisible staircase.

Even Gilbert, the famed Harvard psychologist and professor, the author of bestselling books such as Stumbling on Happiness, even he forgets that the rest of the staircase is invisible. He went through a flop or two and figured, “Well, darn, my life’s gonna suck forever.” But it didn’t.

Inevitably, everything we go through in life really is a step to help us get to a better place.

It’s hard to see it this way. But we have to, because this study helps us realize we’re prone to catastrophizing. That alone should be enough to zoom backward in your brain and go, “Wait a minute here. I’m tricking myself! Who’s to say I won’t get out of the basement? Meet someone new? Land a plum gig I love?”

See it as a step.

Gilbert ended up figuring out that when it comes to predicting the future, we’re all stupid. Each and every one of us.

Doesn’t that feel better?

This research reminded me of an HR job I had where I had to escort bosses into meeting rooms whenever they had to fire an employee. I was there for paperwork, for witnessing, for emotional support. I was in the room when dozens of people got fired, and it was awful. There were tears and wet tissues and many afternoons when I’d be consoling someone in a freezing parking lot as they loaded up their trunk with framed pictures from their desk saying “I thought I’d be here forever” and “What am I going to do now?” and “I’ll never find another job.”

Those scenes left me heartbroken. I lost a lot of sleep over them.

Sometimes I’d bump into the former employees years later. And what did they tell me? “Getting fired was the best thing that happened to me! If I hadn’t gotten that severance package, I never would have had those crucial six months to spend with my dad before he died.”

Or: “I traveled to Peru and became a nutritional supplement importer, and I love what I’m doing now!”

Or: “I’m working at a smaller company now, and I’ve gotten promoted twice in two years!”

Or: “I used my severance pay to take the time to be with my daughter and son-in-law in the months after her third miscarriage.”

Why did every fired employee tell me this? Why did they all react so positively after some time had passed? How can that happen?

Because we confuse the challenge of picturing change with the improbability of change itself.

We do.

We confuse the challenge of picturing change (“What am I going to do now?”) with the improbability of change (“I’ll never find anything!”).

In other words, you can’t picture yourself changing so you assume that you won’t.

Why?

Because your seeing skills are shit!

And so are mine. So are everyone’s. You think because you can’t see up the staircase there aren’t any more steps. But there are more steps.

And change will come.

It always does.

That’s why it’s so hard to see change as a step. To see this failure, this flop, this difficult life experience as part of a process, as part of a greater whole. It’s hard to see it as a step because you can’t see the next step. And you sure can’t see 10 steps after that.

Why do we always think failure leads somewhere bad? It’s not true. It rarely is. Remember the end of history illusion. Our brains think this is the end. Remember all those people I met after they were fired saying how positive that left turn ended up being?

It’s me, too. How could I have known that failing at P&G would somehow lead us to having the conversation we’re having right now? I couldn’t have. Believe me, I far prefer having this conversation to doing price analysis on eye shadows and mascaras. But when I flamed out there, I pictured myself sleeping in a pile of club sandwich crusts in Cleveland.

So be kind to yourself.

When you’re there, when you’re stewing in the shock of failure and loss, when you’re convinced you’re stuck, when you’re convinced there’s no way forward, just remember: There’s a staircase you’re not seeing. Trust that it’s there, right in front of you, and that it leads to exciting new places. Have the courage to believe in this one thing that you can’t see.

There are so many steps ahead. So many steps. Don’t stop. Shift the spotlight, and keep moving.

It’s very possible and very likely that what you’re going through is a step toward a future you’ll be happy with. But you just can’t see it … yet.

An earlier version of this article was published on TED.com