“I made Yellowstone the first priority, and to insinuate anything else would be wrong,” he said in the May interview.

Despite the drama, Grimes added that Yellowstone had naturally run its course.

“It feels like good timing,” he said of the series coming to an end. “Seven years is a long time to do anything, and I had never played a character for more than a few months before.”

The 50 Shades of Grey alum added that there were still good times: “I’ve loved all of it.”

 “But there is a part of it that — once you read those final episodes — sees that there’s a finality to it. There’s always talks of spin-offs,” Grimes said, noting that he’d be open to doing one.

“I’d do it [but] I just don’t understand how it would work once the story ends. Kayce wants to cowboy and be happy with his family. He doesn’t want to kill people anymore. He doesn’t want the weight of a huge mega ranch that isn’t sustainable in today’s times. He wants his little slice of heaven. It’s that simple,” he said of his character’s fate.

Grimes’ comments to Esquire about Costner’s exit echoed statements he previously told PEOPLE.

“Him not coming back felt like, ‘Okay, we’re going to have to land this plane now for real,’ ” Grimes told people of the storyline.

“I think the patriarch leaving was always going to be part of the story. That’s always where it was headed is like, what do these kids do? What does this family do when their rock is gone?”

He also admitted that it “felt a little different” not having Costner around, calling the cast “a big family now.”

All five seasons of Yellowstone are streaming on Peacock and Paramount+. The series finale airs Saturday Dec. 15.