Meghan Markle and Prince Harry “never agreed” that they wouldn’t have a “half-in” approach to royal life, according to a friend.

The Duke and Duchess of Sussex stepped out in Nottingham last week for a series of charity visits, the couple’s first major joint outing since their recent move back to the U.K., prompting many comparisons to similar engagements they undertook as working members of the British royal family. In January 2020, they announced their intention to “step back as senior members of the royal family,” with terms of their decision discussed at the so-called “Sandringham Summit” led by the late Queen Elizabeth.

Although Queen Elizabeth rejected the idea of the couple being “half-in, half-out” then, ruling that they could not represent the monarchy while pursuing private commercial interests, a friend recently told Hello! magazine that Meghan, 45, and Prince Harry, 42, didn’t agree to that decision.

“It doesn’t trouble them,” the friend claimed. “They never agreed they wouldn’t be half in, half out. That was what was decided and foisted upon them.”

“They’re just living their lives and they can, because they’re financially independent,” the friend added.

PEOPLE has reached out to the Duke and Duchess of Sussex for comment.

Later in 2020, the couple settled in Montecito, Calif., where they raised their children — Prince Archie, 7, and Princess Lilibet, 5 — for six years before returning to the U.K. for an extended period two months ago.

Shortly after their return, King Charles issued a letter via the Lord Chamberlain reiterating that there was “no change” to the Duke and Duchess of Sussex’s status. The letter stated that Harry and Meghan are “no longer working Members of The Royal Family” and that their charitable work “is a personal matter for them both and undertaken in their private capacity. In short, their position is akin to private citizens with commercial and charitable interests.”

A source close to the Duke and Duchess of Sussex pushed back on the description as “private citizens,” telling PEOPLE, “They are public figures. Members of the public know who they are. The idea that they can live a normal, ‘private’ life is sadly not realistic.”

Also over the weekend, a friend of the Duke and Duchess of Sussex told The Times that they “hate” the phrase “working members of the royal family.”

“ ‘Working members of the royal family’ is a really lazy phrase. He hates it, she hates it,” the friend said. “They are not ‘working’ for the institution, but they are members of the royal family, and they work.”

The friend said that the Sussexes also dislike the term “former royals.”

“Things like Nottingham aren’t done in the name of the institution or the crown, but Harry is the son of the King, he is still a member of the royal family, he’s just not a member of the institution,” the friend told The Times. “It’s misleading to call them ‘former royals.’ It has an impact on how they’re perceived and the way people talk about them.”