Prince Charles has invited son Prince Harry and Meghan Markle to stay with him when they next visit the U.K. with their children Archie and Lilibet Diana, the latter whom Charles is yet to meet. However, the Daily Mirror reports, this “olive branch” may be futile, given that Harry has indicated he is willing to launch a legal action against the British government over the security afforded to him and his family. Harry said in a statement over the weekend that it was not safe for the Sussexes to travel to the U.K. Royal sources confirmed to the Mirror that Charles has held several “good-natured and enjoyable” video calls with his son, Meghan, and their children in recent months, even though royal author Christopher Andersen claimed in November that it had been Charles who had raised queries about the color of the then-unborn Archie’s skin, in an incident first raised in the couple’s explosive interview with Oprah Winfrey.

The Mirror didn’t say if Harry and Charles have discussed this, or other personal matters Harry raised about his father and the royal family in interviews. “The Prince of Wales has been saddened that he hasn’t had the opportunity to spend time with his grandchildren, which he really does miss,” a source said. Charles and Harry have been on “a journey of rediscovery” in terms of their relationship, a source close to Charles added.