From a certain point of view...
In his later years Old Ben almost became an acceptable member of the community. He even considered relocating from his cave in the dunes to somewhere in town. He took visitors and people were happy to come. Even Owen Lars begrudgingly allowed his adopted son Luke Skywalker to visit him.
Things hadn't always been that way. When he first appeared, Ben Kenobi had been a controversial figure at best, so closely tied with the bloody chaos of the Clone Wars that many people actually feared him. And with good reason. There were rumors that he had gone wild and begun murdering his fellow Jedi, including his own friends.
To put it mildly, when he first came to Tatooine, Old Ben didn't play well with others. It was a self-imposed exile through and through, although in truth no one wanted him around anyway. There were...incidents, nothing that had gotten too far out of hand, but his reputation had grown increasingly poor.
That was why his son was taken away from him. The boy was young, too young to have remembered anything, only a baby when they had come. There was no one who didn't wonder if the boy was even Old Ben's, but there had been medical confirmations, even if they had no bearing in the decision to place him in a foster home.
That home was Owen's.
There was one thing Owen and Old Ben agreed on, and it was the comforting illusion that Luke's only relationship with Old Ben was that of friendship, something that was only possible because of Beru, who fought constantly if quietly for Old Ben, should he ever earn it, to have a chance to prove himself safe, at the very least. She would visit the hermit periodically to keep tabs.
On Luke's eleventh birthday, she decided it was time. Old Ben had settled down. He began holding normal conversations for the first time since arriving. Until that point he had been almost a mute, antisocial, so that most people assumed he had all but confirmed what was said about him. On a few occasions, he had flashed a temper, and he was known to hold grudges against those who wronged him. All that began to change. He grew restful in his isolation, and at Beru's prodding even learned to reintegrate himself with others.
He brought a gift to present the boy, a small trinket that had no identifiable purpose, probably some Jedi gadget that no longer functioned. Luke was fascinated. Owen was sparing in such matters. It was just what was needed to establish a connection. Soon the boy was venturing off on his own, with sufficient supervision from droids, to the hermit's cave in the dunes.
Beru watched with melancholy in her eyes, which was at least better than Owen's anger. He never thought Old Ben was worthy of such a chance, regardless of whatever might have been the truth about him.
As the years advanced, she wondered if Old Ben would ever tell Luke the truth. She couldn't say why she had ever considered giving him a chance. But she never regretted it.