It sounds as if he knows his way home and will eventfully make his way back. But there's always the chance he could get hurt, hit by a car or maybe someone else takes him in.

My heart was broken when I had my first cat. He stayed in the house, but in the morning he'd want to go out. Rarely out for longer than 1/2 an hour. Would always come back and sit by the back porch screen door. Well one morning he didn't come back before I was ready to leave for work. So I propped open the screen so he could get back on the porch and left a bowl of food and water. I fully expected to come home from work and find him curled up on one of the porch chair. Well he wasn't. For weeks I left that porch door open and at night turned on the porch light. But he never returned. Several months later I was transferred. Had to sell the house and relocate. I felt I was abandoning him. I swear, as I drove away from my house for the last time I saw Spooky sitting on the hillside behind my house. I turned my car around and drove back into the driveway, got out and ran behind the house. He was no where to be seen. Was it him? A ghost? Or my imagination? I'll never know. But I loved that cat. He'd curl up in my lap and sleep on my pillow at the top of my head. I've never forgot him.

So point of story, I've never let another cat out of the house.