The application, for three homes and alterations to access on land at The White House in Tilford Road, was decided at a meeting on Tuesday.
Waverley's development control committee overturned the southern area sub committee's recommendation to allow it.
The application had gone to development control because planning officers had recommended refusal, on the grounds that it wasn't the most effective use of urban land.
Chairman Ann Mugford said the southern area sub committee debate had been "immensely difficult".
"The majority thought that it was a case of 'better the devil you know', with three houses presumably preferable to anything else that might come along," she said.
Two Hindhead sub committee members David Harmer and Peter Isherwood supported the application because they feared that if they didn't, up to 17 houses could follow.
Mr Harmer focused on Hindhead's traffic problems. "A recent Highways Agency report indicated that there has been a growth of 50 per cent in traffic on the A287," he said. "Already we have up to four hours a day when traffic is backed up way beyond this site, with delays in excess of 20 minutes."
He claimed this delay could more than double once the new access opens at the 68-home Tyndalls Pine development.
Mr Isherwood was concerned about the lack of local infrastructure.
"Public transport is very, very limited. Beacon Hill School has three of the six classes full and how many more can use the doctors' surgery?" he said. "We are fast approaching the limit."
Pam Hibbert said that an assumption had been made that if permission wasn't given for this application, then another would follow for between 10 and 17 homes.
"But we don't know that," she said. "If an application for 10 homes comes along, this committee would probably refuse it."
Christine Craig thought that arguing on the grounds of traffic problems was difficult, given previous decisions the council had made.
"I seem to remember some members abstained at the debate over Tyndalls Pine," she said. Now traffic was coming out on to the Hindhead crossroads because of consent given to that previous application, she claimed.
Mrs Craig also agreed that Waverley had no need for more five bed homes. "The fear of what might come isn't a good reason for accepting this," she said.
"We should throw this out and get something more appropriate."
Councillors voted to refuse the application by nine votes to five.