Tories in Tandridge were running the council with a majority of just one, after losing eight seats last year.
The local party lost a further six seats this year, losing three to the Oxted & Limpsfield Residents Group, two to the Liberal Democrats and one to an independent.
A total of 14 seats were up for grabs on the night.
The Conservatives remain the largest party, with 16 seats out of 42.
The Lib Dems have 11, independents have eight, and the residents association has seven.
Leader Martin Fisher (Con) lost his seat to David Stamp of the residents group, who won a 68% share of the vote.
Mr Fisher said: “It’s certainly going to be a very bad night for the Conservatives.”
He said “Brexit chaos at Westminster” had not helped the party, but locally, the key issue was building homes on the green belt.
He said preventing this was what the independents were focused on and “that cost us seats”.
Wealthy East Surrey is traditionally Tory territory but the Conservatives have seen their control there steadily ebb away over several years.
The beneficiaries have largely been Independents and Residents Groups.
Brexit is not to blame, but rather a failure to balance house-building with concerns over the greenbelt.
That is solid Tory policy territory in the South East and could suggest there may be more for the party to worry about than their immediate Brexit concerns.