East Hampton Village’s quiet library is providing the hamlet’s loudest controversy.

In a decision that some are ripping as elitist and exclusionary, the village’s Zoning Board last week rejected a bid to expand the tiny Main Street building by 6,000 square feet.

Livid library officials say the board wants to avoid having children from less affluent families pour into the new facility.

“The old guard is scared that the Latino kids and their parents from [nearby town] Springs are going to invade their precious downtown area,” said one seething library source. “It’s really that simple.”

The library board has organized a public referendum for Aug. 13.

“I think we are going to get a lot of support,” said library Director Dennis Fabiszak. “People are pissed off.”