Byron Trott, the Chicago-based investment banker who advises billionaire investor Warren Buffett, is leaving Goldman Sachs Group Inc. after 27 years at the company, said two people familiar with the matter.

Trott, a 50-year-old vice chairman of investment banking, is leaving to start a merchant banking fund that will invest in and advise companies controlled by families or entrepreneurs.