Ousted Men’s Wearhouse Chief Executive George Zimmer has quit the company’s board.

He submitted a letter yesterday saying the board is avoiding addressing his concerns about board decisions and the company’s direction.

Zimmer, 64, built Men’s Wearhouse from one small Texas store to a clothing-selling giant with 1,143 locations. Shares in the company fell 66 cents, to $35.13, yesterday.