Albany watchers knew something was wrong with Assembly Speaker Sheldon Silver when his traditional buffet lunch on the day of the State of the State speech was canceled Wednesday.

“Everyone was wondering what was going on,” said one source, who was looking forward to the lavish spread. “There was no one in his [Silver’s] office to ask.’’

Acting as if nothing was amiss, Silver was next to Gov. Cuomo during the speech.