The 9/11 Tribute Center can continue construction “at their own peril,” a Manhattan judge ruled Monday, warning that any work on an entrance close to a tony residential building may be torn down if it is later found to inconvenience the condos’ wealthy inhabitants.

Judge Lucy Billings declined to halt construction on the memorial’s new digs at 92 Greenwich St. as lawyers for the luxury condo wanted.

Lawyers for the Greenwich Club Residences, a 37-story building where units sell for as much as $4 million, expressed anxiety that the mobs of tourists visiting the museum — and the merchandise and pesky vendors that will follow — could clog the sidewalk.

“We think this is ludicrous, and are happy to continue at our own peril,” said Joseph Matalon, attorney for Thor Equities, the owner of the commercial property.