The owner of a historic, Chelsea building that is the sole vestige of the Underground Railroad in New York has lost his bid to keep a fifth-floor addition he built in 2005.

Tony Mamounas, owner of the Hopper-Gibbons House at 339 W. 29th St., had first sued back in 2013 after the city ordered the new penthouse removed.

The new floor obstructs the path that abolitionists Abigail and James Hopper Gibbons used in running along the rooftops as they fled the Civil War Draft Riots.

A unanimous Appellate Division decision released Tuesday found Mamounas completed the construction after a permit was revoked and that he should have appealed to a city agency before suing.

Spokesman Nick Paolucci said the city’s Law Department is “pleased” with the ruling.

Mamounas’ attorney did not immediately return a request for comment.