The Baronial Palace was built on the foundations of the medieval 12th century castle, destroyed by the armies of Pope Boniface VIII at the end of 13th century.
The palace, built in late 15th- early 16th century, dominates the town. The rusticated entrance, beneath a clock tower, is particularly interesting, while the clock has marked time for the town for centuries.
The entrance opens into a courtyard from where it is possible to admire the posterior, Renaissance, façade of the palace with a loggia and 16th century portico.
In the late 1500s, the palace became a prison, holding condemned prisoners and those awaiting sentencing.
In 1815, the main salon of the palace was donated to the Municipality of Colonna by Prince Luigi Pallavicini and is now used as the headquarters for the town council.