The castellated tower of Oyace is probably the oldest castle in Aosta Valley, legendarily built by a group of Saracens who took refuge in Valpelline around 1000 AD; it is already mentioned in a document from 1197 which speaks of an "allodio ad Ayacy", donated to a certain Ricalmo to the church of Sant'Orso.

The fortified complex was enfeoffed to the Lords of Oyace, of whom little information is available. Between 1233 and 1253, Amadeus IV of Savoy decreed that the castle be destroyed to punish the Lords of Oyace, guilty of unfair behaviour towards the House of Savoy. All that remained of the fortified house was the tower. Between 1253 and 1287, Oyace passed into the hands of the powerful Lords of Quart. In the second half of the 14th century, the lords ceded direct control to the Savoys. In 1612 it changed hands again, going to the Perrone di San Martino. It is the only example of an octagonal tower in the Aosta Valley. It is 11.7 metres high, with an external diameter of 7.5 metres and an internal diameter of 3.5 metres: the thickness of the load-bearing walls therefore reaches almost 2 metres. The access door, as was customary in the Aosta Valley in the Middle Ages, is raised by several metres to allow the occupants to better defend their position. A removable ladder, probably made of wood, was used to access the tower.

Only a few traces remain of the walls that surrounded the tower: the part that was easiest to reach had a double defensive curtain, while the raised entrance was more protected because it faced the rocky spur.

During the plague of 1630, Oyace prevented contagion by installing a gate in the village of le Bouyoz to prevent access by foreigners who might have carried the disease. The remains of a stone cross that flanked the gate can still be seen.