After Constantinople fell to the armies of Sultan Mehmet the Conqueror in 1453, the castle was attacked by the sultan's armies, but withstood them, as it did again in 1480. But in the 1500s, strengthening with more stones from the Mausoleum was insufficient to guarantee it against the mighty forces of Sultan Süleyman the Magnificent. In 1522 the castle fell, the knights departed, and the sultan's standard flew on the English Tower.