The Spaniards were ready to expel everyone