Yes, wasps will return to a removed nest, because wasps produce pheromones, that help them protect their nests and know the way of the nest, but this happens if the damage is light and minor, and this happens in rare cases, because they prefer to build new nests