They aren't really trying to wipe them out as much as to take the founding titan power for the evil empire marley, the walls are on a island near a mainland and used to be a large empire called eldia who used to inhabit the mainland, who were over.
