To delete, follow the same steps as you would to delete anything else. If it will not let you delete and immediately takes you to the other version, this is because whoever you have selected has two different kinds of relationships with that person. If you click on someone else who has only one or none, the duplicates will disappear.
