This paper draws on studies of the Capgras delusion in order to illuminate the phenomenological role of affect in interpersonal recognition. People with this delusion maintain that familiars, such as spouses, have been replaced by impostors. It is generally agreed that the delusion involves an anomalous experience, arising due to loss of affect. However, quite what this experience consists of remains unclear. I argue that recent accounts of the Capgras delusion incorporate an impoverished conception of experience, which fails to accommodate the role played by ‘affective relatedness’ in constituting (a) a sense of who a particular person is and (b) a sense of others as people rather than impersonal objects. I draw on the phenomenological concept of horizon to offer an interpretation of the Capgras experience that shows how the content ‘this entity is not my spouse but an impostor’ can be part of the experience, rather than something that is inferred from a strange experience.