On Fame/Infamy

So I have a really bad time with people coming up to me and saying hey! And talking about how we met the other night at X persons house or they tell me they've met me multiple times and I have no recollection of who they are. Now I go out and party, but I don't feel like I get drunk enough where I wouldn't even remember someone. I'm bad with names but sometime people come up to me and I feel like I've never seen them before. A part of me wonders if it's me or if my doppleganger have ruined something else for me. Is it people who get too drunk and then meet one of my dopplegangers and then mistake me when sober for one of my dopplegangers? Possibly? Who knows.