I think most people of my age would probably say Tottenham but for me it's West Ham. First time I went to a game on my own got the wrong tube train from Earls Court. Noticed my error at the first station, waiting for the train back a tube full of ICF pulled in - on their way to ambush Chelsea fans - a big skinhead, probably about 17, got off and punched me to the floor. The tube pulled away and they were all laughing at me as the blood poured from my nose, I was 13 wearing a Chelsea top, scarf and bobble hat, rookie mistake, lol. They were our main rivals in the hooligan stakes especially in the old second division and the only fans who would try and take the Shed. Also at gigs there was often fights between us and them.
Every Chelsea fan I know says Spurs, then West Ham. They don't really care about the other London clubs.