All we know is that there's a feeling that comes up inside of you around certain circumstances here. It could be a million things, it could be some old childhood association, it could be mom, it could be mostly being open and connected to that energy for a moment, may have nothing to do with sex or vibe or trying to be cool.
Drop everything you've learned about 'attachment to women' and 'trying to be content on my own but I can't get to the bottom of the feeling sitting alone' because they are all assumptions that may or may not have anything to do with what is actually going on.
Keep inner work direct and simple.
Men of the world: stop treating stuff that comes up for you around the subject of women as being unique and special from any other stuff that gets triggered inside of you. If you were hit with these feelings after visiting a certain location or watching a movie would you assume all these things about what's going on? Treat the feeling as any feeling and work with it accordingly. Literally everything else, including your own mind, can at best make a logical guess about what it might be- or you can just process it directly, by whatever means works for you, letting go etc.
I've been getting better at letting the intense feelings in the moment pass through, but it is like a vicious current that eventually drags me off.
Good. It's good that you can let the triggering happen and subside but it would be appropriate to move into the source now, as I'm sure you don't want to just keep having this happen but be more able to let it subside. Also note that a feeling is incapable of dragging off that which is aware of a feeling. It does not have that kind of power but it does capture your attention in a way that a habitual response follows.