We are all grown ups. Knowing that there is no Right or Wrong in every decision you make as, it is YOUR decision. You are the only person knowing whether its a correct thing to do.
Put yourself in the poor fiance's shoes, what will she feel if she found out this.. Fling that involves you? It couldn't possibly be grateful or joy. Undoubtedly, he's an asshole in disguise of prince charming meat suit.
For a periodical moment of 'fling', or 'romance' or as you call it - love at first sight, don't get me wrong. I truly believe in love at first sight - but your feelings in this situation is not untimely, its pretty selfish.
Why would you want to get involved in this sort of connection? It bothers me because you were okay-ing for it to happen. It was a fling, but you enabled his irrational untimely encounter into a love at first sight. He was irrational, but you enabled him, giving him signs that you are actually indulging in this 'fling' too.
I'm not sure what is your ultimate intention / purposes in this whole scenario, it's your decision to make whether to hit the delete button, or making him yours, or keep it in the vague state. I have no authority to tell you which road to take. Just remember, he's an ass. Married ass.
At least, it is unacceptable for me that you have enabled him for the second and third time. As a friend, i can only advise you on which road not to take; As a human being, this is something I despises in a person. I couldn't possibly say it out loud to your face as writing it out offends me. First mistake, fine.. but second third and forth?
I would like to not to discuss this further. If you feel like sharing, I'll just listen. Because, I do not know where will this lead to, and I treasure our friendship - and this genre of discussion puts me in the situation where, it might jeopardize our friendship.
I'm thankful that you felt compelled to tell me the details, and I have done what I can do to help. No matter how many hours you talked to me on the phone telling me that you are miserable, reassuring that there was nothing going on etc, it all laid waste when you replied to his invitation. Again, he's an ass, married.
I will still be here to listen, but - the comments will be the same as the first advise I gave.
If any of the words above have offended you somewhere, Im sorry. But, Im writing this because I treasure our friendship, and I wouldn't want as ass who gave you a moment of weak knee to ruin it.
Period.