If I remember correctly, you're always allowed to start the first insult. Pick an insult you don't know the answer to. Hopefully your opponent knows the correct response. If he does, you learn a new response to an insult. If he doesn't, you win that round. That means you will advance a bit, and the enemy retreats a bit. If you win enough rounds (insult - retort), you win the fight. Win enough fights and you get to play against the sword-master, who knows all the insults and reponses.

That's how I remember it too. But it doesn't seem to be learning any new ones... Let me try again.

If I remember right, it's a very time-consuming, grind-y process, pretty out-of-character with the rest of the game. Just gotta be persistent through it. The feeling of success from finally beating that puzzle is great, though, due to the time investment.

You learn insults and retorts by fighting other pirates.

In the beginning you don't know how to sword-fight. You will lose. Next time you have a few new insults that you will use and they will know the answer. Next time you will have the answer for that insult.