I flagged an answer, but then realised I shouldn't have done (I flagged it as not an answer, but there was actually a valid answer buried in the text). There didn't seem to be any way to withdraw the flag, though. Is there any way to do this?
There is no mechanism to withdraw the flag.
When it comes up for moderation, the moderator can accept or reject it. On accepting it your flag weight is increased appropriately.