With examples from Saiyuki and Weiss Kreuz.
Past tense is the tense most commonly seen in fiction, but you may have noticed that writing about the past sometimes presents its own unique challenges. There may have been times when you found yourself struggling. There must have been a time when you wondered if you were using grammar correctly in your story. Worst of all, you may well have noticed that even when a story is gramatically correct, sometimes it doesn't read correctly at all.
Fortunately, the use of 'must' and 'may' is generally not so thorny. "Must," "may," and "might" are all modal verbs, which have some special usage rules.
Must in the past tense is generally limited to logical deductions:
Crawford was nowhere to be seen. He must have gone down the back staircase.
There's a certain urgency to 'must.' Think of it as indicating that there is no other choice.
The other common use of 'must'-- like "I must go, now!" doesn't really apply in the past tense. If you or your character are thinking about something that was required in the past, 'must' is no longer necessary. The urgency of 'must' has passed. "I had to" or "I was required to" will do, but "must" no longer applies.
May, in contrast, is far more easygoing.
Hazel considered. He may have been wrong about Ukoku-han after all.
May is conditional, and refers to something that, well, may have happened. 'Might have' operates in much the same way.
He might have been drunk, Nagi reflected. That would explain the karaoke, anyway.
So they're perfectly acceptable-- as long as you're using them with a little care.