Scarlet Letter- Why is Dimmesdale unwilling to take responsibility for his crime?

In the novel The Scarlet Letter by Nathaniel Hawthorne Reverend Arthur Dimmesdale commits adultery with Hester Prynne. The townspeople find out about Hester, but she doesn’t tell anyone that Dimmesdale was involved. Since the townspeople knew Hester committed adultery she had to wear a scarlet “A” on her chest and went through public humiliation. The townspeople didn’t even suspect Dimmesdale of committing any kind of sin. To the townspeople Reverend Dimmesdale is the holiest and most perfect person in the town. I believe that Dimmesdale was unwilling to take responsibility for his crime because he was looked at as the perfect person so if anybody found out it would ruin his reputation...