Well as someone already said I think it’s when he hits his head when robbing the station.
I’m thinking in the trolley station robbery mission, in Saint Denis. During the mission as the trolley crashes, Dutch is hit in the head. I wonder if he had gotten an injury from it, and as more and more members of the gang were lost, lost more of his sanity.
I think there is a chance Dutch sustained a brain injury in Blackwater which made him vulnerable to micah's lies
Perhaps Hosea getting shot in front of the bank, as well as Arthur being diagnosed with tuberculosis. Having lost his 2 pretty best friends, as well as the stress of having the O'Driscolls and Pinkertons right on their tails, caused him to go mad?