It must be "I didn't go". "Did" is already in the past tense. "Go" does not have to be in the past tense if "did" was already in the past tense. You could say I went. But you couldn't say I did not went. You could say I went not. More @Wikipedia
Hover over any link to get a description of the article. Please note that search keywords are sometimes hidden within the full article and don't appear in the description or title.