"But what I remembered most is moving a lot" is correct, with or without "the". Although "the most" is the superlative, preferable. Here, "most" is used as an adverb modifying the verb "remember", meaning "to the greatest extent". There may be other examples, where it can mean "extremely" as in the following:"it was most kind of you", "that is ... More @Wikipedia
Hover over any link to get a description of the article. Please note that search keywords are sometimes hidden within the full article and don't appear in the description or title.