If the fence is made up of metal, it would be very hard for them to go over it because its surface is slippery and smooth enough to climb. Likewise, raccoons cannot climb unweathered metal sheet and glass. Baby raccoons can also climb fences but not more than 5 inches high. But a fully grown raccoon can scramble up fence of more than 8 inches. More @Wikipedia
Hover over any link to get a description of the article. Please note that search keywords are sometimes hidden within the full article and don't appear in the description or title.