The Ice Age refers a period of millions of years in which the planet was continually covered by snow and blanketed by blizzards. It was caused by the Fall of Lavos, but its full extent, nature, or duration is not known. Most likely, the Ice Ages were aided by Lavos's introduction to the planet's crust, which could have altered the regularity of volcanic eruptions. Two scenarios may exist to describe the Ice Age's history; the first is almost a misconception in that it is implausible, but regardless, official word on the history is yet to be received.
Permanent Ice Age
If the Ice Age were permanent and truly did last 64988000 years, then the humans who hid in the caves would have undoubtedly adapted to survive in cold weather and thrive without sunlight by the end of this period. The rest of life on the planet would have done so as well, resulting in an animal kingdom adjusted to live comfortably in a cool climate. It is obvious that humans emerge millions of years later still warm-blooded and desirous of shunning cold, which wounds this scenario's plausability.
Cyclical Ice Ages
This scenario more closely resembles the history of Earth, and is probably what actually happened. The impact of Lavos caused instability in the world's climate at first, resulting in snow falling immediately after the fact. After this, Lavos's sucking the planet's energy, his position, and / or the initial disruption of the world's climate would cause a cycle of Ice Ages to continue in perpuity. Lavos's resurgence in 12000 B.C. to eliminate Zeal would apparently reset his position in the crust and have an effect on the outside climate, steering it back towards a more temperate mode (perhaps limiting volcanic eruption). Alternatively, his eruption in 12000 B.C. could have simply ended the current Ice Age, meaning everything after that year is simply a lull in the Ice Age cycle -- including 600 A.D., 1000 A.D., and so on. In this scenario, the humans who hid in the caves would return outside during the warm cycles, thwarting any serious evolutionary tendencies and allowing the construction of Zeal during such a warm period in the weather.
In any event, it is important to remember that Lavos may have been controlling the Ice Age as part of its speciation program. His initial impact and the onset of voluntary Ice Ages could easily eliminate weak and unwanted organisms.
In the illustration of the Polis War, it is shown that the Sea of Eden was covered in part by ice. This suggests that Chronopolis traveled in time as far back as 12000 B.C., though the fact may simply be that the warming climate caused by Lavos's eruption in that year simply took awhile to melt the ice in El Nido's location.
Thanks to GrayLensman