Spacetime with matter and gravity is not symmetrical. There is a particular singularity relative to which what we identify as our experience includes information (which we call memories) about events “closer” (not in spatial distance but in the spacetime metric) to the singularity (which we call the “past”), but not about those further away (which we call the “future”). Part of what is in our memory is experiences in which the scope of our memory was smaller. This gives us the feeling of becoming progressively further away from the singularity – ie of moving “forward” in time.