Considering that Stark didn't know exactly when Loki came from, That might or might not be true. But Stark believed it, and that was the only relevant point. "I'm entertained by the thought I may be from the earliest point in time, here. Being entirely without context is my favorite thing." It was not. It really was not.