Why did Minkowski know where he was?

On watching this episode a second time, I wonder why Desmond in 2004
does not know anyone or where he is, because he is actually the 1996
Desmond, but when Minkowski is in 2004, he knows everyone on the ****p
and even who Desmond is and that Penny Widmore was calling the ****p.

I thought at first maybe Minkowski was not jumping back in time very
far--maybe just a few days--but then he said to Desmond when he saw
himself in the mirror: "you look a lot older, don't you?" So Minkowski
must have been ****fting back to a younger self as well, for him to
realize what Desmond was going through.

So why does Minkowski know where he is and what his happening when he
is in the present?
-yngver