Is he rushing things, or am I just not good at dating?

I've been chatting online with a guy who's 30 for about four days now, and we definitely clicked during our phone conversations. However, there's one issue that's been bothering me: he's very eager to meet up in person. He's asked me every day since we started talking, wanting me to drive two hours to "hang out." I've told him it's too soon for me, but he continues to push the idea. Now he seems to think I'm not really interested in him. Is it normal for people to want to meet so quickly after connecting online? Am I the only one who feels this way?