Do you think he's happy to JUST be friends with me ....or?

"scared" is unnecessarily critical. It's just as likely he's "cautious" or "considerate" or "gentlemanly". You've also given a lot of evidence he's at least somewhat shy.