Can this relationship work?

Stay with him. Keep him close to you, make sure your kids like him, and most importantly, don't scare him away. By that, I mean don't force the prospect of marriage upon him. Tell him that it's ok if he wants to wait a few years. Those few years will really pay off for you two.