Ehh really depends on your boss. And honestly it depends on you also. If either of you can’t separate work from personal time then you really shouldn’t be hanging out. But for most of the bosses I’ve had, some lighthearted social time outside of work was perfectly fine.
No, I think they’re being literal. There is value that they want in your privacy.