God NO!!!!
I've only just got him into them!!
He used to laze around either in just his dressing gown (rolling eyes emoji)
or his work clothes which can be quite dirty due to his job and I think its always nice to get out of day clothes and into something more sloopy when you are at home alone.
I personally wear some BAM jog bottoms (or jim jams after a bath) but I would never leave the house in them.