I did something like this years ago when we were camping - for the same reason. I didn't make a robe, however, I made a knee length baggy T-shirt from towelling that just pulled on over my head.
For @Greybird’s suggestion of a baggy T-shirt, could you use a couple of large towels? Beach towels might be less “loopy” than bath towels and be patterned.