I can't see any reason why there should be a problem if it's like my softshell (smooth on the outside, fleecy on the inside). Instinctively I would say it needs a sharp needle, not a ball point one - mine doesn't really have any stretch, but then it doesn't contain elastane either - but I'm really not an expert.
Actually, I'm not convinced it's the same fabric, so it's probably better just to ignore me.