Material and fabric are, I suppose, synonyms but wool and yarn are not. 'Material' or 'fabric' is just a small part of how the English language is continually changing.
Wool comes from the back of a sheep and can be spun into yarn, but then so can so many other fibres; cotton, acrylic, nylon, silk....... To label a ball of acrylic as 'wool' would be incorrect and would be against trade descriptions.