It looks frumpy because of a combination of fabric choice (ditsy print is nice, background colour also but not together in a large area), the sleeves, which are too long, and the length. The skirt, as you say, is baggy because the fabric doesn't drape. I'd shorten the sleeves, cut it to mid-thigh and wear over leggings. I think you'd suit a more fitted style in a solid colour, possibly quite a vibrant one (without seeing hair colour/ skin colour it's difficult to say what) just below or just above the knee, with either much shorter or much longer sleeves, also fairly fitted. Have you tried a princess seamed pattern
You did ask - sorry if I've offended.