Is this a good way to insulate drafty windows?
...seal around the rough opening.
Another thing you are probably experiencing is thermal bridging in the floor as concrete would extend from under your feet to the exterior of the balcony, and that transports a lot of heat. Some insulated matts in front of the sliding door may offer some help too.