Do you have a diagram showing effect of winds on the pressure inside the house?
...effect of warm air rising will create a convection in the house as well as create a positive pressure at the top portion of your building enclosure. How much the wind effects your comfort and efficiency will depend on how airtight your house is. Best regards.