Short answer = Yes
Temperature does correlate with number of visitors, but not in a way you would expect.
There is a small weak correlation, in terms of the lower the country's average temperature is the more visitors they receive.
Here is the evidence
Here you can see the countries are ordered in highest to lowest average temperate (in Fahrenheit) and you can see the amount of visitors per country by the columns.
As you can see there is no strong correlation or order, however you can see the faded pink link along the bottom, this represents the trend line of the visitors and as your can see it is gently sloping up as the average temperature goes down.
So there you have it, the lower the country's average temperature the more likely they will attract tourists.
Disclaimer: Correlation does not mean causation, every good statistician knows this.
Here is an interactive map below
If you're wondering about the break down of visitors by country, here is a pie chart.
China receives exactly 5% of all visitors in the world.
Anyway, that is your fun fact of the day, enjoy