If I buy the license for a second site then do I have to download again or can I upload the files from the first download?

Starting with a fresh download would be better. 

I downloaded the first license file yesterday... they are on my computer.

I mean maybe they have a respective serial number

If the 2 licenses include the exact same suite elements, go for it.

But if they differ (example: 1 has Pages, the other doesn't) then download the correct package for the license.

Exactly as mentioned by Nathan there. Please do ensure you download separately if you own different items on each too, otherwise it will cause you issues.

Thanks to all!