It should not be the issue. Page speed and load time depends on several factors. Server optimization is much more important.
Most of the themes and sites are using Google Web Fonts and this should not be impact.
I have checked one of our client blog using Arcane theme and load time is between 2.5 to 3 seconds.