You're thinking flying speed is just size and strength. Not so, say researchers from the University of Lund in Sweden. It's also a behavior issue and it's related to evolution.

Aerodynamic scaling rules that explain how flight varies according to weight and wing loading have been used to compare general speeds of a wide range of flyers, from the smallest insects to the largest aircraft but Thomas Alerstam, Mikael Rosen and colleagues analyzed the flight speeds of 138 bird species and overturn the general assumption that maximum flight speed of a species is solely determined by such rules.

Flight speed doesn’t just depend on mass and wing loading but also reflects the evolutionary lineage of the species in question.

Bird Flight Speeds (Ue; m/s) Plotted in Relation to Body Mass (kg) and Wing Loading (N/m2) for 138 Species of Six Main Monophyletic Groups The lines show the scaling relationships Ue ¼ 15.93(mass)0.13 and Ue ¼ 4.33(wing loading)0.31 as calculated by reduced major axis regression for all species (Table 1). All axes are in logarithmic scale. Inserts show means (6 standard deviations) for the six main phylogenetic groups in relation to these scaling lines. Species of the same group tend to fly at similar speeds, and phylogenetic group is an important factor to account for the variation in Ue.

The authors argue that only empirical measurements of flight speeds enable you to evaluate how general such aerodynamic rules really are. They used tracking radar measurements of the cruising speeds of migrating birds (collected by themselves and others) to do the analysis and provide the comprehensive dataset with the paper (e.g. this contains the flight speed of approximately one-third of all European bird species).

Their analysis reveals that the difference between the speed of small and large birds is not as great as expected; they suggest that this surprising result is likely to be the result of disadvantages associated with very slow speeds among smaller birds and with very fast speeds for larger birds. They also show that the evolutionary history of the species helps explain much of the variation in flight speed: species of the same group tend to fly at similar characteristic speeds. For example, birds of prey and herons had slow flight speeds, on average, given their mass and wing loading, whereas the average speed for songbirds and shorebirds was faster than would be predicted.

This study suggests that there are different functional adaptations affecting flight differently among different types of bird, and that there exists a diversity of cruising flight characteristics among birds that remain to be explored and understood.

Flight speeds among bird species: Allometric and phylogenetic effects,Alerstam T, Rose´n M, Ba¨ckman J, Ericson PGP, Hellgren O (2007) PLoS Biol 5(8): e197. doi:10.1371/journal.pbio.0050197.