The Colonial Americans hated the German and Irish waves just as strongly as I have seen modern America react to the Southeast Asians, the Latin Americans, and more recently (or overlapping, as it goes back to my childhood) the Near Easterners.

I think, in some ways, it has more to do with being afraid that all you built for yourself in the 'new' country cannot withstand too much pressure from 'newer' influences.