I found this recently on the Library of Congress – https://www.loc.gov/collections/united-states-telephone-directory-collection/index/location/; it’s not all states or all counties within a state. Years cover vary from place to place.
About this Collection
The Library of Congress’s collection of telephone directories represents the following states and localities: Alabama, Alaska, Arizona, Arkansas, California, Colorado, Connecticut, Delaware, the District of Columbia, Florida, Georgia, Hawaii, Iowa, Maryland, Pennsylvania, and the city of Chicago. The dates of the directories span most of the 20th century. The Library’s United States telephone directory collection consists of 8,327 digitized reels of microfilm; of these, about 3,500 are presented in this collection. The remainder of the collection may be requested from the Microform Reader Services (LJ 139).