One would think if we citizens are picking up the tab for the city staff to run a survey and then City Council incorporates that data into their decision making process about how to spend our money, one would assume the survey was statistically correct. Or at least we could trust the staff to tell us it was not.
Disclaimer: Dr. Taylor may be a Mayor Jordan supporter, so I don’t mean to imply his loyalties one way or the other. Dr. Phil Taylor outlined in two different letters to the editor (Arkansas Democrat Gazette) the problems with the methodology or lack thereof with the city’s decision influencing devices.
In a letter dated September 16, 2019 Dr. Taylor writes about the plastic bag survey. “Fayetteville city administrators used an Internet-based voluntary response survey to gather most of the information provided to the City Council. An individual proficient in statistical sampling knows sample data gathered in that manner should not be relied upon to generalize about the population.”
He is referring to City Council decisions being made using data from survey results, from a survey that was answered only by volunteers. I wonder about the opinions of those citizens who did not volunteer to fill out the survey? Also, what about those citizens without Internet access who might have wanted to participate? What about those citizens who weren’t even aware that there was a survey?
Even more interesting was this statement by Dr. Taylor, “Moreover, in this case it was possible for individuals with an interest in the subject to participate in the sampling process an unlimited number of times.” In conclusion Dr. Taylor writes, “it is important to note that policies based on flawed data collection are not valid.”
In his letter dated March 10, 2020, again referring to the skewed plastic bag survey, Dr. Taylor notes “As examples: (1) An individual could respond an unlimited number of times; (2) 19% of the responses were from individuals living outside Fayetteville; (3) 14.7% of Fayetteville residents are between the ages of 30 and 39, while 24.7% of the respondents are in that age bracket; (4) 68.9% of respondents to the survey had education levels above a bachelor's degree or higher but for all Fayetteville residents, that number is 49.3%’ and (5) those providing answers to the survey had a median household income of $74, 335, but the actual number for Fayetteville households is $42,101.”
I questioned the validity of other city surveys, like ones about recreation areas, trails, parking, growth, etc. Were all those surveys skewed too? The survey asking about single stream recycling seemed very biased to me so I ran it by a professor, (not Dr. Taylor) who agreed that is was biased. Single stream recycling is the method whereby all different type recyclables are tossed together in a single compartment truck. It results in high contamination, reducing the value of the collected material, and exposing workers to filthy objects. Single Stream was the method used by Fort Smith when they got caught landfilling their contaminated recycling.
Hopefully the City Council of 2021 will be more attentive to surveys supervised by city staff, before spending our money on expensive consultants and pilot projects. I wonder how many hundreds of thousands of dollars could be saved with closer scrutiny of staff supervised surveys.