Hi all,
I work for a non-profit located in Everett, WA called the Adopt-A-Stream Foundation. Currently, we have grant funding from the Dept. of Ecology to talk to streamside landowners about how they can reduce their impact on water quality (and hopefully change their behavior). The watershed that we are currently funded in is polluted by high levels of fecal coliform bacteria so we have catered our questions and conversation with each landowner around this matter. For grant reporting purposes, however, we are required to submit information on how successful our interaction was at changing behavior. Since our funding is limited and we don¹t have time to make a follow up door to door visit, we plan on sending an evaluation via the mail (with the option of using Survey Monkey also). We are struggling with how to get the best response rate from our pre-made contacts and this was what we came up with. Does anyone have any other ideas?
Thanks.
-- Sarah Wilkins
How to Evaluate our Efforts
Sign in or Sign up to comment
Hi Sarah,
Sorry to hear you don't have funding for the follow-up research. That situation is far too common, and restricts the ability to determine best practices. It is frustrating.... Do I understand that you have not yet made the original visit yet? If that is true, it provides some opportunities to make the follow-up research easier and less costly. Let me know if that is the case and I will share some ideas. Keep up the great work!
Cheers,
Ken Donnelly
Ken Donnelly
Vice President, Eastern Canada
LURA Consulting | lura.ca [email protected]
Very interesting, I am glad to see that approach worked so well. We have a situation where I live in that NGO's will sometimes send out pledge forms with an incentive to return them. For example, "send this pledge to commit to change your incandescent to CFL's and you could win a bike". They then tell their funders their program was successful because they received so many commitments. But, in my opinion, they did not receive commitments but ballots for a free bike. For this reason I am very wary of improperly used incentives in CBSM campaigns. However, in this case, there is no incentive issue because the gift was given beforehand. Brilliant!
Cheers,
Ken Donnelly
Could a reduction in bacteria be your actual measure of change? Do you do any water testing, or is that done by someone else? It might be hard to do after the fact, I suppose, but it would make a really strong assessment if you could.
Mary Whitney
University Sustainability Coordinator
412-365-1686
[email protected]
Chatham University Coolidge
120 Woodland Road Pittsburgh, PA 15232
Sarah:
Another idea, along the lines of what Marjorie in NY shared, is to insert a $1 bill with the survey, with the pre-stamped & addressed envelope. Send reminder postcards to those who don't return their survey within 10 days, and another round of postcards again at 20 days. Clearly stating in the cover letter what the survey results will be used for, that all responses are confidential, and that mailing information will be kept secure and not shared with other agencies can help increase the likelihood of having the surveys returned. Lastly, identify a person in the cover letter whom respondents can contact if they have questions, and of course, list a phone number. You didn't mention if any of the landlowners speak or read in another language. Consider having the survey translated into appropriate languages to increase your response rate. Using all of these strategies with a survey we conducted last year with residents in public housing garnered an unheard of 86% return rate. The issue was on smoking inside and whether the manager should ban it so respondents had strong feelings, both pro and con.
Good luck!
Theresa Cross, MS RD CD
Tobacco Prevention and Education
Clark County Public Health
360.397.8000 ext. 7378
1601 E. 4th Plain Blvd Vancouver WA 98666
Mailing address: PO Box 9825
Vancouver WA 98666-8825
Hi Sarah,
It's always a good idea to use an incentive (such as a prize draw) to increase your response rate, and something that would be useful to encourage environmental behaviours would be a win-win situation. With this though, you don't want to make the prize too niche market, so that only a particular type of person responds (and hence you have a bias response/result). For example with a travel survey at a workplace, in past programs I have worked on, we offered a new bike, but found that the people you most want to influence and determine their travel patterns (car drivers) were not always the ones motivated to reply to the survey to win a bike. We then offered prizes like wine or movie tickets and received a relatively good response rate (40% on average). The other idea is to include a brief letter explaining the benefits of them being involved - that it would help to inform your strategy and would mean a more effective program for controlling pollution levels. If you can address envelopes with individuals names, instead of with a generic salutation, this would probably increase response rates. In the book 'YES, 50 secrets from the science of persuasion' it talked about surveys that were sent out to residents, and if the cover letter was signed by someone with a similar sounding name to themselves, the response rate rose by 20% - or something similar. i.e. If the respondee was called Natalie Wood, the researcher signed it 'Nathan Woodley'.. Not that I would suggest going through and making up names for each of the people you intend to send surveys too, just an interesting little bit of research, that we are more likely to positively respond to those we feel some connection with (whether they are from the same town, have a similar name etc).
Hope this helps.
Karin
Hi Sarah,
If you're not required to evaluate each and every interaction, you may be able to 'sample' a representative portion of the group you interacted with, and if so, this can cut your costs significantly. You would need to ensure though that your sample is representative and large enough to be statistically significant. Perhaps Ecology can help you with that, or a local Stats dept at a University may be of help.
Norm Ruttan
iWasteNot Systems
www.iwastenotsystems.com
A successful method I've seen is telling respondents that for every survey returned, fifty cents or a dollar will be donated to a charity - and as part of the survey, you list three different, but preferably local, charities (eg, a children's hospital, a wildlife sanctuary and a soup kitchen) and ask them which of them they want "their" money to go to. I can't remember the response rate but it was very high.
Katherine Lustig
Environmental Management
System Coordinator
Strategic Asset Management
Parramatta City Council
PO Box 32 Parramatta NSW 2124
Hello, Sarah -
I would highly recommend contacting the Surface Water Management department for Snohomish County (conveniently located in Everett as well!). I got in touch with them when I was beginning work on a campaign very similar to their very successful pet waste pickup program, and got some very helpful and insightful information on evaluative methods. They've already gone through a lot of trial and error with testing which methods seem to produce the most authentic results, Another program I came across when I was researching evaluative methods was that of the Anchorage Waterways Council, which offered the incentive of having utilities bills paid for three random winners who participated in the survey. So even if your budget doesn't allow for individual gifts, perhaps you could use an incentive to encourage people. Something like a raffle prize or a donation from a local organization or business. I'm really interested to hear what other tips people have to offer, since I'll be constructing our own evaluation techniques soon.
-Elena
Elena Cronin
Clean Water Community Outreach Coordinator
Clark County Public Works Vancouver, WA
Hi Sarah,
Have you thought of doing a phone survey? It isn't an ideal technique if you're strapped for time and human resources and need to do the survey with a lot of people, but it certainly seems to be really effective for getting your response rate up and is completely feasible if the sample size isn't too large. I am working for the Canadian Wildlife Service, and we are doing a survey of landowners who have been involved with various private land stewardship programs with NGOs in B.C. I have been conducting the surveys myself for one of the NGOs we work with, and I have found that people are very cooperative over the phone. I also did up an online survey with KwikSurvey as well, so that people have the option of completing that if they are really averse to the phone survey. So far, of 20 contact names, I have only had one person refuse the survey altogether. I have completed the survey with 13 people on the phone, 3 people chose the online survey method, and 3 people are so far unreachable. Our survey is 15 minutes long, if all questions are answered as they are asked (mostly on a Likert rating scale, with some yes/no questions and a few open-ended questions). A few chatty folks have taken up to an hour to do the survey. All in all, people have been very positive about doing the survey and many have even thanked me for calling them! If you've already designed the mail survey, you probably will not need to alter it very much to make it phone-ready. Let me know if you are interested in doing something like this, and I'll send you some more information,
Yours,
Vanessa Kilburn
Habitat Stewardship
Program Evaluation Analyst
Pacific and Yukon Region
Canadian Wildlife Service - Environment Canada
R.R. #1, 5421 Robertson Road,
Delta, B.C. V4K 3N2
Phone: 604-940-4724
Fax: 604-946-7022
Email: [email protected]
Hi Sarah
Have you looked at trying an intercept survey? They can be completed in a weekend and if you get volunteers to conduct the survey it is relatively inexpensive and painless. You only need a sample size of around 200. Maine has one that the stormwater communities are using and a few 319 funded projects might try. It was developed by a market research firm. You can find Maine's survey at http://www.maine.gov/dep/blwq/doceducation/nps/outreach.htm
Kathy Hoppe
MDEP 207-760-3134
Sarah,
We had an excellent return on surveys when we preceded the mailed survey with a mailed postcard from our Mayor. The postcard told residents to expect the survey in the mail within a week, and told residents why it was important for them to take a few minutes to respond. The mailed survey then had a cover letter from the Mayor, and had an option to respond through an online survey or by completing the paper survey. A postage paid envelope was provided.
Wendy
Wendy Smith
Environmental Education Coordinator
Town of Chapel Hill
Stormwater Management Program
Dear Ms. Wilkins -
I understand about your funding limitations, but my suggestion is to send a gift with your survey. I once sent a survey to an entire (small) county of around 25,000. I had a great response. It was several years ago so I've forgotten the percentage, but it was way above average. I sent a small magnet with my organization name, logo, contact information and slogan. The questions were about a newly-created curbside recycling program, a very hot topic at the time - as I would imagine yours is as well. Between the current interest in the topic and the magnets, I got a great response. Oh, and also a self-addressed, stamped envelope.
Good luck!
Marjorie Torelli
Western Finger Lakes
Solid Waste Management Authority
Lyons, NY 315.946.7650