Analysis & Summary of Evaluations
Analysis & Summary of Session Evaluation Results
Evaluation forms were collected for each of the 56 break-out sessions. Although not everyone chose to complete evaluation forms for the sessions they attended, the forms that were completed provided us with some very interesting and important information. Our Reports & Evaluation Committee, ably chaired by Eric Boyd, former Executive Director of Los Angeles Days of Dialogue, compiled and analyzed this data and wrote the following narrative.
It was clear from the results of the quantitative section of the survey that Conference participants overwhelmingly favored workshops that exposed them to new and innovative methods of D&D practice over those workshops that were topical (i.e. political, race relations, etc.) or even international. This was very telling, especially with international issues so prevalent in light of impending war with Iraq.
Tom Atlee of the Co-Intelligence Institute presenting a session about Citizen Deliberative Councils.
As such, the highest session ratings were consistently garnered by the newer, more ?cutting edge? workshop sessions. Top scorers were Liz Lerman's session entitled ?Dance Exchange on Dialogue and Making Dance,? Duke Duchscherer's ?Nonviolent Communication: Practical Tools?,? The Animating Democracy Initiative's ?Arts & Civic Dialogue Exchange,? Irene Papoulis and Beverly Wall's ?Using Informal Writing to Foster Democratic Dialogue,? and Active Voice's ?Using Film to Build Dialogue.?
Clearly, the hunger for new applications and exposure to new models should be a primary focus for future conferences. However, since some more topical or traditional workshops (i.e. those focusing on Bohm dialogue, the achievement gap and white privilege) garnered the highest numbers in participant attendance, an equitable balance between traditional topics and new models/applications should be pursued.
It was also apparent that there is a hunger for more interaction, evidenced by the fact that session presenters received high marks for their delivery, but relatively low marks for participant contribution opportunities. Time allotment will also require a great deal of thought in future conferences, as it registered the lowest ratings of all measured items. Still, participant objectives were largely met, materials and presenters scored high, and overall session ratings were highly positive.
Next Steps for Field
A snapshot from the plenary session organized and facilitated by AmericaSpeaks.
For the most part, participants felt that if in fact the dialogue & deliberation community views itself as a ?field? or ?movement,? then that body should work assiduously to ?show its wares? and/or ?showcase its stars.?
Relative to the individuals who comprise our ?field,? a significant number of members view the question of neutrality as one of vital importance to the field. In addition to the conference Organizing Committee's recognition of this emerging issue ? as evidenced by verbally addressing the issue at the final plenary session ? a number of individuals attending workshop sessions also indicated concern in this regard. One participant who attended the ?Creating Space/Restorative Justice? session stated the following: ?Sentiments for Offenders were troubling (almost pro offender). Need more work on what it means to 'create safe space' vs. advocacy.?
Whatever the next steps are for the field, feedback from participant evaluations make it very evident that practitioners believe that addressing ?The Neutrality Question? should be included in our list of priorities.
Compilation of Evaluation Forms
Respondents were asked to rate how strongly they agree with 11 statements about the session they just attended. 9 of the features were rated, on average, between agree (4) and strongly agree (5). Only two statements rated between neutral (3) and agree (4) ? those having to do with ample time and ample participation. Here are the statements and their averaged ratings, in order of rating.
- The session leader was effective in his/her delivery, and held my interest. (4.66)
- Session leader used language understandable to all sectors of practice, or defined jargon. (4.40)
- Handouts (if any) were useful. (4.29)
- Overall session was satisfactory. (4.25)
- Workshop objectives (if stated) were met. (4.20)
- The session will help in my dialogue/deliberation work (4.16)
- The session leader answered my questions appropriately, and augmented my knowledge. (4.14)
- Audio/visual aids used during the session were helpful. (4.12)
- The workshop met my expectations. (4.07)
- Session participants had ample opportunity to contribute to the discussion. (3.67)
- Time allotted for session was adequate. (3.28)
We also asked a series of questions to help us understand more about participants' interests and needs, and about session outcomes. The first question was multiple choice; the rest were open-ended. Here are the most common responses we received.
Please tell us why you chose to attend this particular workshop session.
- The topic (40%)
- To learn about a model that is new to them (30%)
- The session leader (20%)
- Content relevant to a current or planned initiative (5%)
- Content relevant to current community need (3%)
- Other (2%)
What was the best thing about this session? Top three responses, in order of popularity:
- The presenter(s)
- Having the opportunity to use the model/technique
- The materials and handouts
How can future sessions be improved? Top three responses, in order of popularity:
- More time
- More practice; less lecture
- More in-depth workshop on various topics
What next steps does this session prompt you to pursue (if any)?
- Incorporate model/learned material into our own program; combine techniques
- Seek collaborations with presenter or similar organizations/individuals
- Study the model/workshop content further (learn more online, attend an event using the model, speak and/or correspond with presenter, etc.).
- Explore other related fields (i.e. mediation, counseling, peace negotiations, etc.).
What next steps does the session convince you the dialogue/deliberation field should pursue (if any)?
- The D&D movement should promote this model nationally and/or advocate for its use (various models).
- We should use the model at future conferences.
- The field should focus on/emphasize neutrality more.
Summary of First Day Survey Results
97 completed surveys out of 241 participants (40% response rate)
Motivation and Expectations
Participants indicated that they attended the conference to build their skills (38%), to learn about dialogue methods and tools (38%), to help strengthen and unite our field (26%), and to meet their colleagues in the field (25%).
Other reasons mentioned by 8% or fewer were to improve their job prospects, to discern similarities and differences between models, to make culture more essential to understanding dialogue, to improve personal relationships, to stay in touch with others throughout the field, and to share resources.
When asked what their primary expectations were for the event, participants stated their hopes to contribute to the conference from their own experience (20%), to network with their colleagues (19%), to obtain and share resources (14%), and to learn about new methods and models (13%).
Other expectations mentioned by 8% or fewer of the participants (in order of frequency mentioned) were: to advance knowledge and understanding in the field, to reenergize my commitment to this work, to participate in dialogues, to learn about evaluation techniques and tools, to broaden my studies and experience, to get new ideas, to initiate collaborative projects and relationships with others in the field, to examine and rethink my own personal viewpoints, to learn about ?what's next? in the field, to hone my skills, to learn about the U.S. dialogue and deliberation community, to share concerns about the field, to discuss a stronger democracy, to explore new ways to engage, and to have fun!
Participants were asked whether there were any topics or activities not listed on the conference schedule that they would have liked to see. Only 16 comments were offered, half of which suggested that more youth-oriented sessions should be offered. Two comments stated that more K-12 and college-based sessions should be offered.
Other comments stated that more sessions should be offered that focused on dance and theatre, the use of dialogue in business, non-verbal dialogue techniques, additional open forums, dialogue and deliberation as tools for social justice, and identifying and recruiting elected officials for participation in dialogue and deliberation programs.
A space was provided for additional comments. Comments ranged from questions such as ?How does the D&D field see itself in relation to efforts such as the critical thinking movement in education that emphasize dialogue between teachers & students?? to suggestions such as ?Ensure that facilitators model good practices.? And from compliments such as ?Well planned conference! The handbook is awesome!? to critiques such as ?Some of the workshops need longer time slots.?
Additional comments were:
- Have a better balance between teaching and doing.
- We should be less process oriented, and more focused on value/sustainability.
- Assumptions about a field are an asset and a problem.
- Get the big names here (Yankelovic, etc.).
- Need more outcomes and convening workshops.
- Have places outside the conference to ?clear our heads.?
- Vary food for vegetarians.
- Gauge pros & cons for next conference.
Experience/Professional Identification Data
Respondents acknowledged having direct experience with the following models of dialogue and deliberation:
- Study Circles (27%)
- World Caf? (14%)
- National Issues Forum (13%)
- Public Conversations Project (11%)
- Bohm Dialogue (9%)
- Appreciative Inquiry (8%)
- Future Search (6%)
- Intergroup Dialogue (5%)
- Open Space (5%)
- NonViolent Communication (4%)
Additional models mentioned by three or fewer respondents were: Black/Jewish Dialogue, Jewish/Palestinian Living Room Dialogue, Bill Isaacs' Model, Story Circles, Wisdom Circles, Critical Response, Deep Listening, the International Communication Association model, Consensus Building, Mobius model, National Coalition for Community & Justice, Public Dialogue Consortium model, Restorative Justice model, Search for Common Ground, Track II Diplomacy, and Transformative Mediation.
When asked what roles they played with those models, 82% said that they had facilitated those models, 68% had participated in programs utilizing the models, 49% had organized events utilizing the models, and 49% had trained facilitators for the models. 32% had written discussion guides or other materials for the models, 26% had served as MC or moderator of programs utilizing the models, 26% had observed the models, and 25% had served as resource persons or panelists regarding the models.
Respondents had addressed the following issues through dialogue and deliberation:
- Civic engagement/participation (48%)
- Race relations (47%)
- Diversity/immigration (42%)
- Interreligious issues (29%)
- Youth issues (29%)
- Local public policy (27%)
- Education reform/achievement gap (26%)
- International issues (26%)
- Violence (23%)
- National public policy (20%)
- Police/community relations (14%)
- Abortion (6%)
- Sexual orientation (6%)
- Environmental issues (5%)
- Family conflicts (4%)
Additional topics mentioned 3 times or fewer were: Business/Economic Development, Affirmative Action, Healthcare Crisis, War/Military Action, Workplace Issues, AIDS, Arts, Pedagogy, Rights of Privacy, and Sexism.
The applications or purposes of these dialogue and deliberation programs were:
- Community awareness/education (55%)
- To improve intergroup relations (55%)
- To foster civic engagement on the issue (53%)
- To address a local issue (47%)
- To improve group interaction/teamwork (39%)
- Community organizing (35%)
- To foster innovation (31%)
- To influence policymakers (28%)
- Violence prevention (23%)
- Research (14%)
- To influence the media (1%)
Affiliations and Collaborations
Participants were asked to list any and all D&D-related associations or networks with whom they were currently affiliated. Responses were:
- The Association for Conflict Resolution (including SPIDR) (13%)
- The Dialogue to Action Initiative's Dialogue Leaders listserv (11%)
- The Study Circles Resource Center (7%)
- The National Association for Community Mediation (5%)
- Tom Atlee's Listserv (5%)
- The Animating Democracy Initiative (4%)
- The Deliberative Democracy Consortium (4%)
- The International Association for Public Participation (3%)
- The International Association of Facilitators (3%)
- The Public Conversations Project (3%)
2% of our respondents also mentioned the following: Libby and Len Traubman's listserv, AmericaSpeaks, the National Issues Forum, the International Communication Association, and the Open Space community. Various conflict resolution organizations, nonprofit organizations, citizen engagement organizations and education-based affiliations were also mentioned.
Finally, conference participants were asked to list any collaborative projects they or their organizations were currently working on with dialogue/deliberation colleagues outside of their own organizations, and to list their partners in the projects. 31 collaborative projects were listed, plus several other yet unnamed or untitled projects:
- ASPIRE (CA Center for Regional Leadership, Policy Link)
- Best Schools Initiative-NH (Study Circles, Antioch Institute, NIF)
- Campus Race Reconciliation Dialogues (American Univ., Princeton Univ., U of So. Maine)
- Citizens Governance Porgramme (The Commonwealth Foundation ? U.K.)
- Coalition for Effective Juvenile Reform (Metro Crime Commission, Orleans Parrish Juv. Ct.)
- Community Dialogues (Colorado State University)
- Community Empowerment & Involvement Project (IMPACT, City of Silver Spring, MD)
- Community Human Relations Resource Calendar (Human Relations Commission, NAACP, UVA)
- Dayton Race Relations Dialogues (Dayton Mediation Center, Hope in the Cities, others)
- EQUIP (Virginia Tech, Montgomery City Public Schools)
- From Tragedy to Transformation Post 9/11 (City of L.A., Omar Ibn Al Khattab Fndtn, Days of Dialogue)
- Hair Parties (Animating Democracy Initiative)
- Hampton Roads Community Solutions (UVA, VA Conflict Res. Assn, NAFCM)
- Heritage Panels (Study Circles, NCCJ, local universities and students)
- Intercultural Dialogues (NCCJ, Carpetbag Theatre, Highlander)
- Iowa Consensus Project (Univ. of Iowa, NIF, others)
- It's Time (Foundation for Global Communication)
- Land Bridge Project (Children's Theatre Co., City of Montvideo, MN, local farmers, others)
- Listening to the City (America Speaks, Tri-State Port Authority, NY/NJ Devel Corp.)
- Living on the Edge (NOAA, Costal States Organization)
- Municipal Issues Forums (Albany Law School, Study Circles)
- National Issues Convention (McNeil-Lehrer Productions, University of Texas, & others)
- Personal Freedom Forum Post 9/11 (So. Maryland Region Human Rel. Comm., Study Circles)
- Project SYO (Providence College, Liberty Museum, Philadelphia Enquirer, local High Schools)
- Public Conversations Project (PICAR, Harvard Univ., Israeli & Palestinian journalists)
- The Common Ground Partnerships (ACK & Others)
- The Kind of Canada We Want (collaborative partners not listed)
- Transylvania Youth Speak (Study Circles, City of Transylvania, area schools)
- Wisconsin Energy Forum (EPA, Environmental Council of States, others)
- Women Speaks Tours (partners not listed)
- Wythe Horizons (County of Wytheville, Kettering NIF, others)
Summary of Final Day Survey Results
108 completed surveys out of 241 participants (45% response rate)
Conference Venue & Logistics
Respondents were asked to rate 9 conference features on a scale of 1 (poor) to 5 (excellent). 8 of the features were rated, on average, between good/above average (4) and excellent. One was rated between average (3) and good. Here are the features and their averaged ratings, in order of rating.
Feature Average Rating
- On-Site Registration & Check-in Process 4.58
- Organizing Team Response to Logistics Issues 4.53
- Overall Rating of Conference in Terms of Logistics 4.37
- Conference Geographic Location 4.36
- Pre-Conference Registration Process 4.31
- Participant Handbook (the binder) 4.25
- Quantity of Materials & Resources Available to Participants 4.18
- Quality of Conference Resource Tables and Exhibits 4.03
- Conference Setting & Facilities 3.75
Location & Timing of Future Conferences
Respondents were asked whether future NCDD conferences should move to new regions (50%) or continue to be held in the Washington, D.C. area (37%). They were also asked whether they would prefer a Friday through Sunday conference (44%), a Thursday through Saturday event (29%) or a weekday event (11%).
- Break-out session topics and models seemed to represent the spectrum of dialogic and deliberative practice well. (4.09)
- I plan to be active in the group I joined at the Next Steps Forum. (4.01)
- The plenary sessions flowed well from one to the next. (3.84)
- Overall, my own skills and knowledge were enhanced. (3.83)
- I plan to contact some of the people I met here and possibly start collaborative projects with some of them. (3.76)
- The plenary sessions utilized an adequate variety of dialogue and deliberation methodologies. (3.68)
- Plenary (large-group) sessions were effective in finding ways to help the dialogue/deliberation community unite and develop. (3.57)
- Throughout the conference, there was sufficient opportunity to improve skills in organizing dialogue/deliberation programs. (3.34)
- Throughout the conference, there was sufficient opportunity to improve facilitation skills. (3.16)
- From my perspective, conference participants engaged with one another according to the Principles of the event. (4.18)
- The conference began the work of developing an agenda and building an infrastructure to support the future activities of this network and the rest of the dialogue & deliberation community. (4.08)
- The conference helped develop greater cohesion and community among practitioners, and built relationships within the field. (4.0)
- Participants developed collective knowledge and shared considerable information. (4.0)
- Collective insight emerged about the key questions in our field. (3.67)
- Participants improved their skills in facilitating and organizing dialogic and deliberative processes. (3.36)
Additional Comments about Any of the Above
- Overall this conference was highly successful and all of you need to be thanked for providing the space to come together. I am very encouraged that this beginning will go a long way towards developing both dialogue and deliberation. Thank you for all that you have done.
- Try having just one large session for Plenary meetings. This is about dialogue and having many small groups may be workable but we miss a sense of togetherness that one large meeting could bring.
What, if any, key questions facing our field did the conference help participants to identify and/or begin addressing?
- The question of identity. Who are we? What is dialogue and deliberation? Is it in fact a ?field? or a loose network? (34 people mentioned this)
- Diversity/neutrality of the field and its practitioners. How do we involve and welcome ideological opponents ? specifically people with conservative viewpoints? (13 people)
- How can we identify and ?sanction? the many models, and avoid infighting over authenticity? (8 people)
- How can we link with grassroots and/or elected officials and public policymakers? How can we expand the circle? (6 people)
- How do we move from dialogue/deliberation to action? (3 people)
- The need for assessment and lessons. What is the impact of dialogue/deliberation? (3 people)
Did the conference meet your personal expectations?
92% of our respondents said that yes, the conference did meet their personal expectations (8% said no). Comments included:
- There was not enough interaction; not enough actual dialogue (4 people said this)
- There needs to be more varied subject matter and more skill levels (3 people said this ? they mentioned too much of a focus on race, too much emphasis on academic topics, and too high of a skill level)
- There was too much variety; there needs to be more focus (1 person said this)
What do you think was the best feature of the conference?
- The people, and the spirit in the room (19 people said this)
- The workshops and the variety of offerings (19)
- Networking with colleagues (16)
- The plenary sessions (11)
- The diversity of the participants (10)
- Conference content (9)
- Learning/seeing new models (8)
- Conference organization and planning (8)
- The whole conference (5)
- The materials provided (4)
- The accommodations (3)
Additional responses were: the consensus/group decision making, opportunities for dialogue, freedom of choice, and St. Anger's artwork.
What do you think was the worst feature of the conference?
- Time/schedule (primarily the tight schedule and too-short sessions) (41)
- The plenary sessions (7)
- The site/logistics (7)
- Not enough opportunities to actually engage in dialogue (6)
- Not enough minority representation (5)
- Lack of different perspectives (3)
- Lots of ?talking at?/selling ideas (3)
- Academic emphasis (2)
- Unresolved issues (2)
- Participant hypersensitivity (2)
Additional responses: Some moderators dominated; unfocused breakout sessions; some presenters were not as described.
If you discussed or developed any new partnerships or collaborations with other conference participants, please share with us what you will or may be doing.
We were ecstatic to see that 57 respondents (out of 108) indicated that they had begun exploring the possibility of partnering or working collaboratively with other conference participants. Here are some exciting examples of what conference participants were considering:
- "A group of us plan to get together to develop some integrated dialogic models, combining the models we experienced at the Conference."
- "I intend to collaborate with several area colleagues to organize a statewide (Connecticut) dialogue summit on child care."
- "Several of us plan to form a consortium to promote/use dialogue on college campuses."
- "I met representatives from several agencies with whom I intend to develop dialogues on race relations on both the East and West coasts."
- "A group of us are going to help each other with some local work, and then collaborate on some national research."
- "A few of us are forming a 'Deliberation on the ground' group, which will develop some experimental programs."
- "I plan to collaborate with new colleagues to write an article on Deliberative Democracy in a leading Green Party publication."
- "I have several potential regional partnerships, and great resources (the handbook) with which to develop them. Thank You!"
Some of the organizations which were identified by respondents as potential partners were:
- The Center for Disease Control
- The University of Michigan
- Community Mediation Board
- The Forum Foundation
- The Green Party
- Jewish & Palestinian Communities
- The Kettering Foundation
- Local Governments
- MacNeil/Lehrer Productions and By the People
- The JFK School of Government
- University of Southern California
- Local mosques/Muslim community
- Urban Bush Women
- Specialty/Arts Organizations
- Youth Groups & Senior Centers
Are you likely to attend the next National Conference on Dialogue & Deliberation?
85% said that yes, they would be likely to attend the next conference. 11% said maybe, and 4% said no.
How would you rate your overall Conference experience on a scale of 1 to 5 (5 being excellent)?
Our average response was 4.21.
Most people said one of two things:
- This conference was a very important step in the advancement of dialogue and deliberation. (43 people)
- Thank you! (51 people)