Analysis & Summary of Evaluations

Analysis & Summary of Session Evaluation Results

Photo of Eric Boyd.

Eric Boyd

Evaluation forms were collected for each of the 56 break-out sessions. Although not everyone chose to complete evaluation forms for the sessions they attended, the forms that were completed provided us with some very interesting and important information. Our Reports & Evaluation Committee, ably chaired by Eric Boyd, former Executive Director of Los Angeles Days of Dialogue, compiled and analyzed this data and wrote the following narrative.

Narrative Analysis

Break-Out Sessions

It was clear from the results of the quantitative section of the survey that Conference participants overwhelmingly favored workshops that exposed them to new and innovative methods of D&D practice over those workshops that were topical (i.e. political, race relations, etc.) or even international. This was very telling, especially with international issues so prevalent in light of impending war with Iraq.

Tom Atlee presenting a break-out session.

Tom Atlee of the Co-Intelligence Institute presenting a session about Citizen Deliberative Councils.

As such, the highest session ratings were consistently garnered by the newer, more ?cutting edge? workshop sessions. Top scorers were Liz Lerman's session entitled ?Dance Exchange on Dialogue and Making Dance,? Duke Duchscherer's ?Nonviolent Communication: Practical Tools?,? The Animating Democracy Initiative's ?Arts & Civic Dialogue Exchange,? Irene Papoulis and Beverly Wall's ?Using Informal Writing to Foster Democratic Dialogue,? and Active Voice's ?Using Film to Build Dialogue.?

Clearly, the hunger for new applications and exposure to new models should be a primary focus for future conferences. However, since some more topical or traditional workshops (i.e. those focusing on Bohm dialogue, the achievement gap and white privilege) garnered the highest numbers in participant attendance, an equitable balance between traditional topics and new models/applications should be pursued.

It was also apparent that there is a hunger for more interaction, evidenced by the fact that session presenters received high marks for their delivery, but relatively low marks for participant contribution opportunities. Time allotment will also require a great deal of thought in future conferences, as it registered the lowest ratings of all measured items. Still, participant objectives were largely met, materials and presenters scored high, and overall session ratings were highly positive.

Next Steps for Field
A photo from the conference.

A snapshot from the plenary session organized and facilitated by AmericaSpeaks.

For the most part, participants felt that if in fact the dialogue & deliberation community views itself as a ?field? or ?movement,? then that body should work assiduously to ?show its wares? and/or ?showcase its stars.?

Relative to the individuals who comprise our ?field,? a significant number of members view the question of neutrality as one of vital importance to the field. In addition to the conference Organizing Committee's recognition of this emerging issue ? as evidenced by verbally addressing the issue at the final plenary session ? a number of individuals attending workshop sessions also indicated concern in this regard. One participant who attended the ?Creating Space/Restorative Justice? session stated the following: ?Sentiments for Offenders were troubling (almost pro offender). Need more work on what it means to 'create safe space' vs. advocacy.?

Whatever the next steps are for the field, feedback from participant evaluations make it very evident that practitioners believe that addressing ?The Neutrality Question? should be included in our list of priorities.

Compilation of Evaluation Forms

Respondents were asked to rate how strongly they agree with 11 statements about the session they just attended. 9 of the features were rated, on average, between agree (4) and strongly agree (5). Only two statements rated between neutral (3) and agree (4) ? those having to do with ample time and ample participation. Here are the statements and their averaged ratings, in order of rating.

We also asked a series of questions to help us understand more about participants' interests and needs, and about session outcomes. The first question was multiple choice; the rest were open-ended. Here are the most common responses we received.

Please tell us why you chose to attend this particular workshop session.

  1. The topic (40%)
  2. To learn about a model that is new to them (30%)
  3. The session leader (20%)
  4. Content relevant to a current or planned initiative (5%)
  5. Content relevant to current community need (3%)
  6. Other (2%)

What was the best thing about this session? Top three responses, in order of popularity:

  1. The presenter(s)
  2. Having the opportunity to use the model/technique
  3. The materials and handouts

How can future sessions be improved? Top three responses, in order of popularity:

  1. More time
  2. More practice; less lecture
  3. More in-depth workshop on various topics

What next steps does this session prompt you to pursue (if any)?

  1. Incorporate model/learned material into our own program; combine techniques
  2. Seek collaborations with presenter or similar organizations/individuals
  3. Study the model/workshop content further (learn more online, attend an event using the model, speak and/or correspond with presenter, etc.).
  4. Explore other related fields (i.e. mediation, counseling, peace negotiations, etc.).

What next steps does the session convince you the dialogue/deliberation field should pursue (if any)?

  1. The D&D movement should promote this model nationally and/or advocate for its use (various models).
  2. We should use the model at future conferences.
  3. The field should focus on/emphasize neutrality more.

Summary of First Day Survey Results

97 completed surveys out of 241 participants (40% response rate)

Motivation and Expectations

Participants indicated that they attended the conference to build their skills (38%), to learn about dialogue methods and tools (38%), to help strengthen and unite our field (26%), and to meet their colleagues in the field (25%).

A snapshot from one of the plenary sessions.

Other reasons mentioned by 8% or fewer were to improve their job prospects, to discern similarities and differences between models, to make culture more essential to understanding dialogue, to improve personal relationships, to stay in touch with others throughout the field, and to share resources.

When asked what their primary expectations were for the event, participants stated their hopes to contribute to the conference from their own experience (20%), to network with their colleagues (19%), to obtain and share resources (14%), and to learn about new methods and models (13%).

Other expectations mentioned by 8% or fewer of the participants (in order of frequency mentioned) were: to advance knowledge and understanding in the field, to reenergize my commitment to this work, to participate in dialogues, to learn about evaluation techniques and tools, to broaden my studies and experience, to get new ideas, to initiate collaborative projects and relationships with others in the field, to examine and rethink my own personal viewpoints, to learn about ?what's next? in the field, to hone my skills, to learn about the U.S. dialogue and deliberation community, to share concerns about the field, to discuss a stronger democracy, to explore new ways to engage, and to have fun!

Participants were asked whether there were any topics or activities not listed on the conference schedule that they would have liked to see. Only 16 comments were offered, half of which suggested that more youth-oriented sessions should be offered. Two comments stated that more K-12 and college-based sessions should be offered.

Other comments stated that more sessions should be offered that focused on dance and theatre, the use of dialogue in business, non-verbal dialogue techniques, additional open forums, dialogue and deliberation as tools for social justice, and identifying and recruiting elected officials for participation in dialogue and deliberation programs.

A space was provided for additional comments. Comments ranged from questions such as ?How does the D&D field see itself in relation to efforts such as the critical thinking movement in education that emphasize dialogue between teachers & students?? to suggestions such as ?Ensure that facilitators model good practices.? And from compliments such as ?Well planned conference! The handbook is awesome!? to critiques such as ?Some of the workshops need longer time slots.?

Additional comments were:

Experience/Professional Identification Data

Respondents acknowledged having direct experience with the following models of dialogue and deliberation:

Additional models mentioned by three or fewer respondents were: Black/Jewish Dialogue, Jewish/Palestinian Living Room Dialogue, Bill Isaacs' Model, Story Circles, Wisdom Circles, Critical Response, Deep Listening, the International Communication Association model, Consensus Building, Mobius model, National Coalition for Community & Justice, Public Dialogue Consortium model, Restorative Justice model, Search for Common Ground, Track II Diplomacy, and Transformative Mediation.

When asked what roles they played with those models, 82% said that they had facilitated those models, 68% had participated in programs utilizing the models, 49% had organized events utilizing the models, and 49% had trained facilitators for the models. 32% had written discussion guides or other materials for the models, 26% had served as MC or moderator of programs utilizing the models, 26% had observed the models, and 25% had served as resource persons or panelists regarding the models.

Respondents had addressed the following issues through dialogue and deliberation:

Additional topics mentioned 3 times or fewer were: Business/Economic Development, Affirmative Action, Healthcare Crisis, War/Military Action, Workplace Issues, AIDS, Arts, Pedagogy, Rights of Privacy, and Sexism.

The applications or purposes of these dialogue and deliberation programs were:

Affiliations and Collaborations

Participants were asked to list any and all D&D-related associations or networks with whom they were currently affiliated. Responses were:

2% of our respondents also mentioned the following: Libby and Len Traubman's listserv, AmericaSpeaks, the National Issues Forum, the International Communication Association, and the Open Space community. Various conflict resolution organizations, nonprofit organizations, citizen engagement organizations and education-based affiliations were also mentioned.

Finally, conference participants were asked to list any collaborative projects they or their organizations were currently working on with dialogue/deliberation colleagues outside of their own organizations, and to list their partners in the projects. 31 collaborative projects were listed, plus several other yet unnamed or untitled projects:

Summary of Final Day Survey Results

108 completed surveys out of 241 participants (45% response rate)

Conference Venue & Logistics

Respondents were asked to rate 9 conference features on a scale of 1 (poor) to 5 (excellent). 8 of the features were rated, on average, between good/above average (4) and excellent. One was rated between average (3) and good. Here are the features and their averaged ratings, in order of rating.

Feature Average Rating

Location & Timing of Future Conferences

Respondents were asked whether future NCDD conferences should move to new regions (50%) or continue to be held in the Washington, D.C. area (37%). They were also asked whether they would prefer a Friday through Sunday conference (44%), a Thursday through Saturday event (29%) or a weekday event (11%).

Conference Content

Conference Goals

Additional Comments about Any of the Above

Participant's Narrative

What, if any, key questions facing our field did the conference help participants to identify and/or begin addressing?

Did the conference meet your personal expectations?

92% of our respondents said that yes, the conference did meet their personal expectations (8% said no). Comments included:

What do you think was the best feature of the conference?

Additional responses were: the consensus/group decision making, opportunities for dialogue, freedom of choice, and St. Anger's artwork.

What do you think was the worst feature of the conference?

Additional responses: Some moderators dominated; unfocused breakout sessions; some presenters were not as described.

If you discussed or developed any new partnerships or collaborations with other conference participants, please share with us what you will or may be doing.

We were ecstatic to see that 57 respondents (out of 108) indicated that they had begun exploring the possibility of partnering or working collaboratively with other conference participants. Here are some exciting examples of what conference participants were considering:

Some of the organizations which were identified by respondents as potential partners were:

Are you likely to attend the next National Conference on Dialogue & Deliberation?

85% said that yes, they would be likely to attend the next conference. 11% said maybe, and 4% said no.

How would you rate your overall Conference experience on a scale of 1 to 5 (5 being excellent)?

Our average response was 4.21.

Additional Comments

Most people said one of two things:

--

Return to either the NCDD Conference page or the top of this page...