Documentary Impact Evaluation

One model for measuring the impact of documentaries is presented in a 2014 paper by Diesner, Pak, Kim, Soltani, & Aleyasen

Elements in the Model

 * Dimensions: Content, Medium, Target, Impact,
 * Level: multiple levels exist within each dimension, Diesner et. al defines this as a "set of sub-categories of evaluation criteria"
 * Evaluation Factors (what Diesner et. al call "Index") the set of evaluation factors in a given level.
 * Analytics = analytic methods that can be used for each evaluation factor
 * Items: sets of "specific features to be measured" within each evaluation factor.

Levels:

 * 1) Message = This can be identified by interviews with the film-maker, document analysis of governing and planning documents for the film, and analysis of the content of the film itself. Diesner et. al define a message as "the main message that a film wants to convey." Reflection questions:  1) Do you think that this definition is necessary and sufficient for the "Message" of a documentary?  2) What would you add or subtract from this definition? 3) Why?
 * 2) Expected Outcome = These can be found in the grant application, through interviews with the film-makers, etc.
 * 3) Evaluation Priority = Create a list of priorities based on the outcomes identified in the step above, and rank them in order of importance.
 * 4) Resource = The resources needed to produce the film. This level is necessary for any evaluation of cost-effectiveness.

Evaluation Factors:

 * 1) "Guiding Factor"

Analytics:

 * 1) Description
 * 2) Ranking
 * 3) Weighing

Items:

 * 1) Reports by producers (Diesner et al, 2014, Table 2, p.6)
 * 2) Reports by funding agencies (Diesner et al, 2014, Table 2, p.6)

Level 1

 * 1) Individual
 * 2) Communal
 * 3) Societal
 * 4) Global

Diesner et al identify four categories of measurement that For each of these levels:
 * 1) Cognitive <- Evaluation factor "Awareness", analytical methods: "Statistics, Text mining, web analytics, network analysis", as well as surveys
 * 2) Attitudinal  <- Evaluation Factor "Sentiment", analytical method: "Sentiment Analysis"
 * 3) Behavioral <- analytical methods: text mining, web analytics, network analysis, as well as observational protocols.
 * 4) Engagement <-
 * 5) Enactment
 * 6) Connectedness
 * 7) Capacity
 * 8) Expansiveness
 * 9) Centralization
 * 10) Temporal <- Evaluation factor: "Impact Dynamics", analytical method: "Longitudinal analysis"

Diesner et all call these the "four stimulous dimensions" (p. 8)

Analytics:

 * 1) Text Mining (mentioned twice, for Behavioral and Cognitive levels)
 * 2) Web Analytics (mentioned twice, for Behavioral and Cognitive levels)
 * 3) Network Analysis (mentioned twice, for Behavioral and Cognitive levels)
 * 4) Longitudinal analysis (mentioned once, for Impact Dynamics)
 * 5) Sentiment Analysis (mentioned once, for Attitudinal level)

Applicable to Temporal categories:
 * 1) Longitudinal Analysis