Helping Give Away Psychological Science/Creating an Open Teaching Toolkit

=Creating an Open Teaching Toolkit to Level the Playing Field for Access to Research Skills for Analysis and Presentation at Conferences =

This is a project that was funded by the Association for Psychological Science and currently is getting implemented.

=Background= Only one in 20 people alive today live in the United States. Of those, <1 in 50 pursues a graduate degree, and only a fraction of those are able to attend a “Research I” university. Those “fortunate few” have a tremendous adventure in terms of institutional resources, professional networks, research mentorship, and teaching support. For those on the outside, barriers to entry include expensive software licenses, firewalled networks, and difficulty accessing data.

Open Science and open-access have been striving to level the playing field and democratize access to materials and resources. Helping Give Away Psychological Science (HGAPS.org) is a 501c3 dedicated to bringing the best information about psychological science directly to the people who would benefit. Over the five years since incorporation, the organization has grown to more than 300 members and alumni (dashboard here), with an international Board of Directors. It also has an affiliated Wiki User Group. This all grew from prior grants of a few thousand dollars (listed here), including a prior APS Teaching award to UNC Chapel Hill, with prizes and pizza to engage students.

COVID-19 created both a crisis and an opportunity. Disrupted teaching, disrupted meetings…forced us to flip classes and convert materials to online formats. We have been experimenting with combining Wikiversity and OSF.io to replace Sakai/Blackboard/Canvas as a more open way of teaching. Our class and HGAPS meetings have been on a steep learning curve, exploring videoconferencing for synchronous activities, and how other free platforms can augment and improve the pedagogy in high-access ways.

The proposed project will gather teaching materials, convert them to open access formats, deploy them in an organized way that is freely accessible, and share them with a series of state and national conferences. The project uses objective metrics to track usage and success, and will share the results (many in near real time).

= Project Goals = The APS funding will convert teaching materials from an Archival Data Science class, and the Evidence-Based Assessment (EBA) resources that HGAPS has gathered over several years into a suite of teaching resources to build a hybrid “Conference 2.0” Toolkit. Activities and pedagogical materials will use open-access online formats to create an enriched conference experience. The Conference 2.0 Toolkit will meet the needs of three audiences: students, teachers, and conference organizers. Engaging these three audiences will make high quality learning experiences more accessible and engaging, helping students to learn how to conduct statistical analyses and visualizations and present them at scientific meetings. Teachers will enjoy the activities and resources, as well. The conference-oriented toolkit components include innovative conference programming options, open source ways of packaging and sharing meeting products, and a suite of objective utilization metrics that are easy to share with membership.

= The project’s methods, justification, and how they address an issue relevant to the field = The project will adapt materials from traditional classes and pilot projects, with the specific goal of creating a Conference 2.0 format that transforms access to the skills and resources, promotes experiential learning of open science principles and practices, and enhances the dissemination of conference materials. The toolkit will be designed to be crowdsourced and updateable, using open platforms (e.g., R, OSF.io, Zotero, Wikiversity) instead of commercial ones (e.g., SPSS, Dropbox, Endnote, Blackboard). As conferences adopt the methods, their events will further expand the library of examples and resources, also raising visibility and diffusing the innovation.

Feasibility is high because many of the raw materials have been developed and piloted in existing courses. The APS grant is investing in converting this from “walled-garden,” UNC and Lancaster University specific offerings into open source versions, with clear organization and access as Wikiversity teaching resources, a YouTube subchannel, and templates for conference-specific versions. Students are empowered to make, record, and upload their own presentations (with no added work for the instructors or conference organizers), using forms, templates, and automated processes to provide scaffolding.

= How the project advances high-quality, innovative teaching of psychological science = We repurpose tools not traditionally applied to teaching, such as the OSF Meetings platform, as well as open-access data and R notebooks to create teaching resources versus project-specific replications. We also combine the HGAPS EBA materials, repurposing them as teaching support materials. Funded by a series of small grants, including an APS teaching award, the EBA project now has 210+ web pages that would fill a 640 page printed book, with 300 PDFs for free measures and Qualtrics modules for more than 70 that can be imported by the student or instructor. The EBA pages have been viewed 300,000 times (metrics here), and are already on Wikiversity -- the Wikimedia Foundation’s core tool for supporting open teaching.

The project further innovates by using the same platforms to develop conference-specific resources that offer free options for online and hybrid programming. More subtly, because these are themselves open-access, adoption by conferences is a win-win that gives them more content and also enriches the open “library” of examples and templates for future students and teachers to use.

= Impact the project will have on its target audience = The project will share new conference program formats and tools that augment traditional offerings, make them more accessible, and expand the open-access material generated by conferences.

The project improves access to more powerful statistical methods (e.g., worked examples for mediation, moderation, psychometric analyses), as well as making larger curated open-access data available. Students and teachers will have more and clearer examples and more scaffolding as they pursue secondary analyses.

Reliance on open source software promotes skill transportability and generalization. Learners will always have access to R, Zotero, OSF, and Wiki, whereas they will lose their institutional student licenses to commercial products when they graduate.

Finally, the focus on “Conference 2.0” as the teaching goal, and the adoption of these formats by multiple organizations, creates tangible products from learning activities. These give a focus to experiential learning activities, and level the playing field in terms of creating CV-ready research products and open science skills.

= How the project speaks to unmet needs in the teaching and learning community and possibly assists international or other under-served populations = The potential benefits of adoption of the processes and products of the Conference 2.0 format include much greater access and engagement for traditionally underserved populations and international students. The curated open-access data has prioritized a focus on more representative samples (e.g., low income, community mental health, or international samples, inclusion of nonbinary gender). Similarly, the R code and teaching examples put methods for evaluating potential bias in research tools front-and-center as teaching and research topics (e.g., tests of statistical moderation--are the effects the same for different groups; multi-group factor analyses, and differential item functioning). Sharing these materials offers knowledge that has previously been siloed within labs or institutions and makes it globally, freely available.

The open-access platforms are internationally accessible, eliminate cost barriers, and the asynchronous projects (e.g., recorded videos, uploaded materials, individual “homework” assignments) avoid time-zone challenges to scheduling. The pilot activities have resulted in more than 160 student presentations at peer-reviewed national and international meetings to date, with participation from more than 13 different countries.

Drawing on the affiliations of members of the Board of Directors for HGAPS creates natural partnerships for several professional society meetings, many of which piloted aspects of the Conference 2.0 Toolkit at their own conferences. The partnership with the North Carolina Psychological Association in particular offers opportunities to engage smaller teaching colleges and different student demographics.

= Similar resources or projects and how the current proposal adds to or differs from them = During the pandemic, conferences either cancelled or flipped to online. All of the teaching, HGAPS EBA, and Wikiversity conference pages were built and used in face-to-face traditional formats. Their online components were adjuncts--not the primary delivery format. The HGAPS YouTube Channel and recording lectures are new initiatives.

Other meetings have used some of the platforms, but none with an explicit connection to Open Science or Wikiversity. For example, the STEM Village organized a virtual symposium, where some presenters recorded their posters onto YouTube (link to example). Similarly, #LIBRE2020 had poster snapshots presented publicly on Twitter. We also differ via incorporating the OSF.io and Wikiversity elements.

= Summary = The proposed project uses funding to focus activity on taking prior materials and pilots and building an organized, open-source platform that can augment existing conferences while creating shareable teaching materials that improve access to high quality data, analytic code and examples, and templates and support to transform the access of traditionally marginalized learners. The result will be a “Conference 2.0” toolkit that teaches open science practices through experiential learning in a way that credentials learners with peer-reviewed presentations and a portfolio of work that prepares them to engage with their personal and professional development.

=External links for related resources= Here is the link to suggest videos to be uploaded to the HGAPS YouTube page.

=References=