Workshop 1
Have you ever read a paper and thought "I wish I could get my hands on that data!" Have you ever thought about doing a meta-analysis, using data from several different studies, including data you did not collect? Have you wished there was a way to make your data available to others? Have you developed a new analytic technique and wished for data to test it on outside of the projects you yourself have been involved in? Sharing computing-education data would make these and other new projects possible.
Common data sets are widely used in other fields. Researchers interested in Child Language Acquisition have used CHILDES (the Child Language Data Exchange System) since the 1980s. Molecular biologists routinely submit their data to permanent public repositories, and indeed, doing so is a condition of publishing their research in certain journals. Within computer science, fields such as information retrieval and vision have also benefited from such datasets.
This workshop will discuss the key challenges in constructing and maintaining a repository of empirical computing-education research data. Important questions include: Where would it be? Who would maintain it? What hardware and software would be needed? Who would contribute data, who would have access, what format would the data be in, and what should be done to protect the privacy of the human participants involved in the experiments? The Institutional Review Boards (IRBs) of the institutions where human-subjects data is collected in the United States must approve the experiments in advance -- what steps must be taken to convince the IRBs that placing the data in a repository would be acceptable? What comparable restrictions apply in other countries?
It is our intent, by gathering members of the empirical computer-science education research community together, to determine whether there is interest in such a repository, and if so, begin to build a consensus as to the form it should take. Participants will be divided into subgroups, each charged with examining a particular set of issues and preparing a preliminary analysis before the workshop. These reports will be presented and discussed at the workshop. The final output of the workshop will be a report summarizing our analysis and discussion, which will be submitted for publication in an appropriate journal.