ACCESS THE PUBLIC VERSION of SAWHET v2 HERE!

 

What is SAWHET?

The STEM Automated Writing Help Tool (SAWHET) is a web-based workflow we developed initially for an NSF IUSE grant to evaluate automated support of student writing in biology courses. SAWHET v1 collected and analyzed reports in 3 stages:

• Intake validation. Students submitted lab reports using a web form that checked if all sections were present and met minimum length requirements. Each completed submission received a unique submission ID, and the raw text was passed to the next stage.
• Rules-based screening. Texts of reports that passed initial checks were forwarded to GATE (Generalized Architecture for Text Engineering) which performed additional checks of the items below. Responses were appended at the end of the report text.
  • Is there a clear hypothesis statement in the introduction?
  • Are there citations in the introduction and discussion sections?
  • Are the methods written in narrative form or as bulleted/numbered steps?
  • Do the results refer to any figures and/or tables?
  • Is the title overly short or long?
  • Are results of statistical analyses included and formatted properly?
• Formatting, distribution, and storage. After GATE screening, the text, appended screening reports, and image files were merged and converted to DOCX format. Files then were either sent back to the original student authors for review and revision, or forwarded to GTAs for grading. Duplicate plain text files also were added to our local reports archive.

SAWHET v1 proved a valuable research and teaching tool but was not scalable. It used proprietary web services licensed to our home institution, and required significant Java programming skills. To share SAWHET with others we needed a more nimble framework.

 

SAWHET v2.0

SAWHET v2.0 is an open-source R Shiny application that we and other users can modify or expand easily. Version 2.0.0 provides the report collection web form and basic content validation.

Other users can install a fully functional copy of SAWHET in about 1 hour. Course numbers, instructors, and help tools are modified by changing the plain text files that contain the data. Full instructions are in the Deployment Guide.

Our Resources

Resources	Links
Current version (v 2.0) on ShinyApps.io	Web page
Minimum viable version of SAWHET v2.0	Web page
Code for R Shiny package (released under GNU GPLv3 license.)	GitHub repository
Deployment Guide	R/MD file   DOCX file

 

Looking Ahead

We are recruiting collaborators with experience writing R Shiny applications to help us expand SAWHET's capabilities. These are a few of the potential screening functions that can be added to the core report collector framework. Items marked "*" have been implemented already; as new functions are added, we will update the demo version of the app and release updated code to GitHub.

Types of Checks	Examples
Binary (yes/no) checks	Are all the required text sections present?* Are there images (figures, tables)?
Count checks	Is the "X" section at least "#" words long?* Are there at least "#" citations? (partial)
Rules- &  REGEX-   based checks	Do the introduction and discussion sections have citations?* Are the citations formatted correctly? Is there a statement of the central hypothesis in the introduction? Are the methods in a narrative format or bulleted list format? Are statistical results summarized correctly?
Comparison checks	Are any 4-5 word phrases in this report also present in other reports in the data archive? Are any words in the master vocabulary list misspelled?
NLP checks	At what reading level is the report written? How many noun phrases are present?

 

If you have suggestions for other functions to add to SAWHET v2.0, please share those idea in the Comments. Check the list of To Do items in the automated support sub-project, and let us know if you want to take on one or more items.

We also are interested in exploring two educational research questions related to automated feedback.

• How well does automated text analysis support student learning? Can it extend what instructors do beyond current capabilities, or does it only replicate what instructors already do?
• Students claim to want more opportunities for feedback and revision, but do they actually act on automated feedback?  

 

---

Where to Learn More

Introduction to R Shiny Applications and Development  

1. Building Web Applications with Shiny
2. Resources for Learning Shiny App Development
3. Mastering Shiny, by Hadley Wickham

 

R Packages for Text Mining and NLP

Package documentation provides some general background on text mining methods. All of these packages are compatible with R Shiny.

1. corporaexplorer
2. koRpus    
3. OpenNLP    
4. Quanteda    
5. Text2Vec    
6. TM