.st0{fill:#FFFFFF;}

WriteWise: Improve Paper Quality by 22% and Productivity by 15% 

By WriteWise Team

August 26, 2021

EdTech, English

How are we so sure WriteWise can help you improve your writing quality and speed?

Not like other writing tools.

At our core, we are a team of researchers that has developed a writing tool for other researchers. And since we are researchers like you, we wanted to objectively quantify just how useful WriteWise is for writing research articles

WriteWise is not like other writing tools that claim they are the "best" without any supporting data. Instead, we can confidently say that WriteWise will improve your paper quality by 22% and writing productivity by 15%.

separador

Experimental Design

Evaluation of Writing Quality

Writing Quality Experimental Design 1

Writing quality was defined through the use of a validated assessment rubric (see Data Analysis). Prior to starting the evaluation, participants were requested to submit an abstract based on their own research. This pre-test abstract was written by each participant without the use of WriteWise. During the evaluation, the participants were asked to revise their abstract using the WriteWise software. The pre- and post-test abstracts were then compared to evaluate any changes in quality.

Evaluation of Writing Productivity

Writing Productivity Experimental Design scaled

Writing productivity was defined as the speed with which participants wrote a new abstract. Two research articles were summarized by the WriteWise team to approximately 1,000 words each (i.e. Article A and Article B). The summarized research articles were from different areas of study but were easily understood by non-specialists, being ranked by the WriteWise team as approximately equal in difficulty.

Participants were randomly assigned one of the two summarized research articles in Round 1, and they were assigned the other summarized research article in Round 2 (i.e. all participants received Article A and Article B, but the order was random). The participants were then given 30 minutes to read the summarized article and write a 200-word abstract based on the given information. 

In Round 1, participants did not use the WriteWise software to write the abstract. In Round 2, participants used the WriteWise software to write the abstract.

Participants

  • Convenience Sampling
  • Informed Consent 
  • Included PhD Students and Research Faculty from:
  • Universidad de Concepción
  • Universidad Andrés Bello
  • Universidad Santo Tomás
separador

Data Analysis

Evaluation of Writing Quality

All pre- and post-test abstracts were subject to evaluation of writing quality by a team of trained linguists specializing in the academic discourse. The steps for objective quality assessment were as follows:

  1. 1
    Rubric Development (Experts in Academic Discourse)
  2. 2
    Rubric Validation (External Reviewer)
  3. 3
    Rubric Improvement
  4. 4
    Training of Rubric Raters
  5. 5
    Randomized and Blinded Revision and Rating (2 Raters per Abstract) 
  6. 6
    Minimum Rater Agreement Required (Cohen's Kappa Coefficient)
  7. 7
    Ratings Averaged for Each Abstract
  8. 8
    Statistical Validation and Results (Student's T-test)

The linguistic parameters considered in the Evaluation Rubric were as follows:

  • Theme and Topic Suitability
  • Fit to Audience
  • Fit to Communicative Purpose
  • Semantic Relations
  • Use of Specialized Lexis
  • Sentence Length
  • External Structure
  • Presentation of the Introduction
  • Presentation of the Methods
  • Presentation of the Results
  • Presentation of the Conclusion
  • General Appraisal

Evaluation of Writing Productivity

Participants used a specialized portal to write each abstract. This portal monitored all keystrokes and associated timestamps. These data allowed for analysis of writing speed. Statistical significance was assessed using an alpha = 0.05.

separador

Results

Writing Quality

The results of the agreement index indicate that, using the holistic criterion, the percentage of agreement was above 60%. The Kappa analysis was moderate, and there was an average positive correlation (0.21-0.40) between the analyses performed by the raters.

Regarding the quality of the writing, all of the evaluated criteria presented higher post-test scores (Figure 1). The pre-test average was 3.3, and the post-test average was 4.1 (out of a maximum of 5).

Criteria Comparison Figure 1

Figure 1.

When applying the T-test for comparisons, there were significant differences in criteria 12 and 13; i.e., average of all criteria values and average of overall holistic appraisal, respectively. Other criteria showing statistically significant difference were as follows: theme and topic suitability; fit to audience; fit to a communicative purpose; semantic relations; sentence length; and presentation of the conclusion.

Given that the criteria of the general average and overall holistic appraisal of the raters presented significant differences as related to the writing quality of the abstracts, the percentage gain in writing quality notable (from 44% to 67% achievement). If the ideal score is 5 (100%) for each criterion in Figure 1, a 22.1% gain in achievement is observed in the post-test for the overall appraisal and a percentage gain of 15.04% or 18% was recorded for the average of all criteria or only criteria with significant differences, respectively (Figure 2).

Pre and Post test Figure 2

Figure 2.

Writing Productivity

With respect to productivity, it was hypothesized that the mean time (ms) used to write abstracts in Round 1 would be greater than abstracts written in Round 2  (alpha = 0.05). Any reduction in writing time would presumably be due to the didactic intervention and the application of WriteWise.

The results indicated that there was a significant difference (p = 0.016) in the difference of means. That is, in Round 1, the temporal duration was greater than in Round 2. Therefore, the time used to write an abstract in Round 2 was significantly shorter (5.9 minutes on average). As a percentage, this represented a 14.9% reduction in writing time as compared to Round 1.

separador

Interested in more published studies conducted by WriteWise?

separador

Other Learning Resources:

{"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}