The GRE is a test required in order to get into many graduate programs including a philosophy program. Part of the test is on analytical writing, which requires us to give an “in-depth analysis of complex ideas” and provide “logically compelling reasons.” I am not convinced that the people grading these essays are qualified to know what counts as an in-depth analysis or a logically compelling reason and the company that runs the GRE tests, ETS, has no accountability. In particular, the company has refused to explain to me specifically why my grade was a “4” despite my best efforts to get them to do so. A human being grades these tests and should know how to justify their grades, but we are just expected to have faith that everything is done perfectly right without a shred of evidence.
The GRE analytical writing graders are often English professors, who might have never taken a single logic class. (I don’t think a single logic class would even be sufficient to understand what it means to give “logically compelling reasons” because it can take years of experience to realize how logic applies to real-life arguments.)
I have spent years figuring out the best way to write arguments, I tutor students how to do so, and I have written about it in a logic reader (made to teach students introductory logic). As far as I could tell, my essays did provide the criteria the GRE requires to get a “6” (the maximal score), but the company insists that I deserved less without any explanation. If I am taking the test wrong, I want to know why. I would like to correct my mistakes, but that isn’t possible when no feedback is given. I suppose I could take the test several times and figure out what they are looking for via trial and error, but that would cost a fortune and it would only be effective given the assumption that the test isn’t a complete sham in the first place.
The first time I took the GRE around six years ago, I pretty much just told the reader what to think. I said what I believed with little to no justification. However, I might have used an example that really got the English teacher graders excited. I ended up getting a “5” and I was in the 80th percentile.
I had to retake the test in December 2009 and I got a “4” (I was in the 40th percentile.) So, I’m basically twice as bad at the task after getting a master’s degree in philosophy, teaching logic, teaching people how to write argumentative essays, writing about it, and spending a lot of time thinking about it. Now that I wrote an essay that demonstrated arguments to be valid, questioned the justification of premises, explained the logical ramifications of my arguments, and tried to provide independent justification of the arguments, I deserve a much lower score. Apparently my “examples” weren’t as fun as the readers would have liked. I suppose I might have had some grammatical errors or something as well.
I contacted ETS and I was told that they could do an investigation to find out why I got a “4” and to regrade my essays for a $55 charge. I didn’t care so much about getting regraded. I mostly just wanted to know what I did wrong, but I decided to pay for the investigation anyway, and you can see my fax request here. A month later I got a letter from ETS stating that the grade was correct but no explanation was given. I contacted ETS again and I was told that getting an explanation would be impossible. I was misinformed. I then sent the following complaint to ETS:
I want to know why I got a certain score on the analytical writing test. I got a lower score than before and I need to know precisely what I did wrong in order to further understand how these tests are graded. I have written a logic book, I have tutored students about writing argumentative essays, and I have taught philosophy classes, so I have an interest in writing good essays and I know quite a bit about it.
It would be a great idea to compare what made my first test a higher score than my newer test. This could help clarify what I did right before and what I did wrong recently.
I contacted ETS through the toll free number and spoke with a woman who believed that I could get information about why I got a certain score. I was told that I should request a “Score Review for Analytical Writing Measure” in order to get information about what I did wrong. I thought I made it clear in the fax that my intention was to know what I did wrong, but there might have been some misunderstanding on both sides.
I contacted ETS again and spoke with an employee. He said that there is no way to find out what exactly I did wrong on the test and there is no way for me to contact the people in charge (other than filing a complaint).
I have the following two complaints:
- I was told that I could get information about what I did wrong, but the information was false. I sent money to get the information that I would have not spent otherwise.
- There should be a way to find out what I did wrong on the test. It is important for ETS to prove its accountability. The best way to do that is to provide information about how we are being graded.
You should be willing to prove your accountability for the following reasons:
- There is no way of knowing what exactly lowers scores without precise information given.
- There is no way to know if we are being treated property without knowing why we are losing points on the test.
- There is no way to know if the graders are doing their job properly without knowing why they are giving low scores to certain people.
- If you are unwilling to prove your accountability as a company, then your company is unwilling to be accountable for its actions and the accuracy of your tests will be questionable.
- If no one can know why they get a certain grade, then no one could question the judgment of your graders. No one could criticize the graders.
- If it is impossible for the outside world criticize your graders (or grading criteria) then (a) graders have less incentive to do a good job and (b) it will be more difficult for graders to know how to improve their grading. (If grading criteria is at fault, then it will be difficult to know how to improve the grading criteria.)
- If you are unwilling to be accountable as a company, then schools have little reason to require applicants to take your test. Your company will no longer have a purpose.
I know that you mainly use English professors to grade these papers and I have to wonder if they are qualified to grade all the papers. As I mentioned in my fax request: There is no requirement for English professors to know logic, but the score information states that I am graded on providing “logically compelling reasons” and “precise vocabulary to convey meaning effectively.” For this reason I used logical vocabulary, such as “modus ponens” and “valid form,” and I discussed the logical implications of the arguments. It is possible that this actually hurt my analytical writing scores. It is quite possible that your graders are unable to properly grade the essays based on the supposed criteria required for high scores.
I would like to get specific information about what I did wrong on my test. If that is impossible, I want a refund. I didn’t spend $55 just to get my test re-graded. I never even disputed the grade considering that I don’t even know exactly how you grade the essays in the first place.
It has now been around 3 months and I haven’t heard anything back.
I suspect that the GRE analytical writing test is not graded by qualified professors and I am evidence that my suspicion is correct given that I am a straight A student with a master’s degree in philosophy, which is greatly concerned with good reasoning, and I have spent so much time learning and teaching logic. I have taught and tutored many students about how to understand logic and how to apply logic to their essays.
I suspect that I was punished for using logic and logical terminology. They do score you partly on your vocabulary, and the logical terminology I used had no room for interpretation, such as “modus ponens.” However, they might want you to define “modus ponens” because it’s not part of an English professor’s vocabulary. Of course, it would be unreasonable to have to define “modus ponens,” “valid logical form,” and so forth for a 30 minute essay, so it might be impossible to use logic to maximize clarity. You are graded on vocabulary, but I’m not sure how to use vocabulary in the appropriate fashion. (Vocabulary might be best when used to manipulate the reader rather than to be clear and honest in a logical sense.)
Additional evidence that the GRE does not require logic
To further corroborate my “suspicion” that the GRE scores are unreliable, consider the “perfect score” example here. There we are presented with an argument that supposedly presents “relevant supporting reasons and examples.” The student argues that “The safety of consumer goods can best be ensured not by way of government regulation but rather through voluntary efforts of the private businesses that produce those goods.” Why should I agree? Because “One compelling argument against such regulations is that they are costly to administer and enforce, and can even be counterproductive.” This is irrelevant. If regulations are costly to administer and enforce, it might still be the “most effective way to ensure safety.”
Then the student considers an objection — Someone might think that corporations will make unsafe consumer goods to save money. (Well, this is exactly what should have already been proven by now.) This objection is found to be unconvincing by the student because “this argument overlooks the offsetting economic and social benefits of free enterprise.” Wow, now I’m convinced! It would be impossible for a corporation to make unsafe products because government oversight is so costly, right? Wrong. This is nothing more than a red herring. The student has completely changed the subject to the financial burden involved with government oversight. That’s not what we were talking about. We were talking about whether or not government oversight helps corporations make safer products.
Do English Professors Know Logic?
I have good reason to suspect that English professors know little about good reasoning. Not only do many English professors detest logic, but even English professors who have an interest in logic can’t be expected to know a whole lot about it. Purdue University has a lot of advice about writing essays on their Owl website, and it even has information about formal logic. However, much of the information concerning logic is false. I contacted the person in charge around four months ago, but apparently he has been too busy to make the changes. Here are the errors that I found:
1.) It is stated that a syllogism is false (example D). A syllogism can’t be false. The conclusion can be false, but that isn’t what is relevant. What is relevant is that the syllogism is invalid. An invalid syllogism can have a true conclusion, but an invalid syllogism can’t prove the conclusion to be true, even if it is.
2.) Validity isn’t explained. That must be defined in order for the students to know what logic is about. Invalidity should also be explained and students tend not to know how to define it.
3.) The next example (E) states that the conclusion is “logically valid,” but it is then said to be “only true if an audience accepts Premise 1, which is very unlikely. This is an example of how logical statements can appear accurate while being completely false.”
Conclusions can’t be valid. The syllogism itself can be valid, but it can’t be true. Additionally, the fact that someone “accepts Premise 1” doesn’t make the conclusion true. Truth isn’t determined by our beliefs. What should be said is that the argument is unpersuasive to anyone who doesn’t accept Premise 1.
We have no reason to trust the GRE analytical writing test and ETS seems to have ignored my complaint. ETS refuses to explain why we get lower scores than we would like, so it is impossible to effectively increase our grade based on the appropriate information (assuming that the essays are graded using reliable criteria). Additionally ETS lacks accountability, hides behind secrecy, and seems to have no mechanism to improve itself. I suggest graduate schools stop requiring applicants to take the GRE analytical writing test. It would be best for graduate schools to refusing to do any business with ETS until the company is improved and proves its accountability. ETS has no reason to improve if there are no negative consequences to its actions and I doubt graduate schools are willing to do anything about the situation.
If you would like to know why you got a certain score on a GRE analytical writing test I would suggest contacting them to find out. If they refuse to give a detailed explanation, you should insist to fax a complaint. The fax number for complaints is 1-609-771-7715.