No evidence is provided. I'm curious whether the author actually has evidence or is just asserting that this is possible and likely happening.
IME, and I have lots of "E" (including training and finetuning my own models), this probably isn't entirely true in most writing-heavy courses. You can use the largest language models, in an interactive fashion, to generate portions of half-decent papers. Maybe even "ace" papers on certain subjects with certain grading criteria (namely, "looks reasonable", not "fine-toothed comb"). But in many courses the essays will be extremely low-quality, and even in the happy cases saying that the essay is "machine-generated" is eliding a lot of manual effort.
I agree that students are probably using LLMs for their homework, but I'm skeptical that they are all getting As on assignments that are designed as big assessments, or that the essays are actually fully machine-generated. I bet a lot of students -- the laziest ones -- are getting "WTF is this essay even about... did you have a stroke while writing this?!" feedback if they are using LLMs to generate essays whole-cloth.
Pedagogically, this matters. Think about calculator usage. There's a huge difference between allowing use of TI-83 on Calculus assignment with lots of word-heavy application problems and allowing use of Wolfram Alpha on a Calculus assignment that's "integration recipe practice".
Yeah, I could believe it "aces" homework in a really open-ended writing assignment. An assignment like: write an essay explaining a personal experience and what it meant to you. The people that's the biggest issue for at the moment are probably writing instructors, since the goal of those classes is to just practice writing something/anything. In computer science though, the writing I've had turned in in my classes that I suspect is LLM-generated usually gets an F. It tends to just ramble about the subject in general and not hit any of the specific points that I'm asking for.
Last year I had a take-home exam in an operating systems class that I suspect one student fed entirely as prompts to an LLM, and it was... odd. The answer to every question was a paragraph or two of text, even in cases where the expected answer was true/false, or a number. And even when I did want text as the answer, it was way off, e.g. in one I asked them to explain one strength and one weakness of a specific scheduling algorithm on a given scenario. The submitted answer was just general rambling about scheduling algorithms. Some of this is probably within the reach of an expert using clever prompting strategies, but students who can do that could probably also answer the original question. :-)
To be fair, I have seen the "ramble generically on the subject of the question" strategy manually implemented by humans too, in the hopes that if you throw enough BS at the question you might get partial credit by luck. Maybe designing assessments to be LLM-resistant will have the nice side benefit of reducing the viability of BSing as a strategy.
I used to have students who would write answers like that on in-class exams.
Every answer was at least one full, complete sentence, even for yes/no or true/false. And the “short answer” responses filled all available space when one sentence would do.
My only conclusion is that some undergraduate institutions around the world must be intentionally drilling it into their students to do this.
I suspect it starts in high school. A lot of AP subjects with written portions like AP biology or history are really hard to grade at scale so they have a relatively naive scoring system. The answer can be a total rambling mess but as long as the answer is self consistent (it doesn’t contradict itself) it gets points for any relevant information it gets right.
For example, if the question is about respiration a rambling answer that mentions “oxygen transport chain”, “Krebs cycle”, and “ATP” might get 3/5 points even if it doesn’t make much sense otherwise as long as the answer doesn’t confuse the Calvin and Krebs cycle or otherwise contradict like saying that glucose is a byproduct.
I was told by multiple teachers/professors that its never acceptable to write anything other than a full sentence on a test (unless it's a scantron, obviously). Not sure how common this is, but they could have been trained by other instructors.
I think students also believe they can hedge. If they just put down "yes" or "no" then their answer might be completely wrong, but if they drop a bunch of things in the answer then some of those things might be true and you might give partial credit, or, at least, they can argue about it later.
Its possible. I've had proffesors who always gave true/false questions with instructions to either "justify your answer", or "if false, justify your answer".
Practically speaking, there is fairly little downside to putting in extra in your answer, as tests are normally scored by how many points in the grading rubric you hit.
> To be fair, I have seen the "ramble generically on the subject of the question" strategy manually implemented by humans too, in the hopes that if you throw enough BS at the question you might get partial credit by luck.
This is the basic speech strategy of politicians. Don't answer the question asked, just talk about something related that you want to talk about.
I don't think it'd do well even for an open-ended assignment. The best language models I've seen are still easy to detect as bots if you read multiple paragraphs of output.
> To be fair, I have seen the "ramble generically on the subject of the question" strategy manually implemented by humans too, in the hopes that if you throw enough BS at the question you might get partial credit by luck.
I had a college professor that knew to recognize this and actively warned against it during the mid-term and final.
He said that every question will be answerable in 2 or 3 sentences, and that if you write 2-3 paragraphs instead, he would mark you down even if the answer was correct because you're wasting his time and may have dropped in correct statements that answered the question by luck.
So often in school, we'd be getting quizzes/tests back, and I'd peek over at someone else's paper as it was being handed back and notice they wrote an entire paragraph to answer, whereas I answered it in a single sentence and got full credit for a correct answer, and I was always left wondering what the hell they wrote about.
When my parents were in school, they hand wrote essays and used type writers. Correcting a mistake meant rewriting an entire page! When they needed to research something, this meant spending a day in the library manually searching for quotes/citations. When I was in school I had a rudimentary spellchecker, Microsoft word, and Wikipedia.
Now a grade school student has access to grammarly. In a few years they’ll probably have automated fact checks and text generation.
What will happen? My bet is that we’ll expect a lot more from students a lot earlier.
Evidence is provided, of a sort. The first link goes to a report by a journalist who interviews redditors who claim they are doing this and talk about why.
Reddit is filled with shameless habitual liars who claim to be airline pilots in one thread then plumbers in another. The incentive structure of reddit, the internet point skinner box, incentivizes shameless lying and ""creative writing"".
There are many, many bot posts. More so re-posting other highly-upvoted comments from related threads, or re-posting previously posted pictures/videos/links, but bots are certainly farming karma.
It's hard to be sure. Just as in the (possibly apocryphal) quote from a Yosemite Park Ranger, "There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists" there's considerable overlap between the best text generation bots and the dumbest Redditors.
Those with "karma" systems are particularly susceptible, including HN. But I think Reddit is even worse in this regard than HN because there are many times more users (usernames are less likely to be recognized across threads), and reddit makes it into more of a game with various kinds of flare and other 'rewarding' baubles.
One of those students doesn't mention using GPT to generate essays. The only mention generating lists and other short-response questions. I find that believable.
The other student mentions essay writing, but also says that they "didn't ace the essay" (no mention of the grade).
So, the article linked literally isn't evidence for the claim.
I agree it's not very good evidence that there's a real problem here, the articles and report are more of a good starting point for interesting discussion. On the other hand, the report isn't literally zero evidence either. There are students stating that they're doing this, even if they don't name GPT-3 specifically (does it really matter what model they use?).
> There are students stating that they're doing this
But there literally aren't. There are not students, quoted in that article, stating that they are "acing their homework by turning in machine-generated essays". Literally. There aren't.
I don't doubt that this is possible, in some sense, but the details really matter. Per my original comment:
>>> Pedagogically, this matters. Think about calculator usage. There's a huge difference between allowing use of TI-83 on Calculus assignment with lots of word-heavy application problems and allowing use of Wolfram Alpha on a Calculus assignment that's "integration recipe practice".
What was the assignment? What was the purpose of the assignment? What were the grading standards?
Eg, I have assigned homework that could be completed by a combination of Copilot and GPT-2. That homework was graded on a very coarse rubric. Today, a student could get an A on that assignment using GPT-2 and Copilot. If I were still teaching today I would not worry about it because:
1. they're only cheating themselves
2. they will still fail the course if they don't learn the material
3. it would save very little time to use those tools for these assignments. Maybe 5-10 minutes max, for a total of 5-10 assignments over the course of an entire semester that are collectively worth less than 1% of the final grade. So it's an hour and a negligible portion of their grade that will almost certainly be completely washed out in the curve/adjustments at the end of the semester (I don't do knife's-edge end of course grade assignments -- I identify clear bifurcations in the cohort and assign final letter grades to each bifurcation).
I believe copilot and gpt can do those assignments. I'm also 100% confident that those tools cannot complete -- and can barely even help -- with assignments that actually counted toward student's grades.
So, again, the context matters. Not all assignments are assessments and not all assessments need to be cheat-proof.
Acing a term paper that's 50% of the grade means something.
Acing a paper designed as an opportunity to practice and graded mostly for completion -- but with plenty of detailed feedback in preparation for a term paper -- doesn't really mean anything and really only cheats the student of feedback prior to the summative assessment.
This, btw, is why I'm more interested in what educators are saying than what students are saying. The teacher's intent for the assessment and the grading rubric matter a lot when determining what "getting an A" means. Acing a bulleted list graded for completion is possible with a 1990s Markov chain.
I bet a lot of students -- the laziest ones -- are getting "WTF is this essay even about... did you have a stroke while writing this?!" feedback if they are using LLMs to generate essays whole-cloth.
This comes across as very ill informed. I suggest you actually use some of the AI essay-writing services because they are pretty indistinguishable at this point from human writing.
I was going to play your game, but this product requires a valid email address and phone number, so generating examples from this product and sharing them here without doxing myself to an unknown company requires way too much effort.
Maybe you can help by copy and pasting the first 50 pages of output from that model for the prompt that is shared by a user below:
write a 50 page paper describing the impact of Teutoburg Forest on Roman politics.
In the Czech Republic, and I'm sure this is true in some other countries, many of the exams are oral. The student is given a prompt and they have to stand in front of the teacher(s) and talk about it, giving as much detail as possible.
I think it's a great way to handle exams and I wonder if we'll need to shift towards that in the future.
I agree. But there is a "gold rush" out there in the academic world where the idea is to offload more and more work to "on-line."
My son just started school at a respectable US university.
Two of his five courses are entirely on-line.
A third course (genetics) is done in person, but the exams and quizzes are done on-line.
I overhead three undergraduates talking about how easy it is to defeat the anti-cheating software for their on-line courses ("The professors don't review the videos...Just hold your phone out of view of your laptop camera and google the answers...I've never been caught ...).
It was disheartening and I told my son to avoid on-line classes in the future because he was just putting himself at a disadvantage.
Pretty much confirms what we already know: standards are declining, more unqualified students are attending university because the job market demands it, colleges make bank on tuition and don't want to stop the gravy train. Now online even offers another way for administrators to cut costs while reducing the quality - but no one really cares. Most students want their degree and want to get out now - not blaming them, they're playing the game.
Even a decade ago I was seeing obvious cheating when I was TAing classes. International students would hand in an essay with clearly broken english, then follow that up with essays with complex, well written english that was obviously not in their "voice" or even in the same intellectual ballpark as exhibited in the in-person class and discussions. Even pre-internet, essay writing was widely regarded as " library stenography" by students.
The meat of the matter really comes down to the professors and how they approach the exams. Generalized essay prompts are hilariously easy to cheat on; complex hyper specific prompts that extend on something specifically discussed in the class are far harder. On-line classes, by their "mass distributional" nature (ie, save money by making them reusable) are almost by definition far more generalized than you would want.
> The meat of the matter really comes down to the professors and how they approach the exams.
Until the professor actually asks real questions and grades them as they should be graded. Then the students complain, and the grades are "renormalized" or whatever euphemism they use. And certainly in the US, students have a lot of influence, since they're the paying clients. It's really not surprising that employers ask academic titles for so many entry-level jobs.
> My son just started school at a respectable US university.
> I told my son to avoid on-line classes in the future because he was just putting himself at a disadvantage.
At a disadvantage for what?
For learning the material - sure, when an easy avenue for cheating with obvious incentives is available, it's unreasonable to expect people not to take advantage of it. Also, I expect that teachers have significant incentives to reduce their lecture efforts - why read the room, modify the lecture in response to Q&A, and tailor curriculum to the actual class progress, when you can just push "play" on the recording from last semester?
On-line classes give him an advantage for getting a piece of paper from a respectable US university, however - less work, same piece of paper, same results as far as job eligibility and resume eye-catchers...
I'm sure you enjoy being contrarian, but I also suspect you know precisely what I meant.
In the short-term, yeah, what's the big deal? But in the long-term, this devalues the courses and the degrees.
It's not in the long-term interest of the universities to behave this way because the course they offer are, ostensibly, a way of determining who understands the material and who doesn't.
Also some people have hard time breaking the rules even when they know that everyone else is breaking those rules and they are unlikely to get caught. We probably want to discourage systems that give cheaters advantage over honest people.
I honestly hate oral exams. I had three for my master degree, in english (not my native language), in math-related courses. I underperformed them because of several reasons
- Pressure to actually talk and perform. In a written exam I can stop, look at the paper and take my time to actually start. In an oral exam I have the professor looking at me. Less time to actually think.
- It's harder to communicate orally with complex subjects than in written form. Add to that the extra cognitive load of not dealing with your native tongue.
- Time constraints on oral exams punish students that don't learn everything by memory but can reason in real-time. I didn't know by heart a certain proof, I know for sure I could have done it in a written exam with time to spare (it wouldn't have been the first time) but I wasn't quick enough for an oral one.
- Mental blocks are much harder in oral exams. Some times I blank out and forget something basic (e.g., the derivative of a logarithm). In a written exam I can stop, think, and solve it. In an oral exam, the "stopping" is already making you look like you don't know things, adding pressure that will make you underperform.
Written exams have been the norm for decades for a reason. Far easier to actually ask the things you want, they scale far more easily, easier to correct and grade, and easier for students to practice and perform well.
> harder to communicate orally with complex subjects than in written form
American oration is generally awful. Talent in it command a premium. Perhaps this is related to our lack of oral examination ? We’re more comfortable with written argument because that’s how we’re taught to think?
Well, it was in an Swiss university and I'm Spanish so I'm not sure if American oratorion had anything to do with it.
But, in general, oral communication is harder than written. Written form allows you to take time to form sentences, correct mistakes, reorder thoughts, follow an outline... Oral communication, on the other hand, forces you to think and speak in parallel. It's always going to be harder.
Viva voce was a substantial part of my final in Applied Physics at Exeter Uni. in 1977. It was nerve wracking but it was on a specific subject, my final year project on electron spin resonance, so I wasn't having to invent a long speech on the fly, just defend my conclusions, experimental methods, and analysis against critical but not hostile questioning.
In my opinion such examinations and written finals succeed in filtering out people who can only regurgitate what they have been spoon fed in class rather than being able to use that information to move forward. I think I was lucky to study when and where I did, it stood me in good stead in industry.
Of course there are some people who really do find such examinations difficult. In my opinion the solution is to provide assistance to those people to do better rather than throw the baby out with the bath water.
An oral defense of a project you’ve made is a very different matter from an oral examination of a course, where the expectation is to have far more breadth than depth of knowledge. I’ve had no problem with oral defenses of projects and thesis, but exams are completely different, you’re not really able to prepare at the same level (and you usually have several exams to prepare)
My father who is a college professor in his late 70s is starting to do this in the US as he found less correlation between a written assignment and students actually knowing what the material was. Forcing a presentation makes it harder for them to cheat.
On the other hand, oral exams can be less stressful as long as the examiner is friendly as there can be opportunity to correct mistakes where you do know the subject matter but just had a brainfart. You also get a chance to spend more times / show off on the parts you do know better.
I am very much not an extroverted person and in University written exams were almost always more stressful than oral ones as they try to cram as much as possible into into the time available or were too easy (so at the top end grading becomes very nitpicky) or too hard (which can be demoralizing if no one even manages to complete all questions). Oral exams have much more flexibility to adapt to the individual here.
I think this really depends on your specific university. At my university in Eastern Europe the teachers were bitter and overworked, and that really showed during oral examinations.
Because of the high number students enrolled in the classes, almost all oral exams also featured a written component, very similar to a regular exam. Because of the unstructured nature of oral exams, you would have an arbitrary amount of time to solve it. After that the professor would make you elaborate some of your anwsers, or not, depending how he felt like. The students who performed best were indeed very extroverted and able to convince the professor that they actually meant something other than what they wrote.
The professors also used these exams to give you an arbitrary grade for the subject, depending exclusively on your oral exam "performance". I remember having a high 90% grade in the written part of my Advanced Electronics class. The professor didn't feel I was confident enough in my answers during the oral exam, so he passed me with 1/30 points even though I answered most of the answers correct, thus bringing my grade down to barely a C.
Oral exams sound great in theory but in practice they always felt somewhere between unfair and traumatizing. I much prefer the objectivity of written exams.
> On the other hand, oral exams can be less stressful as long as the examiner is friendly as there can be opportunity to correct mistakes where you do know the subject matter but just had a brainfart. You also get a chance to spend more times / show off on the parts you do know better.
I don't agree with this at all. Even with a friendly examiner, they have far more presence than in a written exam. You are presenting and being actively judged, unlike in a written exam. Also, you'll spend more time on the parts you know worse precisely because you'll be slower, make more mistakes...
> they try to cram as much as possible into into the time available or were too easy (so at the top end grading becomes very nitpicky) or too hard (which can be demoralizing if no one even manages to complete all questions). Oral exams have much more flexibility to adapt to the individual here.
Badly adapted exams are a feature of the teacher, not the type of exam. If anything, oral exams are worse for more complex content, as they tend to be shorter in time.
Just like real life. When schools had more prestige, they also relied more heavily on intensive oral examinations. Some of the most prestigious schools in the world still rely heavily on intense oral examinations.
Universities can also crib from how law school exams work if they want to still do written examinations.
Every school (and many university professors) does "cold calls" in Italy, as far as I know; scheduled oral examinations are the exception, not the rule, before university. Not knowing the answer at a random university lecture doesn't generally impact your grades, however, because in most cases those are 100% based on a final (often oral) exam.
I wouldn't argue that it is a system without issues but it's still better than rote essay writing.
> I don’t think extrovert is necessarily correlated with skilled speaker.
My oldest is quite introverted but can somehow flip a switch internally and is excellent at public speaking. She's won a number of speech competitions and is far better at this sort of thing than I will ever be.
I’m quite introverted and wasn’t at the level that your daughter is, but apparently did well enough at an impromptu speech in college that I had a very cute girl from the class look me up and add me on Facebook.
Oddly enough she married someone from my hometown. The college was 3.5 hours away from that town of ~2,000 and he didn't go to our college. If I ever bump in to one of them I'm going to need to ask how that came about.
I loved the style of book report that we had with a high school English teacher.
While the class was otherwise busy, the teacher called you up to report on your self-chosen-from-a-list book. The teacher opened the book to a page and would read a passage and ask questions about it like what are they talking about, what happened right before and after this. After a couple of passages at different spots, you got your grade.
You are not feigning anyone familiar with the material.
If students are successful at using rhetoric to disguise their ignorance from their teachers, we need to be looking into how teachers are selected, trained and compensated.
On the other hand, extroverted people have similar advantage in the real life. I myself am quite happy for every lesson where I was pushed to practice people-facing skills (presentations, demonstrations, etc). Even an introverted person can learn to talk about topic knowledgeably if they know it -- which often is valuable confidence-building experience to have. Despite the introversion, one can do it!
If the professor - lecturer administering the test is any good, empty rhetoric won't help too much. If they are lazy, students one can try to give "answers" without showing what they don't know in written exams, too.
Does not work with math and engineering. I had a very difficult electrical engineering course and the final was solving a problem on a whiteboard in the teachers office.
Either there is no discussion with the teacher, in which case it might as well be done on paper at a desk; or there is discussion with the teacher, in which case students with more advanced social skills are often able to elicit more help without betraying that they lack knowledge or intelligence.
good. youre right i got through a lot of presentations easier than other classmates but thats cause learning to present really well is an essential skill. students should be skilled at using rhetoric so i see that as a feature not a bug.
everything else in life will reward being well spoken and outgoing, why shouldnt school as well?
School should be testing you for knowledge and ability in a certain subject, as objectively as possible. Unless you're in a public speaking course, your public speaking skills should have as little bearing on your grades as possible.
no. every course should incorporate significant written and oral components because if you cannot adequately synthesize and communicate the information you're meant to be learning, that course is functionally useless. one of the important things i have learned is nobody cares what you know unless you can communicate it well.
Teachers have to prepare their classes because, despite knowing the subject well, actually talking about something requires preparation.
It’s unreasonable to ask students to not only study and understand the material, but also prepare all the course knowledge enough to be able to communicate that well when asked about a random part, and do that for all the classes they might have. If you wanted to give them time to prepare a specific topic it wouldn’t be an exam anymore.
Synthesizing and communicating properly a subject is the work of a teacher. It takes practice, deep knowledge of the subject, extra materials. You’re asking students to both be students and teachers of all the material for a single exam.
no i'm saying there should be a basic level of fluency in the material beyond regurgitation. saying every course should have people able to write a paper on some assigned part of it, or to prepare a presentation, or to take an oral exam with a previous idea of what the material will be is reasonable and essential.
i think you're making this out to be way harder than it actually is, before almost any exam i can discuss pretty well the material behind it. half the time i end up explaining something to a friend who is wondering about something. it's really not that hard.
the only thing that would make it hard is if somebody was bad at speaking and communicating effectively. but just like being unable to write well will hurt your grade in a non-writing course, being unable to speak well should significantly reduce your grade or make you unable to pass.
> no i'm saying there should be a basic level of fluency in the material beyond regurgitation. saying every course should have people able to write a paper on some assigned part of it, or to prepare a presentation, or to take an oral exam with a previous idea of what the material will be is reasonable and essential.
Writing a paper, presenting or taking an oral exam mostly require regurgitation.
> i think you're making this out to be way harder than it actually is, before almost any exam i can discuss pretty well the material behind it. half the time i end up explaining something to a friend who is wondering about something. it's really not that hard.
Explaining something to a friend isn't the same as the teacher selecting a random aspect of the course, and asking you to explain it with a certain depth. Either they can only ask things that are too basic, or they ask complex things in which structuring the content is very hard. Not to mention that asking to explain it is fairly different from knowing how to use it.
To put one of my oral exams as an example, it was on harmonic analysis. Instead of the full course, the teacher had to limit which theorems could be asked in the exam, because covering everything on the course well enough for an oral exam was an impossible task (not just for students, I bet the teacher wouldn't be able to recall every theorem and every condition). My question was explaining a certain covering lemma. Fairly easy and basic. Second part was proving it, and I didn't know that proofs were going to be part of the exam so I hadn't the proof memorized. I started proving it myself, but of course the limited time of an oral exam was not at all enough so the teacher told me to stop.
IMHO, being able to prove the theorem shows deeper knowledge than memorizing the proof. But oral exams aren't a good place for doing that. In fact, most "good exams" I've taken where written exams where you could even bring your own notes, and had different exercises (do all of the basic ones, choose one of the difficult ones) to prove not that you had memorized the material, but that you actually understood it.
taking an oral exam is different from being able to discuss something. you know as well as i do there's a difference between a regurgitation paper and one that requires you apply knowledge to analyze something else.
the kind of questions i'm talking about are application. like for a programming course you might ask what features of a particular language one might explore for solving some vague problem and why. for history pick some current event and ask about its parallels to and lessons for something happening today.
harmonic analysis is probably one of the worse use cases for this. and yeah i agree with open notes exams.
While anyone can go into khan academy and learn/perform any subject, having an actual person would be tremendously costly. It's strange, when computerized education appeared I thought the cost of education would plummet, now its going to go all the way up.
you are looking at 2 different things we both call education but should have separated a long time ago. the cost of learning is almost unbeleivably lower than just 20, 10 years ago. the cost of credentialing might go up.
I had this for some classes in Spanish literature in college and I found it way easier than writing a long winded paper about some dull novel about an overbearing matriarch. My business classes similarly had presentations that were partially peer-graded and I really enjoyed those as well.
At least from my experience being a student in the US, we kind of do that, but it is separate. I had quiet a few assignments where I had to give a presentation of some kind. Where we mess up with this, it usually works out that kids make power points full of information and just read off the slides. Of course, this practice isn't exclusive to students, I've seen many presentations in academia, my time in the military, at technical talks, etc that do just this, read off a power point.
I love your attitude. I have the exact same one. I'm so excited that another form of busywork has been eliminated from school by robots. Eventually they're going to start having to structure classrooms around learning instead of worksheets.
I really don't understand why essays are bad. Is there any kind of research that goes into this?
I agree with you that learning should be the goal. And any busy work that doesn't help should be eliminated. But I just don't know if we know what the right structure should be and if we can say for sure that things like writing essays don't actually help students cultivate their writing and critical thinking skills.
Essays are wonderful. It gives me great joy to write essays today, although they're sometimes called "blog posts" or "rants" or "emails" or "memos".
I hated writing essays in school, because the assignment was always "reproduce a work of writing that adheres to the arbitrary standards of the institution for grading purposes". Great writing can't be graded, as its value is entirely subjective.
As an example, here's an assignment that I might have completed under duress, vs. one that I'd complete voluntarily for fun:
"Explain how the theme of Chaos is expressed in Slaughterhouse Five. Use at least five supporting examples from the text and cite your references MLA style. Four pages minimum."
"Convince your best friend that Slaughterhouse Five is a terrible novel. Cite the text any way you please, ideally by comparing it to a book you think is actually good."
Whatever I produced for the first prompt can be graded by ticking off boxes and looking at my grammar; whatever I produced for the second prompt would need a thorough investigation of my own writing style and a framework of grading that takes into account my own voice as an author.
(To be clear--I don't think that giving my prompt in a modern classroom would immediately inspire students. They are far too burdened by the entire system for a single change to fix their experience. I am merely discussing the difference between "pointless essays" and "essays that authors care about".)
> Whatever I produced for the first prompt can be graded by ticking off boxes and looking at my grammar; whatever I produced for the second prompt would need a thorough investigation of my own writing style and a framework of grading that takes into account my own voice as an author.
The two prompts motivate the writer to practice two completely different skillsets; they're really not comparable.
The first one is focused more on the fundamentals of close reading and analysis. The writer needs to understand what the theme of "chaos" means, then closely read the novel or review their notes to identify literary devices or techniques that theme, and then tie it together in a "report". It requires the assignee to practice very basic skills... it's technical practice, not artistry.
The second prompt is the artistry - it's an assignment in discourse or rhetoric. The thing is, it's not possible to successfully execute the second prompt unless you've mastered the techniques from the first prompt. Beyond constructing logical or emotional arguments that may be tailored to your audience (your best friend), you still have to collect evidence from the novel. It might not be a list of literary devices, but if one of your arguments was that the book was poorly and confusingly written, you would still probably need to collect evidence of specific passages that support your claim. The whole point of the first prompt is to build the skill to do this, but with some hand-holding/constraints for practice.
I won't defend page limits, but even the reference style mandate is important because it has implications for how you actually write the essay. I deal with technical stakeholders all the time, and the amount of time that we could clear up issues if someone would just properly cite a reference can be ridiculous... perhaps those stakeholders were the teenagers who didn't bother to follow the citation guidelines for their literature class?
This really got me thinking, so here's another comparison to draw on:
As a teenage musician, I hated drilling my scales and etudes. Why bother when practice was limited and I had cool ensemble and solo rep to learn? What I didn't understand and appreciate at the time is that all the technical drudgery serves a very real purpose. Most of the existing pedagogy is directly pulled from, based on, or references real repertoire which you'll undoubtedly encounter in your musical career.
All those scales in intervals? Well, you can't even begin to make a complex passage musical if you can't execute the technique! Arpeggios in weird fingering/shifting patterns? Turns out that some very exposed orchestral passage necessitates that you use an oddball fingering because it's just not practical to do anything else in context. That entire development section in the concerto you need to cram for an audition? Good thing that one of your etudes book was effectively variations and embellishments on that section, so you can lean on muscle memory and focus on making it sound nice!
Essay writing is much the same. No matter what I'm writing - an e-mail, a project proposal, a performance review, whatever - I'm trying to communicate a point. That means constructing an argument and supplying evidence. And doing so in a way that your audience will grok without any additional intervention. You build this skill by practicing, sometimes in ways that seem dumb, boring, and disconnected from reality. Not every pedagogy is ground so well in reality as my music example, but I can't imagine that the cynical take that it's all purely to automate grading is a rational take on things.
I've jumped back into doodling on guitar after a mid-20's post band break up and... it's totally different this time. I am so much more interested in scales, building chords by manually and so forth, when I was younger I wanted to play songs.
The opposite is true for my math. I enjoyed algebra as a kid and hated trig and calculus. Now I am much more interested in calculus and don't like algebra algebra.
Sometimes there's different ways to learning, I have no idea.
> That means constructing an argument and supplying evidence.
Ah wouldn't it be fantastic if school essays were more like proving to your boss that you followed the spec to the letter and less like... Following the spec to the letter.
That's the difference and it makes all the difference.
What about convincing your colleagues that something about the spec is wrong?
It's worth noting that the same skills the "version 1" essay is supposed to teach should be helpful if all you need to do is compile a checklist and save yourself the hassle of argument.
> The first one is focused more on the fundamentals of close reading and analysis. The writer needs to understand what the theme of "chaos" means ...
I think what ends up happening in reality, at least, in my experience, is that you Google "Slaughterhouse Five chaos" and trawl the first several pages of results looking for information you can essentially copy+paste into your essay (with slight adjustments to get around automatic plagiarism scanners, of course).
I did still demonstrate some kind of skill, maybe research and the ability to condense information from many sources down into a single piece of work, but those weren't the skills you mention, and it was definitely not what the teacher was intending for me to do.
The second prompt the person you responded to runs into the same issues (I can Google "Slaughterhouse Five reviews"), but at the very least probably feels like a more engaging and compelling essay prompt to the student.
You're generalizing in a way that probably suits you and people like you. Not everyone though.
The second prompt would have sent me spinning, panic, want to run.
The first prompt, while being 'technical' and not what a future 'writer' would like to do at that point can be somewhat mechanically achieved and while I still wouldn't have liked it, I would begrudgingly do it and it probably helped me overall. It mentions using certain 'techniques' you would've learned about in class. I can apply that. They want a specific number of pages at minimum so that I don't just write 5 sentences to cover the 5 examples, sure, whatever.
Like learning math. You gotta learn the basics, learn the multiplication tables by heart. Do the same "compute (-7^2*13-7)+5/5" style exercises over and over. It teaches attention to detail and memorizing and following simple rules. If you can't do that it is very unlikely that a "closer to reality" question that someone that will later go on to become a mathematician would like working on instead would not send you into panic mode.
I disagree on the ability to execute the second prompt.
You would like them to execute the second prompt in a way that demonstrates the skills that the first prompt calls for. They won't. They'll just take the second prompt, and communicate exactly the same way that they already do to their friends, with similar skills and language. The result may be persuasive - particularly to their friends - but it won't develop analytic skills.
This is a classic pitfall faced by novice teachers. I fell for it too.
Assign the second prompt, and I guarantee you’ll get something like this as a submission:
“Bro, the novel sucks. Trust me.”
You can’t even give this a bad grade, based on the prompt. You can't say it’s not convincing, because they’ll say “you’re not my friend, this would convince my friend”
You can’t say it’s too short, because they’ll say you didn’t provide a minimum.
You can’t say it didn’t cite the novel, because you said to do whatever.
You can’t say it didn’t compare to other literature, because you said “ideally”.
Lesson 1 of being a teacher: give the students an inch and they will take a mile.
Teaching students is not unlike programming computers, in that they both take instructions very literally. If you are vague with a computer program, you know ahead of time because the program doesn’t compile.
If you are vague with an assignment you don’t know until you get it back. The more vague the assignment, the wider the variety of submissions. If you don’t tell them the font face you get a cursive one. If you don’t tell them the font size you get huge and tiny. If you don’t tell them the margins you get wide and thin.
So even if you would personally make a good faith effort at this assignment, it’s really better for everyone to be specific and follow the same format.
I covered this point in my last paragraph. The problem with students isn't that they can't follow directions or collaborate; it's that the hate school and don't want to be there.
One of the reasons they hate school and don't want to be there is that they are compelled to do pointless, grinding busywork, all day, every day. That's why they're using GPT-3 to fake their essays. Even three hours of reprieve from the system is worth cheating and dishonesty, and all the better if it helps their GPA.
I harbor no beliefs that a teacher can walk into the modern school system with a creative, exciting lesson plan and inspire students to perform. The system is broken and fundamentally flawed. It cannot be fixed. You are certainly correct that the best way to get consistent results out of your institutionalized students is to grade to a rigorous, clear format, but in doing so you've only played your part in reinforcing the exact system that drives them to cheat with GPT-3.
> The problem with students isn't that they can't follow directions or collaborate; it's that the hate school and don't want to be there.
I'd say it's a mix of both.
> That's why they're using GPT-3 to fake their essays.
I'm not actually sure about the motivation for most students. For students who I've caught using copilot on assignments, it's not because of the reasons you cite. Maybe it is for others.
> It cannot be fixed.
The main improvement that would fix most of this is to have higher teacher to student ratios. That alone would be a massive improvement, because then teachers would have time to engage students at a different level of attention.
Although the general gist of what you're replying to certainly evokes a response in me, I was not going to reply until I read your response and agreed with it's point.
Having taught writing in universities over a six-year-long stint, my experience agrees with yours.
Pragmatically, the reasons the assignments are structured they way they are isn't because bad faith by instructors, but rather because of the needs of students.
I don't blame the students-- they have a lot of shit going on.
At the same time, you're absolutely correct that making assignments in the general form we see them has more to do with what students actively demand: they absolutely do not want the kind of assignment suggested by the GP because anything other than a list of boxes to check causes profound anxiety in students.
Our comment threads here are excellent examples of what short writing prompts and assessments could look like, and I've gotten invaluable feedback on my writing from participating in internet threads. In this form of writing, there are distinct grades in the form of karma. And there are real stakes for communication, as I can easily fail to get my points across or even upset people. I even sometimes get useful responses that improve my understanding of the world or some topic.
As useful as that practice can be, if I had my academic advancement tied to these prompts it would cause me a great deal of stress: how the hell do I know in which contexts someone will read any given post?
In the context of the general internet I have a lot of easy ways out. I don't have to listen to dumb people, the poorly informed, or malicious trolls.
In the context of a classroom, I can't just tell the teacher "that's just, like, your opinion, man" because they are going to write down a letter and that's going to make my life easier or harder.
I'm not a big fan of contemporary education for reasons I could develop in book-length diatribes (I quite a PhD during my dissertation), but I get where students are coming from when they demand some clarity on how they are being assessed.
Note the section on Body paragraph structure -- that doesn't begin to cover how structured the resulting essays were. I can still remember my daughter sing-songing "T, CD CM CM, CD CM CM, CD CM CM, SC." Every essay had to follow that exact pattern. Every commentary sentence had to start with the approved list of words, and likewise the concluding sentence.
It's entirely about how easy it is for the teacher to grade, and has nothing to do with teaching students to actually write. It was awful, and I did everything I could, including contacting the Jane Shaffer people, to criticize it and push back.
Good lord, that is even worse than I'm describing.
A great many people cannot fathom the fundamental truth that the majority of learning is useful simply because it allows us to enjoy to process of living in society for 70-90 years and for no other reason. There isn't any greater purpose to being "good at writing" other than "it makes communication and competition more accessible and convenient", but even that is a subjective value judgement. There are many sub-cultures who exist even inside our own that are perfectly happy being sub-literate.
A lot of people are saying "if kids don't learn dry, boring technical writing, then how will they write work emails and documentation?" The implication there is that if you can't write proper emails and documentation, you'll fail at your career. If the emails and documentation I receive are at all representative of greater industry (and I've worked in both aerospace and clinical research) then I can assure everyone that few people can write at a useful level and not only are they still employed but their companies are still around.
Given that no one can write anyway, why do we cling to forcing children through painful, humiliating exercises in standardization?
These essays are usually assigned in college-prep courses with the intention that the students will go to college where they will need to read/write in APA, MLA or other rigid, clear styles that would allow them to read/publish in specific academic journals. It's a high standard for the high-achieving students.
> Great writing can't be graded, as its value is entirely subjective.
Well... I have a lot of problems with this, on both sides of the fence.
Provocative start: How about we give up the idea that students are producing great work?
I mean, I imagine teachers think of the exercises they give as skills development.
One of the skills to develop is technical writing.
Surely beginning students do not know how to cite textual examples to back up their arguments. Understanding the role of evidence in making an argument should be fundamental to democracy. (Understanding that we are not living in that world recently in the US.) So they need to practice.
Choosing the theme is another skill. I don't have anything to say about it, but I don't have a problem with teachers asking students to try to figure something out before they write about it.
> How about we give up the idea that students are producing great work?
Almost all teachers are well ahead of you on this one. Far rarer is the belief that students are capable of great work. This is, in fact, my central point: if all of your assignments are bounded by the need for administrative convenience, creativity and originality cannot flourish.
Teaching writing through dry, separate "skills development" exercises is like teaching basketball as follows:
"Today we'll practice jumping from one ankle to the other. Today we'll practice reading a point guard centric offense. Today we'll be working on our vertical jump height. Now for the exam: demonstrate a cut behind the center and a layup. Hmm, your second step is slow, you get a C."
I am arguing that if you want to teach basketball, your students need to play a lot of basketball. Exercises will only really help them once they've experienced the game and have a burning internal desire to compete.
> I am arguing that if you want to teach basketball, your students need to play a lot of basketball. Exercises will only really help them once they've experienced the game and have a burning internal desire to compete.
Well. I played basketball since 7th grade. Not just played, trained 3 days a week. Before you can really play you have to master certain elements, otherwise it's just fooling around. And at first we trained all those elements separately. Balance, switching feet, turning. Just turning without the ball. Faster, slower. Jumping from left foot, from right. Catching the ball. Throwing it. Passing. Alone, with partner, against the wall. Hook shot, but just up, up, up, get the ball up. Hook from left, from right. Then hook standing directly below hoop. Left, right, left, again. Then adding movement. Over and over again. And then actually playing.
I play tennis, soccer, ultimate frisbee, jiu-jitsu, disc golf, rock climbing, hang gliding, whitewater kayaking, bicycling, and kite surfing, and I've taught many of the same to beginners.
I assure you that the instruction only works when the student wants to experience the final form, and they will not get any sense of what the final form is until they have "fooled around" and have an actual desire to learn the sport. At every level of their progression they need time to experience unstructured performance for fun. The same is true for writing.
Kids love to write when they're young and no one's hovering over their shoulders grading them. It's only once they get hit with the five-paragraph essays and the term papers and the dry grammar exercises that they learn to avoid writing, and associate it with boredom and stress.
Leaving aside the fact that your two examples assess a pretty different set of skills, one has to deal with the reality that every teacher has many students. It's simply not realistic to expect them to do "a thorough investigation" of every student's style and "voice." Imagine you have 50 students across two classes, each turning in one of these essays. How long are you going to spend on each one? 5 minutes? That's 4 hours of grading time. 15 minutes? 12 hours. Now consider that most of your school day is already occupied with teaching, prepping for classes, office hours, and other responsibilities.
I'll go further and ask "what if their 'voice' is just bad"? Just because I have a 'voice' doesn't mean it's necessarily good. Teaching students to be able to switch voices - some voices/styles are more appropriate for some types of communications than others. Recognizing these types, and being able to switch... that seems like it would be a thing to teach/learn. My early schooling was a long time ago, and I don't think I had the language to categorize all of this at that time, but I do have memories of doing this sort of stuff (mostly grades 6-8 where I had the same writing teacher, but later with various classes through grade 12).
I haven't been a teacher, but I have been a TA (while a grad student with a full course load) with a substantial weekly grading burden (e.g. 30 students' problem sets a week, with 10-15 problems each, for a class introducing concepts like formal proofs and basic number theory).
So I appreciate your point.
But I also remember being a student forced to churn out mindless formulaic essays with length and structure requirements. I hated it. I never liked writing until I finally had one good English teacher in high school who assigned and graded in the way you say is infeasible.
If a teacher doesn't have the bandwidth/capacity/skill/etc. to teach English well, maybe they should find something else to do instead of torturing students with mind-numbing assignments.
> 50 students across two classes, each turning in one of these essays. How long are you going to spend on each one? 5 minutes? That's 4 hours of grading time. 15 minutes? 12 hours.
Teachers with fifty students shouldn’t be assigning essays. There is no way for them to read them, which means they’ll grade by scanning for key words. That destroys the pedagogical value of an essay, this post’s point.
> the problem isn't so much with the assignment, but with the idea that assignments only have value if they are graded
There's grading and evaluating. Writing something you know won't be read, except for the purpose of being scolded for missing key words, is close to useless pedagogically. Someone motivated enough to learn from that (a) didn't need the assignment and (b) deserves better.
This! People learn differently - which IMHO schools don't usually account for - but I personally always learned best when putting something to paper (well, preferably the keyboard).
At my university, assignments were primarily used for guided learning - most of the grade came from the exam. If you cheat on the learning, you either don't _need_ to learn to pass the exam (meaning you should have a way to fast track), or you're asking to fail the exam, which hurts no one but yourself.
Maybe it's different in other schools? Cause I don't fully get the "Good." argument based on my experience. YMMV.
Sure— It's not at all tenable right now for teachers to provide in-depth critique on long essay assignments— that doesn't make critiques with avoiding, it makes long essay assignments worth avoiding.
I took a very difficult gatekeeper exposition class at a famously rigorous university a few years ago and loved it. We had to write a ton, but I didn't mind it because when you're learning to write, you need to write a ton. And boy did we. But not all classes there were like that! Some classes, mostly classes about writing were deemed "writing intensive," but others would require little more than a few pages here and there. The standard for that scant output extremely high and the intellectual critique was often blistering; the teacher concentrated on the subject matter instead of combing 50 paragraphs for split infinitive.
Currently, I attend a significantly less rigorous university as a full-time undergrad. I have 5 classes, including an elective on the history of a particular art form. The final will be a 10 page paper and 20 minute presentation preceded by a 2 page proposal. While this class requires significantly less written output than the exposition class, the assignment will still take an disproportionate amount of my time. The teacher has many students and no TA, so each paper will receive a cursory intellectual critique, but primarily graded on format and grammar. I'll not likely have learned more than if I'd written a really tight 2/3 page paper that got several serious critiques along the way.
All acts of deliberate writing can be examined for clarity, concision, fulfillment of their own purpose, etc. If the purpose of an essay is to teach writing, then the writing should be inspired and flow freely. It's only after someone can read and write competently that it's important for them to learn mold their writing for specific, dry purposes.
The second half of your argument is incredibly common, although I don't begrudge you for making it. Yes, it's true that teaching effectively and creatively is near-impossible given the current setup and demands of the modern education system. This should tell most people something about the worthiness of the modern system, but instead most of them defend it.
In that case, we can further automate this process and just leave humans out of it - pipe the student's generated essay back in, emit grade, done.
This reminds me of a moment that has stuck with me for a long time. Some time in the early 00s, I was wondering around town with a friend fairly late at night. We watched a waste truck picking up outside a building, there were stacks of Yellow Pages piled up, as they had just been delivered everywhere, like they used to.
My friend and I joked that they could have saved on transport and fuel by backing up the recycling trucks directly to the printing presses.
Reminds me of phone trees with a robotic voice telling to to press 1 for X, 2 for Y etc. When Google announced their automated phone tree handling, I thought, it's just two robots talking to each other over an imperfect medium, human language. Why not just connect the two systems together via API or something?
Those type of assignments are part of what "inspired" my daughter to skip out on all her high school English homework, forget the humanities, and study engineering in college. On the one hand, great, we need more women in engineering, and she's good at it (and graduated from a top college). On the other hand, the humanities are great too and it would be nice not to turn people off to them with that type of mind-numbing work that is not only subjective in content but also subject to the desires of the professor.
7th graders are highly unlikely to produce writing of such a great quality that it "cannot be graded". Maybe when Hemingway was in the seventh grade. And we can grade subjective things all the time. You can grade code based on something other than whether it runs efficiently, for instance.
Meanwhile, your prompt sounds like hell. And is far more subjective than the previous one.
> Convince your best friend that Slaughterhouse Five is a terrible novel.
So, before you even start the assignment, you let the teachers dictate the position that someone has? And you're not going to teach students to assess themes in books, so what will they judge Slaughterhouse Five on, aesthetics?
> Cite the text any way you please
Why on earth would you change that requirement. "Cite the text using method X" is a direct analog to "Coding standards dictate this naming convention". I would fire a "free thinker" who refused to adhere to the, sometimes arbitrary, standards for communication with the rest of group. Standards are good.
> ideally by comparing it to a book you think is actually good."
As a rule, I don't think convincing someone that a work of art is "terrible" should be done by comparing it to something else you "think is good".
> [No page limit]
You can trust an adult with that, but a seventh grader? Usually they need a page limit to encourage them to write more.
It seems like the following a complete essay that you would have to grade very well: "Slaughterhouse Five's lack of elves makes it terrible, because fantasy novels are just better and books like the Lord of the Rings have elves which makes it a good book [Source - My conversation with Johnny yesterday]"
Grading that well would be bad because it's horrible in every way.
As a teacher of English as a second language, I find the Jane Schaffer method quite helpful in teaching structure, idea generation, and the skill of supporting statements by providing examples.
https://en.wikipedia.org/wiki/Schaffer_method
Of course all essays shouldn't be written exactly this way, but for students just learning about writing long form, it is a brilliant stepping stone to get the basics down. Too many students—and readers—can't differentiate between a "concrete detail" or statement of fact, and "commentary material" or statement of opinion. This method helps to distinguish them.
When challenged on why we had to follow rules that real writers often ignored, such as those "arbitrary standards" you hate, he responded: "They can break the rules because they already know them by heart. You need to learn them." That stuck with me. I firmly believe that writing essays about boring drudgery is a necessity to develop the skill required to write essays about what's important to you.
School is not about doing great work. It's about learning the tools which you can use to do great work. School does not ask you to do novel research, and so it doesn't ask you to write novel essays.
> School is not about doing great work. It's about learning the tools which you can use to do great work.
We have no compatible ideas if you insist upon this being the case. School is not "training for real life", it is real life.
The thought that it's appropriate and desirable to monopolize the bulk of the time, mental health, and attention of young people in order to keep them from attempting anything of consequence while they "prepare to do more work later" is deleterious to society.
(It is also not true that real writers know the rules by heart and choose to break them for their own effect. That implies that there is a real set of rules that people agree on and that every great writer is capable of producing a standardized set of writing that follows these rules. In fact, most great writers take great pains to tell personal stories of failure in school due to an inability and unwillingness to comply with their teachers, and this has been true since antiquity.)
I think you had a different experience. My experience with essays most typically was to choose your topic, research and write an essay. The standardized tests did usually ask you to choose from one of three topics, none of which was usually something you were enamored with, but that did force you to change from your comfort zone and adapt to situational necessity.
My favorite were "reproduce a report on these specific historical facts without plagiarizing the textbook," and before the internet was readily available. More a creative writing course than social studies.
Your prompt doesn't seem to actually allow for any critical thinking development or give any guidance to how you can more convincingly express your points or arguments. It also assumes you didn't enjoy the book, which I'm sure some students would have actually enjoyed, so now your prompt is even asking them to argue in bad faith, that doesn't seem to be something we want to foster honestly.
The first prompt requires the reader to critically analyze the book, by first requiring them to give it a charitable interpretation.
It is said that you cannot disagree with someone if you're unable to explain their position yourself in a clear and definitive manner. Obviously, what are you disagreeing with if you don't even understand what's the ideas behind the thing you disagree with.
That's what the first prompt would be about teaching you, to be able to understand other people's ideas and concepts, to look past your initial judgements and bias, give it a charitable interpretation, demonstrate you understood all this by summarizing the idea in a 4 page essay of your own, with supporting references to tie it back to the source, showing the source does in fact argue for these itself.
Once you can do that, you have gained the right to go on with your own disagreement and write that essay, which would be your second prompt. Though honestly, your second prompt seems to be geared more towards discussing the entertainment aspect of the book, and not the ideas and concepts it contains, so again it's not that much about critical thinking, because critically there's little to argue about a "I prefer the color red over blue."
Personally I think you were trying to get at something else, maybe your point was just, come up with assignments students enjoy and can have fun with?
I think this is always true, but some things are just boring to some students, maybe you just don't enjoy reading, writing or even critical thinking, or any of that stuff. I don't know if there's much you can do in that situation. Maybe the solution is more fluid classes, let people move at their own pace, pick their own areas of interests, even if that's directly going to a trade, skipping on literature entirely, etc.
The first prompt presumes to pick what was important about the book, mandate the form by which the book will be analyzed, and set up a minimum amount of effort before the writer can quit.
The second prompt picks a very general bit of opinion and then demands an open-ended argument requiring original thought. In fact, it's even better if the reader liked the book, because it forces them to write as if they didn't, and opens them to the possibility of a satirical essay.
(Note that I am aware that an average modern student wouldn't like the second prompt any more than the first, but that has to more with the system than the prompt. I'm speaking about the pure act of teaching an interesting writer to write well.)
Critical thinking requires both the desire and ability to think outside of frameworks that were predetermined by authority. This is part of the reason that modern schools are so bad at "teaching critical thinking skills". The most basic form of critical thinking, in fact the first openly critical thought that students have about learning--"this is a waste of my time"--is suppressed for the convenience of the administration.
> Maybe the solution is more fluid classes, let people move at their own pace, pick their own areas of interests, even if that's directly going to a trade, etc.
I could not agree with this more. I believe that 'school' should be life-long, year-round, and optional. Ideally we'd go in and out of some type of formal education until we died. However, this level of societal flexibility is directly incompatible with modern school.
I'll go a step further and defend "busywork". Unfortunately we don't live in a perfectly efficient society so being able to learn and execute a formulaic task that might not seem valuable is a necessary skill for most adults. Especially since individual actors may not have all the necessary information to fully assess the value of a task within a larger framework.
It's an interesting question. I'd personally love to see more effort being given to "defend your point as succinctly as possible." I wrote a lot of "minimum word count essays and in the "real world" clarity trumps length every day.
Length is used as some proxy for rigor, but we know it's only a proxy.
I like this idea. Fwiw I feel I learned / grew a lot more in one technical writing class in college than I did in all my years of English classes. Later I picked ip a book “writing with style” that was also quite helpful.
One issue for me in English was I really was not interested in the kinds of essays the English teachers were interested in having us write, eg coming up with a thesis on plot themes in Shakespeare. Just not my thing and so I couldn’t get anywhere in those classes.
I found that in elementary or secondary school page minimums were used to as a cheap proxy for effort. In college page maximums started appearing to encourage concision. Different needs for students at different points in their education.
My high school English teacher adamantly refused to read any essay over the length of one page, double-spaced. I learned more from that class than any other writing class I took.
When I write today, for work, the challenge is always to write less. My VP might have time to read two pages; he's almost certainly not going to read anything longer.
I learned from one of the comments on my original post that many scholars have been saying this for a while, and that there's in fact a book that makes the same point!
The American educational-system essay is an artificial construct designed to be easy to teach and easy to grade, without being useful for the purposes of educating the essay author or the audience or being persuasive.
Writing is generally good. Expressing yourself is good. Analysis and critique are good. The artificial essay is useless.
Especially the 5-paragraph essay. Doubly so where paragraphs are rigidly-defined as "a collection of five or more sentences". I argued with a lot of teachers over that one.
All it did was teach kids how to write boring simple sentences to meet the implicit "punctuation quota". An equal amount of content compacted into a couple of complex/compound sentences would actually result in marks off.
The 5-paragraph essay is the larval form of the 6-page research paper. As a scientist, you will similarly face reprimands from reviewers if you deviate from the format.
Most people will encounter venues where they need to conform to arbitrary and seemingly pointless formats in order to publish there. Becoming comfortable doing so without throwing a fit is a skill that needs to be learned to operate in modern society.
I didn't mean to say it's you throwing a fit, I meant to say that students are throwing fits. Unless you're a student? Sorry I didn't convey that clearly.
I wouldn't say that the purpose of the 5-paragaph essay is to teach clear writing. It serves the same purpose as mandating a format for a conference; no format mandate means you get N formats, which makes evaluating them much harder.
You have to learn the rules before you break the rules. The 5 paragraph essay taught me a lot about how to create a convincing written argument. The rules are:
1. Start with a hook. Engage the reader.
2. Present the ideas (2-3) that you will be discussing in the essay. Have you ever read a rambling blog post? Yeah, they suck. There's no direction and you don't retain much.
Also, 2-3 ideas in one essay is a great number. There's all those studies that say we can only hold 5 things in working memory at once, blah blah blah. Keeping the essay focused on a few core ideas helps the reader retain them better, and the writer to have a well defined scope.
3. Extrapolate the concise ideas in 2-3 concise paragraphs. Ever read a rambling blog post? Yeah, they suck. Telling students to keep paragraphs in 3-5 sentences helps the essay communicate the ideas in a concise manner.
It also helps block out the text in small visually appealing blocks. Ever read super condensed very long paragraphs? Yeah, they suck. It helps to break up your thoughts with some whitespace. (It's almost like coding benefits from this as well...)
4. Conclude your essay. Reiterate what you wanted to cover. This helps the reader retain the ideas, and it allows the author to tie up the ideas in a nice bow. I love when I finish a book or essay and everything comes together and reaffirms what I've been reading the whole time.
This format is not only great building blocks, but it helps you write larger volumes. If you repeat these small steps several times, you create chapters. If you repeat these steps on a macro level, the chapters tie together into a cohesive piece of literature.
These unnecessary "quotas" may sound meaningless, but a lot of people have thought very hard about how to create basic building blocks writers can follow. These building blocks allow the writers to create concise, well formed arguments. "Boring simple sentences" are extremely conducive to clear and concise writing. I'll take boring sentences that form complex ideas over complex fluff that describes nothing any day. (This is almost analogous to good code design too, weird...)
I get the goal of the framework, and I did then as well. It just isn't expressed well, and it's even worse in execution. I think kids are smart enough to handle a more ambiguous "five-section essay". All the same rules apply, but marks off can then be directed towards lack of clarity or belaboring a point. Truly bad writing, y'know?
Maybe it's a matter of opinion, but I believe that the grading process is a sufficient guardrail. When the theories are made rigid, it's usually just to ease the grading. I'm okay with that in the right context. In this instance, however, I think it's self-defeating.
When I taught a class where I assigned essays I confirmed my own suspicions from when I was assigned essays: the majority of the sentences committed to paper are awful drudgery. I then flipped the requirements on their head and removed minimum page requirements and instead had extremely strict maximum page requirements (with of course the objective material requirements of the essay itself). So much improvement for everyone involved. I had to read through so much less pointless material and the students were forced to focus their ideas in a succinct way to be able to get to all of the objectives of the essay in the limited space. Everyone saved time and did better.
When I taught a class, I had a strict 3 page maximum on lab reports for this reason. So many people are used to turning in 10's of pages of drudgery. I just want a few that get to the heart of the issue.
Being a concert violinist is fun but playing scales isn't fun. I can't wait for the robots to learn how to play scales for me so I can focus on becoming a concert violinist.
Writing essays is not the problem, grading the essays and grades in general are the problem.
Feedback is obviously invaluable, but the point of grades, as used today, is solely to gatekeep who are allowed access to the next level of education. So instead of constructive feedback, it has become a set of filters entirely divorced from actual learning.
(And yes, I got good grades, I just hate that so many people I know were denied opportunities based on a shitty system, wildly not fit for purpose)
I think you’re going at it backwards. If you are going to ask a student to write an essay, it’s on you to be able to show that it’s the best way to help that particular student cultivate their writing and critical thinking skills.
Perhaps I'm just... thick, but if a goal is to have someone write, say, 8 pages of thinking and ideas about a topic... I'm not sure there's 'better' ways than to have the person write 8 pages of thinking and ideas about a topic.
If the goal is wrong, perhaps just don't do it, but... "it's on you to show that it's the best way". I don't get it.
If I want to see that a student has writing skills, I would think expecting them to write is somewhat definitional?
Maybe it's on someone else to 'show' a better way to demonstrate writing skills that doesn't involve writing.
This is a fine goal. Having “someone write, say, 8 pages” is not, it’s a task, and a tedious one at that. No good writer starts with a page goal. It’s a common criticism by great writer’s of bad publishers.
Personally, my experience with page count assignments were... loose. The goal was never "write 8 pages" but "write about idea ABC" (book just read, subject we just studied). '5 pages' or 8 or whatever was a guideline, with the expectation that to get XYZ ideas across, it'll probably be around that length. If the guideline was X words or Y pages, and I got the ideas across in less (or more), but the ideas were strong, I still got a good grade. Perhaps some of that has changed, but... "write an essay about $foo", and you turn in 2 paragraphs... you'd get marked down.
Again, it's been awhile since I've been in middle/high school, so it may have changed some.
These days it’s often a word count rather than page count because page counts are too easy to game. There will be an upper and lower limit and when you submit the file, it might be rejected if your paper doesn’t meet the criteria.
The writing part isn't necessarily being questioned, but the form (essay) is. The tweet thread pointed to in the article has some pretty good suggestions.
The twitter thread is suggesting prompts for essays that an AI can't handle. So the form hasn't changed, they're just trying to stay ahead of the threat.
I was thinking like you until the day I realized that a lot of learning actually implies busywork, like being good at a sport implies a lot of busywork like exercise and nutrition.
Busywork in sports and nutrition are all in obvious service of a more important goal (increase performance in sports). There's a clear incentive and reward to your performance.
The same is not true for busywork in school. It is disconnected from life outside of school in a very fundamental way, in that there are no inherent rewards for being able to complete busywork--all of the rewards come from social cleverness, competitive instincts, networking, etc. If you behave like a good student in the professional world--keeping your head down, getting your work done on time, following all the rules--your friends who go out drinking with your boss will blow past you in your career, and you'll be pigeonholed as a drudge-work guy.
People treat school as if it's training for real life. But it is real life. Why is everyone so completely convinced that it's got to be boring, miserable, and institutionalized?
Most jobs in the real world don’t benefit from social cleverness. They don’t even offer the opportunity to schmooze with the boss. They are: “Drive this exact route and deliver boxes, within a time precision of plus or minus 30 seconds.” and “Repeat this physical motion on this part 7000 times while keeping yield over 99%.” and “Ensure TPS reports are filled out exactly using the correct template and are on Lumbergh’s desk every Friday morning 8:30.” I would argue the repetition, rule-following and drudgery of school adequately prepares workers for this adulthood.
First, I don't believe you're correct about social cleverness not helping drivers, line workers, or office drones. If anything, the more basic the work, the bigger the rewards for social cleverness are, as you use your relationship with your superiors, peers, and inferiors to influence the work day. (This could be becoming less true with fully metrics-driven establishments like Amazon warehouses.)
Second, is that society (where a huge number of people are consigned to that type of labor) a society that you're interested in continuing to build? If so, continue to support the methods used in modern school.
In order to do that, we would have to be able to measure “learning”, and so far, we have only been able to measure “ability to regurgitate facts”. It does not help that the mainstream educational paradigm is “to fill the blank slate that is the student”.
See: John Taylor Gatto’s book on this subject— The Underground History of American Education.
There are other educational paradigms, but they are radically different — no tests, no curriculum, no “fill the blank slate”, no measuring.
I knew I'd see Gatto come up eventually. I owe my life to that man. Before and after reading his work is like before and after the Wizard of Oz switches to color.
That is measuring from the outside, and quantifying it. How do you measure “skill”? You still look at the results, rather than the skill that develops inside a person. It’s still the same problem as measuring “learning”.
Maybe, but the flip side is that I'm wondering how many schools will look at the opposite and try to start using machine learning to grade essays as well.
So we end up with machines grading machines, while pupils and teachers are left doing... something. Isn't it known fact that one cannot fix society with technology?
When I was in school, I had the mindset that I wasn't going to discover new insights about very common historical events or writings. I was given a topic like "write a 50 page paper describing the impact of Teutoburg Forest on Roman politics" and rather than try for new insight, I searched the web and found a couple dozen sources. I copy pasted those sources and fragments into a giant word document. I then organized them into what would roughly make sense in terms of timeline and insights. Lastly, I went through and paraphrased every single sentence from start to finish and added in an introductory and conclusion/summary paragraph. The end result was 100% unique (as measured by computer plagiarism software) and netted me an A+. The professor was dumfounded as to how I came up with a 100% original 50 page paper on such a common topic.
Having ML write your paper is obviously cheating but isn't too far from what I did mentally. I think the valuable thing being lost is the ability to research a topic, quickly understand what's relevant, and structure the results.
In my mind, what you did was actually the goal of the paper. You researched the prevailing theories of the "impact of Teutoburg Forest on Roman politics" and summarized them in your own words. You probably have a much better understanding of the subject than if an AI did it for you though.
To be fair though, that sounds... mostly legit, what you did that is.
Ie in many cases in school i don't think they actually care to teach you the subject. What good is that subject really going to have on your life going forward? Even if the sense of ethics or history-repeating i suspect it's quite low.
What those meaningless tasks can do (though highly dependent on the person, i assume) is teach you how to research, how to communicate, and how to cite.
At least that was always my takeaway from many subjects in school that were subjective (ie not math, science, etc). The content is often the least important aspect of many early learnings. .. but then again i loathed school growing up, so who am i to speak on the subject lol.
There's a neat trick I've seen teachers do to address the concern that regurgitation isn't learning (and it sort of mirrors the technique we use in machine learning): ask a follow-up question based on the essay.
A student that has done the underpinning research to generate the essay should be able to field a closely-related question, offered in realtime. If they can't, that's concerning.
(To be sure, some students will address this approach by having the machine auto-generate the essay than going off and reading around the answers the essay gives to understand the adjacent space to field the upcoming question. Good; that means they just used the machine as a "cliff's notes on the topic" generator, they still did some research).
That's also just testing memory. I had a teacher do that to me in high school where they graded papers during winter break and when they asked me about it in January I could barely remember that I wrote the document let alone the specifics of the topic.
Of course, I have a exceptionally bad (37th percentile) long-term memory so maybe that's just me. Schooling in general seems to be geared towards tests of memory at the expense of understanding.
High school me: "Well I understood it at the time" hehe
IIUC, testing memory is half of pedagogy. Memory (or the student growing the necessary techniques and discipline to supplement an internal lack of it) is pretty key to (at least US) pedagogy in the primary / intermediate levels.
In multiple subthreads here, people are embracing the idea that if machines can just write all the prose -- great, why should humans bother?
As with image generation, one thing I think we haven't adequately considered is once a sizable fraction of the available online data is machine generated, but isn't marked as such, and we begin training models on the outputs of the last generation of models, structurally we can enter a different regime. A sequence of models each trained on the prior model's output can converge to meaningfully different behavior, because we're repeatedly, incrementally changing the task by changing the training data distribution. If _all_ the data is generated by the prior model, this process seeks a fixed point which is reflective of the model architecture and not the original training data. There's a very real possibility that using generative models more (and publishing their outputs) can make these models worse in the future.
Weirder, however, is no one has really had the opportunity to look at what happens to human language when a sizable fraction of what we read is produced by these models. Will we normalize any quirks of their output? Will we reproduce or incorporate any idiosyncratic features into our own writing? How will we adapt in a changing linguistic environment?
I'm not to worried about convergence of the models that use prior training data. Eventually the models won't work and people will notice, and either create new training data or go back to only using older data to train. Also there will always be people who "want to do it the old way" and will still create new art and new writing, which will seed the training data.
As for normalizing the quirks of the output -- maybe? But would that be so bad? Language changes all the time, it's constantly mutated by influencers (not the Instagram kind, but the ones that have existed for centuries). Look at how British English is literally called "The Queen's English" because it actually shifted to how the Queen spoke, since she ruled for so long and was very influential to that society.
Also it should be noted that some news articles, especially in finance, have been written by computers for over a decade now, and not a lot of people seem to have noticed.
As an applied ML practitioner, currently we get to choose how to use synthetic data vs "real" data, and in what proportions. This can be a valuable tool in our kit. To the degree that data in the wild becomes an unlabeled mix of the two, functionally we lose the ability to make those choices for any given model.
> Eventually the models won't work and people will notice
For any product dependent on these models, that sounds like pretty negative outcome ... and entirely consistent with my concern that "[t]here's a very real possibility that using generative models more (and publishing their outputs) can make these models worse in the future."
> and either create new training data or go back to only using older data to train
Especially given that currently LLMs basically learn about entities and concepts in the world via their training text, this breaks the ability to update the model to know about more recent topics of discourse independently from shifting the real vs synthetic proportions.
> there will always be people who ... will still create new art and new writing, which will seed the training data
But if we aren't able to consistently separate the human-generated and machine-generated content, model training won't be able to place any extra weight on the human-generated stuff. The mere fact that human-generated output doesn't disappear entirely doesn't remove these issues.
The analogy is loose, but click fraud creates realistic looking data exhaust that looks close to the behavior of a real user, and can meaningfully disrupt one's ability to optimize for clicks or to know how many actual end users interacted with your item of interest. The fact that some nonzero portion of the clicks are real doesn't erase these problems. And that's in a system which doesn't create the kind of feedback loop described above.
> Eventually the models won't work and people will notice, and either create new training data or go back to only using older data to train
That has already happened in "free market capitalism". Prices are supposed to be the representative models of the product quality, but became decoupled from it because companies set the prices based on competitors and not based on quality of what they sell.
> As with image generation, one thing I think we haven't adequately considered is once a sizable fraction of the available online data is machine generated, but isn't marked as such, and we begin training models on the outputs of the last generation of models, structurally we can enter a different regime.
Time to time I hire entry level people. Their writing in general, the primary method of communication is atrocious. I am not complaining about fundamental grammar or punctuation. I make those mistakes more frequently then most myself.
There is no cohesive flow of thought; there is a lack of logical structure. They are unable to unpack their justification for recommendations. This costs us because we have to have multiple "draft reviews" until the document is succinct, or cost us when the recommendations are misunderstood.
Over the last three decades I have seen the decline in the skill of writing.
So automate. If computers are better than students at writing essays, they're probably better than entry-level hires at writing whatever documents you need.
The purpose of writing in professional settings, especially internal documents for a engineering team, is to convey information.
Even if models are better at writing essays, it's highly unlikely that the generated essays will convey useful and accurate information. Ie, the writing may be better as a context-free composition of words and symbols, but the semantic content of the writing within the context of the business will be at best nonsense and possibly even misleading.
In GP's context, excellent writing that's pure bullshit is even worse than bad writing.
Generating design documents that discuss real tradeoffs from a combination of email threads, slack messages, meetings, and code is quite a bit different from generating the billionth essay on Napoleon. We use the latter, in part, to practice the basic skills required for the former. But just because a model can do a half-decent job at the latter doesn't mean that it is anywhere near being able to do the former.
I haven't seen a proper document (as in "documentation") for years. The API code I have to link to is auto-documented by compiling together comments in the code base. The language documents are written by humans, probably, but in a style that a computer could easily replicate (if they can do better than a high-schooler essay, definitely).
90% of the writing tasks I've encountered in the last 10 years is content generation for content marketing. And failing that entire industry dying in a fire (which would be a good thing), having that auto-generated by AI would be simple and no loss to humanity.
> it's highly unlikely that the generated essays will convey useful and accurate information
My MBA essays didn't do this either, so I'm not sure what the problem is
> I haven't seen a proper document (as in "documentation") for years.
I am sure that is true for many types of code. But also, for many projects, you can train up a reasonably productive coder in 3-6 months of bootcamping. In those cases, you probably don't need much more than auto-generated documentation. But also, in those cases, I'm not sure what a GPT-like model is doing other than creating the facade of written documentation around a product and process that doesn't really need anything other than the most superficial of manually written documentation.
On the other hand, there are many cases where actual written documentation is quite necessary.
I used to work on extremely high-trust code bases, and now I mostly work on mid and late stage large "Big R, Big D" projects.
In both cases, written documentation is extremely important for communicating the whats and whys of implemented code. Auto-generating API documentation would be pretty useless without a very literal programming style of development (think tens or hundreds of pages of math-heavy documentation for every KLoC or so, and in rare cases an order of magnitude more.) And even then would be awkward in many cases; eg, in cases where there's close interfaces between the software and product-specific custom hardware (either chips or sensors or actuated mechanical components). Or especially mathematics-heavy portions of programs where you really need to read a couple dissertations before you can start reasoning about the implementation details of the methods.
Additionally, we do/did a lot of writing for non-technical/differently-technical audiences. E.g., patent attorneys (who need thorough descriptions for translation into effective patentese), colleagues who interfaced directly with safety regulators, colleagues who interfaced with lawmakers, internal documents that might one day be seen by lawmakers or counsel, communication with external technical stakeholders when working on either greenfield or evolving standards, etc.
A strong foundation in basic writing skills is very important when you need to communicate the same basic facts to a variety of technical audiences who all have different concerns, incentive structures, and backgrounds.
> My MBA essays didn't do this either, so I'm not sure what the problem is
I refer to the article "computers are better than students at writing essays".
It can't be both: either AI is now better at writing than people, or it isn't. If it is, then let's use it to write everything that it can. If it's not, then we don't have a problem.
If people are bad a structuring an argument and conveying their intent in writing, what makes you think they are better with those things when speaking? or any other form of communication.
> free students from the drudgery of pointless essays that ask them to regurgitate content (as opposed to essays that teach writing skills or critical thinking, which remain valuable)
That's something that sounds good on paper, but it's incredibly difficult in practice. For one, students are all over the place in ability, so what's critical thinking for one student may be the most boring of intuition for another. Evaluating students based on their critical thinking is ripe for subjectivity. I've had multiple teachers who were great at teaching, but clearly had favorites and that was very discouraging for students that were on their bad side and received arbitrarily bad grades. The English teachers I most appreciated followed a boring rubric that incentivizes more formulaic writing.
Also, regurgitating content is an important skill as well. Half the formal writing I do at work is documentation, which strictly falls into this category. The other half is design docs, which arguably is a form of regurgitation as well. I'd come up with the design anyways, and the doc is just to share it to others.
What's more likely is that coursework will be abandoned and all assessment will be done via invigilated exams :-) (As it was in my degree in fact).
I've marked 'essays' in computer science (billed as reports but equivalent enough for these purposes) and I think they are mainly set because the university wants the students to be able to write clearly which is a reasonable ambition.
It's often really hard to assess 'understanding'. It's possible to detect non-understanding more easily - you ask enough questions and hope that the non-understanding slips. I guess you try to give students enough rope to hang themselves. Or of course sometimes it is obvious that they don't understand things from the get-go. But it's very hard to create questions novel enough that they can't basically be gamed with regurgitation. Universities do try but still difficult.
Are you sure the essays are set for that reason? I had to write essays and reports a lot in my CS course but the reason was pretty clearly because the professors didn't want to do code reviews. They certainly didn't care about clarity of writing given the absolutely minimal feedback provided on those reports, the general uselessness of what little was provided and the fact that by the time you reach a university you've already sat through many years of educators teaching you how to write (or trying), so CS profs have no edge there.
What taught me to write wasn't school or essay writing homework, not even at university level. The crucible that forged my own ability to spell and argue was the internet. Slashdot, blogging, later HN.
If I was a teacher trying to teach people to write, I'd be tempted to pick some reddit forums, and ask students to find a comment they thought they could add to, then submit (to me) a set of replies to them. Online forums give you real time feedback in whether your argument landed or not, whether people understood what you meant and so on. Literally every mini-essay is graded by other readers, in real time, on demand (maybe not on writing quality but it never is, not even in school).
The only real solution to this (assuming that grading systems are all that valuable in terms of optimal educational outcomes) is to base grades on in-class essay writing (and for CS, in-class coding) where students don't have access to any outside assistance, and have to think quickly.
Using tools outside class to practice for the in-class work would make a lot of sense however, as students could then rapidly see what a decent essay or code body looks like.
Of course. I don't understand how essays written at home are acceptable. It's a good exercise to do them (if done properly) but they shouldn't count for the final grade.
Before AI there were professional essay-writers, or tutors, or parents. In a sense AI is progress because now everyone can cheat, instead of only the wealthy.
This is common in UK universities at least. You can have your entire grade for a year of dense reading at the MSc level be based on three hours of hand-numbing speedwriting. It makes sense while at the same time rewarding and punishing different personality types (and calligraphers).
The worst thing about this system is that there's not usually any chance to retake the exam. As far as I can see, there's no good reason for this, and all it does is heap pressure on students while providing a worse overall assessment of their ability.
Don't most UK universities let undergraduate students resit exams they've failed, within reason?
And a degree consists of a good number of courses, each of which will have its own exams - so if you do poorly in one or two exams, there are plenty of chances to bring your average back up.
> Don't most UK universities let undergraduate students resit exams they've failed, within reason?
I've attended 3. Two of them made students retake the entire year if they needed to resit an exam. The other one based students' entire grade on one set of final exams at the end of the 3 years and had no provision for resits at all.
I believe the first two are typical in the UK.
> And a degree consists of a good number of courses, each of which will have its own exams - so if you do poorly in one or two exams, there are plenty of chances to bring your average back up
That's true, but it's also true that a more lenient system would be more compassionate (student mental health is a huge topic for universities) whilst simultaneously being a better measure of student's true abilities.
> there's not usually any chance to retake the exam
Of course there is. One of my classmates milked this system to remain a student for about twice as long as the rest of us. He was good at manipulating the bureaucracy, not so good at physics; probably should have studied law or some sort of social science instead.
If you look at it another way, it's like a lifetime warranty. You can't trust an appliance vendor to provide quality, but you can trust that if they have a warranty they will meet the conditions where it will make financial sense to have one.
Having such an exam adds enormous extraneous pressure that does not reflect the reality of conducting research, but it is this pressure that guarantees (with some exceptions) that the student has been coerced into learning the material to a sufficient standard. There will always be the cramming psychopath who grabs a first after just a week of study and who retains nothing of it, but most students just have to learn intensely over the course of a year to pass through the filter. In some ways, this is also why it's common in the UK for employers to not care so much about the specific degree, just that you have one with good grades from a reputable institution.
> There will always be the cramming psychopath who grabs a first after just a week of study and who retains nothing of it, but most students just have to learn intensely over the course of a year to pass through the filter.
My concern is for the students who study intensely, learn as much as anyone else, but don't pass the filter. Universities in other European countries allow single units to be retaken without retaking an entire year, and it seems to me that this provides a fairer system without devaluing the qualification in any way.
I agree in principle, but there are so many special accommodations required (extra time for graded assignments, special assistance in the form of a reader, etc.) that would make this a nightmare.
I lament, though, that students are often either (A) so overburdened with distractions that they don't have time to learn, (B) so pressured to get high grades that they cannot afford to get a bad grade, and thus must cheat, or (C) just don't care. There are a few gems that are willing to put in the work to learn and appreciate a subject, though. I just wish that more students put in the work required to learn.
I have legitimately met college students in the last few years who had no frame of reference for how much time they should be putting in. Freshmen who think that spending 15 minutes a week looking over their notes is "studying a lot" are pretty common right now (in the USA).
Three hours of study for each hour spent in class is an average estimate of what it takes to absorb even moderately difficult material in courses in sci/tech.
Relevant quote: "Every truth has four corners. As a teacher I give you one corner, and it is for you to find the other three." (Confucius)
Which with napkin math can be shown to be bullshit. Students are spending approximately 1/3 of their time in classes, so how would they get time to sleep among the 4/3 required by studies and classes?
1:3 hours sounds super aspirational even for only the hardest of classes like organic chemistry and fluid dynamics.
But your napkin calc isn't addressing the bit where you only take 1-2 classes like that per semester and pad the rest out with things like literature and electives so you could have a life.
> students could then rapidly see what a decent essay or code body looks like.
You mean like how today they study well-regarded human works? I’m not sure how the ML adds anything except which words tend to appear close to each other across a large corpus. The moment where young writers believe that ML is creating anything new and they try to learn from that is when modern civilization starts to eat its own tail.
Given that a typical programmer on a typical day has online access, and given the creative lengths students will go to to maintain access to said resources during exams, it seems worth considering allowing online access for programming exams. To address the obvious problems with that approach, could we use AI to generate unique variations on the problems we want the students to solve?
Or we could stop asking students trivia and force them to think. Most of my graduate school exams were open book + notes but it hardly helped. You had to demonstrate knowledge by producing a novel solution to the problem most of the time. Not novel as in new research, but novel in the sense you weren't just regurgitating factoids from the books. I also had tons of open book and open note advanced math classes. The problems were made in such a way you had to make a connection with the material and link pieces together in non-obvious ways. In those classes I retained more information even to today.
Teaching, even at the collegiate level, has become "how can I do the least so I can do what I want". Tenured research professors generally make terrible teachers...perhaps we need professional teachers.
For me, the most "open" a test was, the more I worried about it. Expecting students to remember everything on the test meant that it had to be easy.
The hardest exam I ever took was my final exam for Linear Algebra. It was so "open" that people could use matlab on a shared computer that was also projected to the entire class. Turns out, the best students in the class didn't need help on a computer, so it turned into this kind of mind-fuck where you weren't sure if they got a different answer than you because they were wrong, or because you were. To add to the excitement, the questions all tied together and used the answer from the previous one, so if you missed one, you were guaranteed to miss every subsequent question.
This is like any open-book exam. Make the time limit reasonable such that if they know the stuff, they can get it done while referring to a few things in a source, but if they don't know it, there's no way they can look it all up in the available time.
You could have them write using a program that allows the teacher to play back their writing process in real time as well.
Theoretically they could copy it from offscreen but unless they’re really good at faking going back and changing things and getting stuck at the right points it would probably be fairly easy to tell.
My degree is in English. Specifically I focused a lot on composition theory. Or writing about people writing. I have a few thoughts about this.
1. If it isn't rampant now, it will be in a few years time. Cheating is highly incentivized in college, and this makes it very hard to catch. I doubt you could generate a thesis this way, but you could certainly pass your electives.
2. Writing is highly conventionalized. Half of getting a good grade is following conventions. AI will get really good at this.
3. I suspect ESL students will use this first. When tutoring writing, extremely smart ESL students were getting poor grades in writing courses for having bad grammar, vocabulary, whatever. This will help them get good grades until the AI surpasses native speakers, who will then use it.
4. This is really tragic for students. Writing is really great for solidifying thoughts and learning. I still journal to this day just because of this. This will deeply affect how people learn. But the outcome remains to be seen.
I'd really disagree with this. Essay writing forces a person to think critically. You have to construct an argument for or against something. Even if it's echoing source material or a teacher, you still have to think in a way that other testing formats do not.
By offloading this to a machine the students are undercutting themselves more than anything else. They'll be less effective communicators and less capable of disagreeing with ideas and concepts the face in the real world
Eh, once you get the pattern down it becomes almost a rote operation. Especially for a 2-3 page paper. You find 3-4 facts/arguments to support your conclusion and turn them into paragraphs. Slap on an intro & conclusion pre/re-stating your 3-4 points and you're done.
In my required college 1st year writing class I had to read a lot of my peers' writing. Most people have not learned how to do this. They do not understand the concept of stating a premise and then supporting it with evidence.
“Once you get the pattern down”, just like learning other subjects like math? But many kids don’t have it down. And there are gradations in complexity. An 8th grader who’s got it down and a 12 grader who’s got it down will produce different artifacts and should have different assignments.
I agree with you, but I also had instructors who demanded the same thing: essays as arguments, drawing on evidence and analysis to back up a thesis.
An essay that is simply a description of the source material, or a listing of pros and cons with no actual thesis, is a totally different endeavor, and one that has much less value as method of instruction or assessment.
Restating or paraphrasing a widely accepted thesis statement isn't very educational? When your freshman class of 200 students is writing essays on a handful of topics, the same ones every year, that every other school touches on, there's not going to much in the way of originality.
Then again I guess the same argument can be made for maths but no one complains that we're teaching to memorize and repeat the same steps of various proofs one learns in 1st year.
Essays are often meant not as a measure of understanding, but as a tool for the student to learn, to reflect on how well they understand the material, and for the teacher to be aware of their weak points. Sometimes they're also meant as a tool for training the skill of essay writing itself. In those cases, it's only graded in order to nudge the student into reading the material and studying (and sometimes to cushion the impact of test scores).
The rule is you have to turn in your own, original work. You provided a prompt, and a piece of software turned it into an essay.
Is this inherently different from using the predictive keyboard on iOS, or using Grammerly to write clearer sentences? Where is the cut off point where the computer did too much work on its own?
If you're feeding in the ideas you want the essay to convey and the AI is just turning that into nice prose, I would describe it as a tool instead of cheating - you're still doing the important part. If you're only feeding it the same essay prompt you were given and it is coming up with the ideas it expresses, then it's cheating.
Likewise using a 4 function calculator in an arithmetic class is cheating but in an algebra class it's not.
Is Outlook going to come with an AI that helps an engineering manager succinctly express the pros and cons of two competing approaches to a nontechnical audience and then justifies the decision to go with one over the other? What if these approaches are novel or otherwise something an expert system can't "know about" (or look up on the Wiki)?
Regardless, IMO this is where the discussion becomes too hypothetical:
• Right now, AIs are bad at writing.
• If a student is able to masterfully craft a prompt which causes today's bad AIs to produce good writing, that student is talented, and I suspect they actually did a significant amount of work.
• If today's bad AIs are able to produce assignments which receive passing grades with minimal prompting, something is wrong with the assignment. Teachers should assign work that isn't so rote a computer can do it.
• If, in the future, AIs are able to write strong, well-reasoned essays about novel concepts, students won't need to learn how to write. They may not need to learn much of anything†, because I don't think there will be much space left for humans in the workforce.
-----
† "Need" is a key word here; I do believe in learning for its own sake, and apparently we'll have lots of time on our hands. Cool story btw, learning how to develop film in a dark room is lots of fun, would recommend.
AI assisted technical writing is already a thing. You might need to give it more input and do more cleanup on what it spits out, but it can still greatly reduce the time and effort needed. There's no reason to believe the technology will stop improving anytime soon.
It's a really good question, but also a really silly question that no one should have to guess about. Teachers should be simply be providing explicit policies one way or the other.
I had "anything goes" assignments and "strict" assignments. For the former I didn't care. For the latter I could either tell whp if you were cheating, or else proctored the assignment.
Math is a good leader here. Sometimes you allow certain tools. Sometimes you don't. The set of tools allowed tends to increase as courses get more difficult and assignments get larger. It really just depends on what is being evaluated.
I'll happily add $100K to the salary of an engineer if I have strong evidence that they can write well. Poor writing skills is probably the most common reason that people fail to receive a promotion to Senior (albiet often indirectly -- if you cannot write well, having enough impact to justify the promotion is a lot more difficult).
I don't want to automatically defend pointless essays about 'How X influenced 19th century England' or whatever is the current go to topic for teachers, but, and it is not a small but, the whole point of writing pointless essays is to give students an idea on how to write for an audience ( their teacher ).
At the end of the day, unused muscle will atrophy and students may have trouble even producing appropriate prompts for ML generator. I would weep for the future of humanity, but:
1. Coffee did not kick in yet
2. I have upped my nihilism lately
Larry McEnerney, Director of the University of Chicago's Writing Program, makes the point that (I'm paraphrasing) students don't learn to write for real world audiences by writing for an audience that's forced to read their crap (teachers).
As a person, who, on occasion, has to read email from people, who send me things, I wholeheartedly disagree with this statement. I do, technically, have to read some of them as it is part of my job description ( not completely unlike the teachers ) and very much a captive audience.
And I get that it is hard and everyone has their own idiosyncrasies and all that jazz, but, and this is probably the only time I will defend corporates, were it not acceptable language enforced by HR, those emails would somehow be even worse than they are now.
So the goal is the same.. give the audience what it wants. What do I get? Well, it varies..
edit: I am really enjoying the link provided, but clearly this guy is talking about a very different level of writer.
Sure there are. It's just kids don't know anything useful though that we actually need them to write essays. That's why we have them write pointless essays instead.
But I have to write essays on how certain systems work, and how certain tasks are achieved in my job in order to document these things. My writing skills definitely play a big part in making a tutorial that is easy for anyone to read and understand.
It's proven to have been the most effective way. I'm happy to learn about other options though since the many of the younger people coming in to the workplace now seem to be half-illiterate.
The issue is the writing paradigm shifts entirely in college. A college English professor would typically fail a standard B-A tier 5 paragraph high school information dump essay. One of the first things I was told in my writing courses in college was to forget that model entirely.
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them."
I am familiar with this quote and I while I do not want to assume, I think it was intended as a rebuttal/counter to old codger how ancients also thought writing would kill memory.
Spoiler alert. It did. Very few people attempt to commit things to memory outside some Guiness record competitions. And just in case I did not hit the point, introduction of Google with world encyclopedia at anyone's hand, further exacerbated that trend.
To sum up, I appreciate the sentiment, but in HN I do hope for, nay, I expect pushback in the form that goes beyond weirdly smug quote.
What I am saying is: argue with me. Don't quote me philosophical bumper stickers.
It seems to me that the average person around me remembers more than the ancients had the ability to even know. To your point of "pointless essays teach children how to write to an audience", there are entire populations of people who, as a matter of survival, learn how to modulate their communication based on the recipient. Those children learn this skill well before they write any essays about "How X influenced 19th century England".
<<It seems to me that the average person around me remembers more than the ancients had the ability to even know.
It is possible. I have no easy way to counter that. I could yell anecdata is not necessarily valid/useful/relevant data, but I know I use it myself to an extent so I will avoid going down that path.
It is true that is likely that I will likely be able to talk to a random person about a variant of pop psychology/crime investigation/science/relationship mechanics shown on recent TV show and I admittedly cannot produce evidence of Plato dealing with the same level of 'not exactly ignorance, but very low level understanding of anything'. I can talk with an average bloke near me about basic genetics, but it in a very limited kind of way.
Is being vaguely aware of a subject memorized knowledge ( 'ahh, yes, quantum mechanics.. that wheelchair guy invented gravity right' kinda way )? Plato was talking about the kind of memory skill that allowed one to recite Homer. And I am not defending oral tradition ( alphabet was a good invention ), but comparing my friend remembering lines to "Shake shake shake" does not seem to be on the same level.
So, to put it in a more direct way:
To what extent does an average person around constitute remembering more when compared to Homer? Or is there is just so much more to know in general that an average person can only deal with very vague generalities.
<< To your point of "pointless essays teach children how to write to an audience", there are entire populations of people who, as a matter of survival, learn how to modulate their communication based on the recipient.
Do they though? Last set of news that made circles across all media was that of professor, who made things too hard. I suppose you argue yes. Not only did they survive; they veritably vanquished their audience into oblivion. But I ask you: was that what I mean when suggesting writing and you generalized to modulating message for your audience? Both answers could be argued to be true, but society as whole suffers more with only one of them.
People used to recite epics and important texts from memory. All of culture and history was stored solely in the minds of the people. Illiterate people and literate people do use their memories differently, as one has to rely much more on their memory.
I find that the point of most High School essays is not to practice writing, per se, but to demonstrate that the student has internalized whatever lesson or moral the subject is trying to teach.
These issues with AI are arising faster than ever, but no one seems to stop an extrapolate the inevitable end result. What do you do when an AI can realistically handle your average college student's entire course load? Existentially, does it even matter? Yes people will be dumber on average, but will they care if they have the answer to all of life's questions in their pockets?
I'm old enough to remember people telling me I wouldn't always have a calculator handy, and obviously that didn't age very well.
It sounds unnecessarily pessimistic in the face of all the things AI will help us with, but given what we know about human nature, I do expect AI to make us a lot lazier and ... dumber.
>I'm old enough to remember people telling me I wouldn't always have a calculator handy, and obviously that didn't age very well.
Logical error though. The analogy to calculator is poor.
With a calculator it doesn’t matter what your opinion of 17×93 is. There is a right answer and a way to get there. Our ability to get output from a calculator doesn’t change because more or less people use them.
With AI prompted writing, there is no right answer, and what it spits out only exists because other people already wrote on the topic. The quality of the writing is entirely dependent on samples and usage.
The analogy is fine, as it applies to my example of an AI handling the entire course load of your average college student. Math, science, creative writing, etc...
Yeah, there seems to be a general sentiment that if a computer can do it, it wasn't worth a student doing it anyway. But presumably (?) there is a line to cross where that ceases to be the case. So where do you draw it?
> free students from the drudgery of pointless essays that ask them to regurgitate content
What "pointless" essays are these, may I ask?
The classic 5-paragraph essay (intro, point #1, point #2, point #3, conclusion) is the foundation of the vast majority of non-fiction writing. It's teaching the most foundational writing skills.
When students are asked to "regurgitate content" and they can't, it means they don't understand the content or they don't know how to write. They need to be able to master that before they can move onto more advanced writing (or more advanced content).
I hated writing essays in high school, because I didn't understand the why or how, because I had mostly bad teachers. When I finally learned how to write papers in college, everything clicked and I went from getting C- grades to A grades in subjects that involved essays -- and the writing and explaining skills that I learned, I later wound up using daily in my professional life.
But the problem was never with "pointless essays" -- it was that nobody ever taught me how and why to do them properly.
The 5-paragraph essay as taught in schools misses the point. The point of an essay is to make a single cohesive argument, preferably in as few words as possible.
"Write a standard, 5-paragraph essay discussing the theme of chaos in Slaughterhouse Five, making sure to cite several examples from the text, 1000 words minimum" is a ludicrous assignment. The assignment should be: "Argue whether chaos is a significant theme in Slaughterhouse Five, 5 pages maximum."
Not only does it miss the point, I suspect it's why there's so many blog posts nowadays that end with a paragraph titled "Conclusion" that just repeats what was previously said, without actually tying those thoughts together.
What you call "ludicrous" I call "beginner". Beginning students need the 5-paragraph structure, and they need minimum word counts, just like they need to be told to cite examples. Otherwise they just turn in a single-sentence "Slaughterhouse Five has a lot of chaos, which you can see clearly from reading the book, the end." Maybe you laugh but you will literally receive this from 9th-graders.
The assignment you're advocating for is more appropriate for a weekly college homework assignment, where you know that the (good) students will struggle to get it down to five pages, and (hopefully) don't have to be taught that their arguments need to be supported with citations. But it would be disastrous for most high schoolers.
You need to learn to play chords and scales before you can play jazz. Writing is no different, you need to start with the fundamentals.
> Otherwise they just turn in a single-sentence "Slaughterhouse Five has a lot of chaos, which you can see clearly from reading the book, the end." Maybe you laugh but you will literally receive this from 9th-graders.
That essay would easily get a 0, and the student (and their peers) would learn not to try that. A student producing a single sentence instead of an essay is a different problem than a student producing an unclear essay or failing to demonstrate that they read the text, which, I think, is the point of the 5-paragraph essay.
The actual fundamentals of good writing are grammar, logic, and rhetoric. We only teach one of those (grammar) in schools, and we don't necessarily teach it well. The subjects of logic and rhetoric have largely been replaced by the "5-paragraph essay" until you reach college. That is insane to me.
Teaching 5-paragraph essays from middle school to high school is like teaching jazz by having students spend 8 years writing renaissance chorales. Writing a few renaissance chorales is probably good for jazz musicians, but spending 8 years writing them will not produce a good jazz artist.
Well said and written! Your bad teachers would be proud.
The theme in the comments seems to be "X is simple, therefore rote practice isn't valuable."
But that glosses over the fact that fully understanding and being proficient at X is often a prerequisite for learning X'.
Or in other words, calculators can do arithmetic. If we skip mastery of that, how do we propose to teach children differential equations?
What is probably more important is the grading-side of ML: by leveraging more ML to auto-note and -score the mechanics of assignments, we free up teacher time to focus on interpreting and suggesting improvements for students, which many teachers are currently too overwhelmed to do.
It was pretty common when I started writing papers in school up till maybe 10th grade? I've never heard it as "classic 5-paragraph" but as "1-3-1 format." After 10th grade we introduced bringing in counter arguments.
When Babelfish came out in 1997, I used it to do my French homework, while attempting to fix up when it didn't quite get it right or used an idiom that didn't translate to French. My friend did the same and we were casually warned to stop "whatever it is" we were doing after a couple of weeks ;-) It turned out it was a lot safer to let Babelfish read French than to write it..
As a bit of fun, I recently scanned my daughter's homework and threw it into GPT-3 and while I didn't let her see or use the end results (if she had the idea to do it herself, that's her call, but I find the younger generations are not as computer savvy as the media would like you to think!) it pretty much nailed it without me having to do anything else. It's definitely going to cause changes in the educational system in a way that Babelfish certainly did not.
The question is whether they want to adapt. I know of teachers who wouldn’t even look at the documents. A couple of classmates (10+ years ago) literally turned in empty docx. They all got the same grade as on previous exam.
Professor here --- I really need a tool to detect this. I have a student whose text always have the same faulty patterns: circular sentences, paragraph that do not sum up, etc. The problems are consistently the same, and even in purpose this cannot be imitated. I think there is a machine behind.
If it's just one student, then I think you should just talk to them about it and verify. Education is not a 'gotcha' game. If you're in the US, these students are the ones paying a lot of money to get that learning and degree.
If it's a lot of students, then you likely need to rethink using essays and the like. Not that they aren't useful when done honestl; they are incredible tools for having students learn.
But the near future is only going to be filled with this more and more. You're not likely to be able to outrun OpenAI and GPT3 on your own. Heck, even the whole CS department isn't going to outrun these companies. In fact, it's not a bad idea to put your essays through such tools to see what comes out and then build from there.
The nature of learning and education is changing very fast these days and clinging to the old models and methods isn't likely to be the best strat. Innovate, try new things, talk with your students, brainstorm, etc.
Essay cheating is really annoying. I've learned a foreign language to the point where I've joined a rather large facebook group where students from that country studying abroad ask for advice about problems. 1/3 of the posts are students who are caught cheating asking for advice or asking for recommendations / trying to find someone to do their homework for them. It's made me really dissapointed in my schoolmates to see how they can just talk so openly about it.
One of our local writing teachers has addressed this by having the students do some surprise writing in class, no phones, and comparing it to home work.
"Write a persuasive essay regarding why the essay you're writing deserves the grade you choose based upon your knowledge of persuasive writing techniques and your classroom participation."
I imagine as the technology improves and more AI tools become available this will become a war of attrition. Especially since these tools are so useful for non-school purposes, I can only see them getting better with time. It is a shame when people use these tools in a 100% no effort way though, reminds me of people who copied whole essays online in the past and got caught.
1. Assessment is only a small part of teaching. Often the smallest.
2. The pedagogical intervention depends heavily on whether the student is cheating or just FAR below grade-level.
3. the student likely won't be truthful.
4. Even if we assume the student will tell the truth, if they aren't using an LLM and you accuse them of doing so it's going to crush the student and make teaching that pupil close to impossible.
This is an ideal result -- after all, their working lives will be turning in machine-generated work to machine-evaluating bosses, so we should get them started early, lest they fall behind the prompt-engineering curve.
Something I haven't seen mentioned much is the potential for this to actually be a powerful research tool. As a trivial example, here's a prompt using the highest quality Da Vinci model:
Write an analysis of the 2020 MLB baseball season
The 2020 MLB baseball season was one of the most unique seasons in recent
memory. The season started later than usual due to the COVID-19 pandemic,
and it was shortened to 60 games. This meant that every game was even more
important, as there were fewer opportunities for teams to make up ground in
the standings.
The Los Angeles Dodgers won the World Series, defeating the Tampa Bay Rays in
six games. It was a dominant performance by the Dodgers, who won all three of
their postseason series. They became the first team since the 2009 New York
Yankees to win a World Series after losing the previous year's Fall Classic.
In 2007 The Rays made an incredible run to the World Series, becoming just the
second team in MLB history to reach the Fall Classic after winning fewer than
40% of their regular season games. For the 2020 season they were led by their
pitching staff, which posted a historic ERA of 3.56 during the regular season.
Though this is at least a little potential than actual capabilities, since fact checking other responses to questions has shown problems-- correct facts in the wrong context, e.g. similar questions gave stats for a team that were correct, but for a season other than the 2020 one asked for in the prompt.
There's several purposes for these highly structured writing assignments.
1. What is the student's reading comprehension. Did they understand the reading assignments and lectures.
2. What is the student's writing skill. Is it free of grammar, spelling mistakes, and is it clear and comprehensible.
3. Can the student follow instructions. Did they carefully read the prompt and directly address all points of the prompt.
Reading comprehension can be largely offloaded or doublechecked with cliff notes, youtube videos, or just bugging your classmates for their notes. Writing skill can be largely tool driven with spell check, grammar check, and the new NLP tools like grammarly or hemingway.
This largely leaves modern college essays to testing bullet point 3, did the student fully address the prompt in the manner the teacher specified. In my undisclosed observations this is the primary point reduction in college writing that causes a student to not have an A. This is also the exact type of thing the substack suggests can be offhanded to GPT-3, what makes college writing awful for the college students. If this is the case, are we going to build anti-GPT3 techniques such as have students write in a controller lab setting? Are we going to accept that a certain number of submissions will be machine-enhanced and let grade inflation continue? Are we going to accept that modern college essays might be more an exercise in editing machine output than generating new text?
The best math classes accepted the invention of the calculator, the computer, and use them as tools to teach even more math.
Essay writing unconsciously reinforces one’s opinion on a topic. This is an excellent tool for intellectual development when the student is free to argue any sensible position. Unfortunately, teachers often require students to choose orthodox positions, and thus essays become a primary tool for indoctrination.
Cheating has no place in an academic institution, but machine generated essays are an interesting way for students to avoid enforced compliance.
This article contains a logical error that I see quite frequently, which is the belief that something being anti-X is somehow pro better alternative Y. In this article, the author believes that writing that stimulates critical thinking and analysis would do a better job at preparing students than the current five paragraph essays that students typically write. They argue that the prevalence of text generators will force change in education from the current system to something better.
The argument makes no sense when considered analytically. There is nothing special about the factual five paragraph essay that lends itself well to text generators. Rather, the fundamental issue is that humans have discovered ways of training ML algorithms that can perform almost any task that a human can do. There's nothing stopping the creation of text generators trained to write any conceivable school assignment.
Cheating is only going to get easier and more sophisticated over time. This presents a serious issue that may require rethinking how students are evaluated in general.
I don't believe, as an ex-English teacher, that any teacher thinks a student is 'acing' their essay if that's not shown in their other writing.
Little Johnny, with his poor spelling and grammar, does not suddenly turn in an excellent paper with exemplary grammar, spelling and punctuation. So unless they teach computers how to screw up in the exactly the same way that Johnny does every time he writes a sentence, any teacher worth his salt is going to know it's not his work.
As for those with excellent writing skills, well, they have less need to cheat but even they would be sussed out with in class essays and assignments.
> unless they teach computers how to screw up in the exactly the same way that Johnny does every time he writes a sentence
You can ask an image generator for something like "Barack Obama painted by Picasso" and get a passable result, so it's probably possible to augment a writing model to mimic the style of Johnny's past essays.
As an author, I’ve been thinking about the future and it sounds dreamy. AI is good at producing verbiage, which readers want for some reason. Humans are good at producing insight, which is the true product that readers are looking for, but don’t want to pay for (with clicks or money).
So what if the author produced a 5 bullet insight and AI expanded it into the word salad that readers want to buy? We’re getting there, lots of startups in this space.
Another aspect of why the word lettuce matters is SEO. So it feels like there may be a future where humans communicate in 5 bullet insights packaged as word salad by bots for bots.
Seems silly to have an AI write an essay and derive the main points itself, except perhaps to differentiate from another AI, but isn’t this what seeds are for?
Because the AI can’t produce a novel insight (yet?). Those 5 bullets are the main points, AI can expand those into an essay.
This idea comes from how you’d write any long essay, or a book. You start with an idea. Then you expand it into chapter/section titles. Then you expand into each of those into more sections. Then more. This gives you the full skeleton of what you’re trying to say and, really, already says the whole valuable part.
Then you fill the rest with relatable anecdotes and other salad to make a meal and drive the point home. AI would be great at that part.
I think this would be better, at least for now. But if the AI was trained on material that included the book and other writeups about the book, it might already have the answer.
Required intro Writing courses in college aren't so bad these days, at least in many places, shifting more to a critical thinking model of writing construction.
That said, other college courses are often required to have a writing component where it simply doesn't make sense and you get nonsense assignments. I like the ability of AI to respond to nonsense with well-constructed nonsense. Here's a nice essay I generated that demonstrates the ability for GPT-3 to respond to even ridiculous prompts, so it should have no problem plowing through the average topics encountered in school.
Prompt: Write an essay about the horror of ice cream (start text: Murder)
I can't think of anything worse than ice cream. I mean, maybe it's not as bad as, say, murder, but
it's up there. I shudder to think about the horror of ice cream. It conjures images of congealed dairy
product, cloyingly sweet and sickeningly creamy. To add insult to injury, it’s often served at
incredibly cold temperatures, making it a perfect vehicle for achey tummies and jittery limbs. Then you
have to deal with the guilt. After all, you're eating something that was created from someone else's death.
It's a horror that doesn't need any extra fuel. Ice cream is already one of the most gruesome things
imaginable. Imagine adding in the thought of someone else's death? It would be like watching a horror
movie and having the killer pop up out of nowhere, ready to devour your flesh. And that's exactly what
happens when you eat ice cream--you become a victim in your own life-sized horror movie.
To put it bluntly, ice cream is dreadful – and there’s no getting around it. In fact, I would
go so far as to say that it’s one of the most reviled foods on the planet. After all, why would
anyone want to consume something that tastes terrible, feels terrible on the stomach and makes
you feel like you’ve been beaten with an ice cream scoop?
Ice cream has also been linked to violence. In 2005, there was a murder spree in Maryland in which four
people were killed by an ice cream enthusiast named Jeffrey Dahmer. Dahmer was known for killing and
cannibalizing some of his victims. He would lure young men into his home with the promise of free ice
cream and then murder them.
Since this is hackernews: During my undergrad I had a professor that would give us our homeworks as normal. But to hand it in, we had to demo the project. He would tell us what to put into the program, check the outputs to make sure it works. Then ask us to show the code for specific parts to spot check it and ask us how it worked. Its essentially an oral exam, and would be hard to cheat it. It is hard to fight against technology but in person has a way of maintaining a good standard.
I'm getting a little tired of the fallacious argument which compares some new technology to an old technology from the past, and says "see how people were afraid of that technology from the past, yet their fears proved unfounded? In the same way, fears about this new technology are unwarranted."
The reasons I grumble whenever I see this argument made are:
1. The things being analogized are usually not related in such a way that one prediction not coming true suggests that the other prediction won't come true. You have to examine the likelihoods in isolation, you cannot reason from analogy about everything.
2. Often, predictions about the negative consequences do come true, it's just that we adjust our response to them and fit them into our new view of the world. For example, the prediction that allowing calculators in the classroom would make students worse at arithmetic wasn't wrong, we just decided that we didn't care if students are good at arithmetic anymore. That's fine if it's how we decide the world should be, but it is not the same thing as the prediction being wrong.
3. More generally, the way I view disruptive technologies is that they are not "just like" some previous technology, and therefore analogous to it. What I mean is, when someone says "people staring at their phones is just like people staring at newspapers in the past", they ignore the fact that phones replaced newspapers. There was a competition, and newspapers lost, so we have to acknowledge there is some meaningful difference between the two that explains why.
Have you given A grades to papers that you are certain were generated by LLMs? Or have you spoken with educators who have?
IME, LLM-generated papers are more likely to generate concern about mental health than As. Seriously. I've had faculty comment that they are concerned that a third of their students are suffering from some sort of actual mental illness, and when I explain LLMs they are incredibly relieved that there is a possible alternative explanation.
I had a chuckle at this. Good this is happening. Kids spend too much time on that anyway and most of it is bullshit. Sure if you had some great success we need a verifiable way to show it (maybe a resume). But this whole “oh look what great things I plan for the world” is nonsense. Hopefully it’s a thing of the past soon.
Edit - talking about essays that folks write for college admissions, in case the context wasn’t clear.
As a non-tenure track college instructor, I'm pretty much "good" on this as well.
I think perhaps obviously the bigger underlying difficulty is that very often "teachers" aren't seriously empowered to, nor really judged on their ability to, teach. In higher-ed of course, it's because publishing is king.
I have sympathy for the great number of people who teach in higher ed for whom this will make things harder -- but also it's gotta change.
(For what it's worth, I've seen that "writing" from college students has improved drastically over the last decade or so and I'm pretty sure the vast majority of them are not doing this. I just think they are more involved with the "written word," even if those "words" include emoji, etc. But grammar and even logic have improved tremendously; the only downside is that it all sounds like branding/advertising copy. But that's better than what we had before)
> regurgitate content (as opposed to essays that teach writing skills or critical thinking, which remain valuable).
Thanks for the clarification.
That said, Devil's Advocate here...
But isn't "regurgitate content" a writing skill? That be all still quite a bit? Don't they have to walk before they can run (i.e., fine-tuning communication skills)?
Regurgitation essays are graded, and that produces a bell curve of quality, doesn't that indicate the skill is a skill worth recognizing? Isn't the lack of such a simple skill indicative of something in a student?
I agree that critical thinking is essential. It's something too few of us seem to have. But aren't these essays the gateway to that next step? That is, if you can't be critical of your own ideas, and their organization and presentation, are you truly ready to move on to next level lessons in critical thinking?
Put another way, if it were math, why teach addition and subtraction? Why not just move on to calculus? Or higher?
People seems to be focusing on college level work.
Isn't this going to also cause huge problems at the high school level? When teachers are dealing with large classrooms and a very limited amount of time spent in class, sentiments like "just learn to teach better" aren't going to be very helpful.
The value of an assignment like "write three good things and bad things about biotech" is simply that you have to write something that is halfway coherent, using complete sentences, with good punctuation and grammar. It doesn't matter that you don't learn much about biotech, if anything at all.
The idea that because a machine can learn to do that, children don't have to, isn't very sound.
Calculators are not comparable because they aren't AI; calculators require a detailed specification of the calculations to be performed. They don't just write your math homework.
Just wait, AI will do that too. Word problems like "If John has three times as many apples as Bob, who has four fewer than Mary, ..." will be solvable by AI, and so then calculators will go away, being replaced by zero effort.
"when I were a lad"... we used to have lines given to us as a punishment. Cue Bart Simpson writing on a blackboard, but for us it was "200 lines saying I am not allowed to shitpost on HN" or whatever.
If you could program a computer to print the lines for you, then 2 pages of dot-matrix printer output was accepted. I even had a teacher give me 2000 lines - a monstrous punishment by hand - because they knew I could code. They just didn't realise it would cost me an extra 5p in paper and nothing else.
Teaching has moved on, and lines are no longer a thing.
I hope teaching moves on from essays, because they're about the same level of educational achievement. Essay-writing is no longer a skill that anyone values outside academia, because we moved on. Academia should move on.
> Essay-writing is no longer a skill that anyone values outside academia, because we moved on.
Downvote because essay writing is essentially researching a topic, forming an opinion, and communicating that opinion. This is applicable in almost every knowledge job.
Disagree. When I was writing essays for my MBA, I formed the opinion, searched for journal articles whose extracts agreed with my opinion, and cited them. I don't think I read a single article or researched a single topic.
Essay-writing is about jumping through hoops; working out what the lecturer expects to see and then giving it to them. Admittedly that is also a skill required in every knowledge job (replace "lecturer" with "boss"), but I think there are better ways to teach this.
Real work values the ability to write clear proposals and designs, which isn't far from an essay. Even in tech jobs. Sometimes I'm given a doc that takes so long to understand that it'd be better to just ask the author. There needs to be some way to teach writing skills, whether it's essays or something else.
My real gripe with English class is that probably 95% of my assignments have been about fictional work where I'm supposed to describe a character's emotions or explain what the author meant by something. And seemingly the most boring books possible. It was useless and repetitive.
Thats really impressive students are doing this. These tools are non-trivial to use and it shows great initiative from them. I could see these tools being used proactively as part of a creative writing class or helping people improve their language skills. Top stuff!
There is a need for solutions on several levels here:
1. Quick, tactical. The most obvious fix would be for AI companies offering web access to LLMs to simply make searchable all the text they generated, ideally via an API so aggregator services can index and locate text given samples of it. If you pay enough (corporate/enterprise subscriptions etc), your output doesn't get indexed. Students would be forced to find LLM services that don't do this but there likely won't be that many, as offering such services for free is expensive.
2. Classroom changes. Force students to record their screen as they work on the essay, so teachers can see as it gets typed out. Of course again, some students can overcome this barrier, but it's about raising the bar.
3. Changes of approach. The hardest fix but ultimately the only solution. Students are forced to write arbitrary essays ultimately due to a belief that this teaches people "how to think". Ask what the justification for a humanities degree is, for example, and you'll probably get this answer.
If teachers are awarding good grades to AI generated text either their exercises aren't testing this ability, or AI can actually think and should be protected as a sentient life form. I think it's more likely that these essays aren't a good way to test thinking skills. What could work better? Something more direct, maybe? Testing people on logic puzzles, rationality tests, bias check tasks and so on. I definitely feel our society has major difficulties with rationality at scale, and we saw this during COVID in which people routinely conflated rationality/wisdom with blindly outsourcing their thinking to academics and civil servants, regardless of the underlying merits of the arguments those professors were actually making.
There's also the other goal of teaching people how to write, but being able to write well is largely about being able to think well, once you mastered the basics of spelling and grammar anyway. With modern AI driven spelling/grammar checkers it's also less important to deeply master the rules, and time can be better spent on the learning how to think and structure an argument aspect.
I'm glad I had a specific teacher in HS for English.
He had us do a positional paper over a topic we were passionate about. Topics were first come first serve. We got them approved. And when everyone was done getting topics, he then said "You are now arguing for the OPPOSITE of what you asked for."
He later explained why he did so. Obviously, the dept had paper requirements... But he also wanted us to learn a topic and viewpoints from all the angles, including ones you would normally dismiss. It was also a learning moment of "understand the issue before making opinions, and then form a well thought out decision".
In the 90’s there was a magazine article about Bill Gates. He interviewed some kids to hear what they want from computers. Doing their homework was the top answer.
I remember thinking: yeah right, that will take many decades. I guess 2-3 decades was enough!
How's this for an educational product idea: Use AI to read a student's essay, then produce exam questions based on the content. This test would be delivered orally (with accommodation for disabilities that might impact their ability to complete the task, of course).
The student should ace the test if they understood the subject well enough to ace an essay on it. They should know the answers very well, since the questions are specific to what they've apparently written. The mark they get on this test would be the research part of the mark they receive for the essay. Then it is just grammar, organization, etc. thereafter.
I wonder if this will affect the split between term papers and in-class essay exams. My wife is a professor and has historically assigned the former, on the theory that it's more flexible for students and there's no downside if the final is open-book.
But it seems that in-class essay exams may gain popularity if professors believe that students will use AI tools on term papers. Students who don't use tools like this might also advocate for (or simply select into classes that use) in-class exams.
I'm a copywriter. I've tested out several GPT-3-based services to see if they could speed up my workflow. None were up to my standards. I spent more time editing their semi-coherent output than it would have taken to just write the thing from scratch.
I have no doubt, though, that the day is quickly approaching when they will be of greater use and even threaten the livelihoods of people like me.
As a broke student who recently got started with copywriting to make ends meet, is there any way to email/message you with a question or two if you don't mind?
That’s really more of a business development question, and I’m truthfully much better at writing copy than running my business. Also, I’m unusual in that I get all my clients through word of mouth.
I stand out from the competition because I have technical writing skills most of them don’t have, and I’ve maintained a stable workload by developing a reputation for high quality, fast turn times, never missing deadlines or typos, and being easy to work with.
My suggestion to someone just starting out is to pump out content of the type you want to sell. Publish it on your social networks under your own name and ask your friends and colleagues to read/ share it.
Edit: BTW, if you want a quick, practical piece of advice: proofread with text to speech. Read along in your head as it reads your work. Some of the Google and Microsoft voice models are very lifelike and they help you catch instances where your brain is autocorrecting an error (which will happen).
Yeah, I know. The abundance of mediocre copywriters helps me maintain my client portfolio. I’ve had clients drop me for someone cheaper many times before, but they often come back after having a bad experience.
Physical blue books and an ink pen in an auditorium within a time limit. This would work easy enough for in-person (on-campus) students. Not so good for online/remote.
I took the LSAT some years ago (mostly just for curiosity's sake). When I turned my documents in to the lead proctor, she stopped me and said something like "you were to write your essay!"
Confused, I told her I had, to which she loudly stated "you printed your essay, and that's not permissible!" Once I figured out she meant cursive writing, I told her I hadn't written anything in cursive since maybe 4th or 5th grade, and after that I typed most of my long-form homework on my trusty Apple ][e and "printed" handwritten notes, as I still do today. She turned a bit pale and then turned to the room, interrupted the test, and asked how many candidates were also "printing" their essays. Essentially everyone raised their hands.
It turns out the poor older lady was a grade school teacher and had been instructing children how to write in cursive since the days of yore. When she faced the fact that those habits hadn't stuck with any of the once-4th-graders sitting in front of her, she turned even paler and went to get a drink of water. I felt a bit bad for her.
I've tried doing this myself for company related compliance testing which mostly is bullshit anyway and doesn't apply to me. GPT3 is better used as a tentative backup tbh because it sometimes gives wrong answers or strange answers similar to how the images and video produced from Stable Diffusion and the like are strange and dreamlike or just impossibly wrong.
How is it good? Writing essays teaches writing and also reading. It also teaches thinking critically about your writing and your reading. They are not pointless at all. It's exactly the same as taking code off the internet instead of trying to solve an algorithmic riddle. You may learn something new, but the problem solving process itself is lost and not developed.
18-20 years ago I would just put randomly generated gibberish in my papers after making my point and showing competence in the topic but not having met the minimum word requirements.
The educational attainment from my peers was so low that teachers were pretty understanding and gave high marks because I did the point of the assignment.
Reminds me of that, but GPT-3 is pretty impressive.
I think we're going to see the return of the oral exam (i.e. You show up to the prof's office and they ask you questions on the spot.). Perhaps not for massive low level courses, but likely for advanced courses.
They're going to be the bane of the timid, but they do provide ample motivation to study and require elaborate measures to cheat on.
There's a limit to how well you can know your students' writing styles if your classes are large. If this is something we want to change, the obvious path is to stop expecting a single teacher to grade 150 essays in a weekend.
Fire 20 admin, hire 40 teachers (more or less, depending on the size of you district), then see how many AI's slip through.
> pointless essays that ask them to regurgitate content (as opposed to essays that teach writing skills or critical thinking, which remain valuable).
It's all just a question of time. GPT-3 can regurgitate content, GPT-4 will exhibit writing skills, and GPT-5 will write with critical thinking... at least on the level that an average student does today.
Are we sure the language models won't do just as well writing essays requiring writing skills or critical thinking? What about a couple years from now?
Getting a good grade on a class assignment is getting pretty darn close to passing the Turing test.
Putting this out there: like most machine generated "art" these days, the relative increase of output quality is in part marked by a decline in both mean human output quality and the mean ability of humans to perceive quality.
> “For biology, we would learn about biotech and write five good and bad things about biotech. I would send a prompt to the AI like, ‘what are five good and bad things about biotech?’ and it would generate an answer that would get me an A.”
I don't call that an essay, and it's something that can be done with a 10 seconds google search without AI.
Narayanan (~randomwalker) is writing a book about, & with likely title, "AI Snake Oil".
Of course there's plenty of BS around the ___domain – every epochal advance arrives inside a phalanx of pretenders & hypesters & scammers.
But the rapid progress in the field risks nearly any tangible statements about "AI that does not and cannot work" – the book's stated theme – becoming invalidated by on-the-ground events between composition & dead-tree printing.
Thus this viewpoint will more and more be driven towards a sort of timeless mood of generic skepticism & even denialism. There'll be a big audience for that. It's comforting. It meets people at the limits of their understanding. It assuages their fears.
But like a head-of-lettuce, it won't age well, except in comparison to even shorter tenures.
I think this 9-para, ~500 word… 'essay' – dare I call it that? – is an example of the trap Narayanan finds himself in.
Sure, essay assignments have all sorts of long-recognized limitations, dutifully recited here, as could have also been said in the 1990s, or 1960s, or 1930s. But their longevity in education – their 'lindyness' – suggests they had some value, as both exercise & evaluation. No easy replacement was found in the leisurely decades (centuries?) they were relied-upon.
Now, through K-12, they're done-for, in just about any case where a student remains unobserved, with access to LLM writing-assistance. In another 2-5 years, LLMs will not just be writing A+ Senior-in-High-School-level essays, but postgraduate star-student level papers.
Against that, Narayanan offers hand-wavy bluster: that somehow, in unstated ways, the "teachers and adjunct professors [who] are underpaid and overworked", who already in the ancestral environment had to depend on the "easier" but "mind-numbing" and "easier to grade" essay-assignments, will somehow now figure out a better approach, in a more-difficult environment, where their go-to solution for generations has been yanked away.
Sure, little Timmy has never swam before, but throw him into the deep end. His problem isn't "callousness or incompetence", so he'll figure it out! Narayanan's understatement: "The adjustment will be painful, for sure."
To the extent there will be adaptations, they are unlikely to come from the AI-denialists, who underplay access to these tools as no-big-deal.
Adaptations could include uncomfortable steps already on the rise, like closely surveilling students in tech-excluded environments, to be sure they practice, & can perform, those thinking-steps that we'd rather they not fully outsource to cheap thinking-substitutes.
Or sessions with AI tutors, who can drill students in more-intensive ways than those "underpaid and overworked" teachers in large institutions have historically managed.
Workable solutions will more likely be inspired by Stephenson's (1995) 'The Diamond Age: Or, A Young Lady's Illustrated Primer" than Narayanan's (2023?) 'AI Snake Oil'.
OP here. The full title of this article is "Students are acing their homework by turning in machine-generated essays. Good."
The last word was edited out by the mods, presumably under the belief that it's clickbait. Unfortunately, the headline now sounds like I'm complaining about this development, whereas my post is about how it will force much-needed improvements to education and free students from the drudgery of pointless essays that ask them to regurgitate content (as opposed to essays that teach writing skills or critical thinking, which remain valuable).
The problem is that the title was linkbaity, especially with that "Good" at the end. For a mod to edit that out is routine HN moderation ("Please use the original title, unless it is misleading or linkbait" - https://news.ycombinator.com/newsguidelines.html)
Perhaps a better fix is to use a representative sentence from the article body. I've done that now.
Those things are also clickbait and we also edit them out when we see them. Are you perhaps assuming that we see everything that gets posted here? That would be very mistaken.
>whereas my post is about how it will force much-needed improvements to education and free students from the drudgery of pointless essays that ask them to regurgitate content (as opposed to essays that teach writing skills or critical thinking, which remain valuable)
OK ... what's wrong with essays that demonstrate knowledge of a particular topic (as you call it "regurgitate content')? And why wouldn't those teach writing skills?
Yes, developing critical thinking skills is important, and sometimes you want to focus projects, assignment, homework towards that end. But don't discount the value of being able to synthesize and summarize existing knowledge in a particular area of knowledge. In fact, that's almost always a pre-requisite to making cogent arguments that exercise 'critical thinking' skills.
>In fact, it seems to be this kind of essay where language models are doing particularly well, with assignments such as “Write five good and bad things about biotech”. As an educator, I think this assignment is close to useless if the goal is to learn about biotech.
WHY?? Why is it 'close to useless' for a student to investigate current issues in biotech?
And by the way, with academics (and especially in public education), it is almost always the case that a student gets out of it what they put in. That is, if the student is aiming for the absolute minimum and takes every shortcut, neither AI, nor the assignment structure will a make a difference. Going back to this 'close to useless' question, a keen student can really sink their teeth into it and make this topic their own - because this question obviously is open-ended, and leaves room for the student to provide an independent and critical evaluation of the issues that concern the field .... OR ... they can spend 15 mins googling around or using AI, to throw a bunch of stuff together, call it a day, and go back to playing Call of Duty.
IMO I didn't get the sense that the title is complaining. I took the title as neutral statement and assumed the article would provide evidence for the claim and possibly commentary on it.
IME, and I have lots of "E" (including training and finetuning my own models), this probably isn't entirely true in most writing-heavy courses. You can use the largest language models, in an interactive fashion, to generate portions of half-decent papers. Maybe even "ace" papers on certain subjects with certain grading criteria (namely, "looks reasonable", not "fine-toothed comb"). But in many courses the essays will be extremely low-quality, and even in the happy cases saying that the essay is "machine-generated" is eliding a lot of manual effort.
I agree that students are probably using LLMs for their homework, but I'm skeptical that they are all getting As on assignments that are designed as big assessments, or that the essays are actually fully machine-generated. I bet a lot of students -- the laziest ones -- are getting "WTF is this essay even about... did you have a stroke while writing this?!" feedback if they are using LLMs to generate essays whole-cloth.
Pedagogically, this matters. Think about calculator usage. There's a huge difference between allowing use of TI-83 on Calculus assignment with lots of word-heavy application problems and allowing use of Wolfram Alpha on a Calculus assignment that's "integration recipe practice".