926
College professors are going back to paper exams and handwritten essays to fight students using ChatGPT
(www.businessinsider.com)
this post was submitted on 13 Aug 2023
926 points (97.7% liked)
Technology
60123 readers
5102 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Here's a somewhat tangential counter, which I think some of the other replies are trying to touch on ... why, exactly, continue valuing our ability to do something a computer can so easily do for us (to some extent obviously)?
In a world where something like AI can come up and change the landscape in a matter of a year or two ... how much value is left in the idea of assessing people's value through exams (and to be clear, I'm saying this as someone who's done very well in exams in the past)?
This isn't to say that knowing things is bad or making sure people meet standards is bad etc. But rather, to question whether exams are fit for purpose as means of measuring what matters in a world where what's relevant, valuable or even accurate can change pretty quickly compared to the timelines of ones life or education. Not long ago we were told that we won't have calculators with us everywhere, and now we could have calculators embedded in our ears if wanted to. Analogously, learning and examination is probably being premised on the notion that we won't be able to look things up all the time ... when, as current AI, amongst other things, suggests, that won't be true either.
An exam assessment structure naturally leans toward memorisation and being drilled in a relatively narrow band of problem solving techniques,^1^ which are, IME, often crammed prior to the exam and often forgotten quite severely pretty soon afterward. So even presuming that things that students know during the exam are valuable, it is questionable whether the measurement of value provided by the exam is actually valuable. And once the value of that information is brought into question ... you have to ask ... what are we doing here?
Which isn't to say that there's no value created in doing coursework and cramming for exams. Instead, given that a computer can now so easily augment our ability to do this assessment, you have to ask what education is for and whether it can become something better than what it is given what are supposed to be the generally lofty goals of education.
In reality, I suspect (as many others do) that the core value of the assessment system is to simply provide a filter. It's not so much what you're being assessed on as much as your ability to pass the assessment that matters, in order to filter for a base level of ability for whatever professional activity the degree will lead to. Maybe there are better ways of doing this that aren't so masked by other somewhat disingenuous goals?
Beyond that there's a raft of things the education system could emphasise more than exam based assessment. Long form problem solving and learning. Understanding things or concepts as deeply as possible and creatively exploring the problem space and its applications. Actually learn the actual scientific method in practice. Core and deep concepts, both in theory and application, rather than specific facts. Breadth over depth, in general. Actual civics and knowledge required to be a functioning member of the electorate.
All of which are hard to assess, of course, which is really the main point of pushing back against your comment ... maybe we're approaching the point where the cost-benefit equation for practicable assessment is being tipped.
Ah the calculator fallacy; hello my old friend.
So, a calculator is a great shortcut, but it's useless for most mathematics (i.e. proof!). A lot of people assume that having a calculator means they do not need to learn mathematics - a lot of people are dead wrong!
In terms of exams being about memory, I run mine open book (i.e. students can take pre-prepped notes in). Did you know, some students still cram and forget right after the exams? Do you know, they forget even faster for courseworks?
Your argument is a good one, but let's take it further - let's rebuild education towards an employer centric training system, focusing on the use of digital tools alone. It works well, productivity skyrockets, for a few years, but the humanities die out, pure mathematics (which helped create AI) dies off, so does theoretical physics/chemistry/biology. Suddenly, innovation slows down, and you end up with stagnation.
Rather than moving us forward, such a system would lock us into place and likely create out of date workers.
At the end of the day, AI is a great tool, but so is a hammer and (like AI today), it was a good tool for solving many of the problems of its time. However, I wouldn't want to only learn how to use a hammer, otherwise how would I be replying to you right now?!?
I found this too generalizing. Yes, most people only ever need and use productivity skills in their worklife. They do no fundamental research. Wether their education was this or that way has no effect on the advancement of science in general, because these people don't do science in their career.
Different people with different goals will do science, and for them an appropriate education makes sense. It also makes sense to have everything in between.
I don't see how it helps the humanities and other sciences to teach skills which are never used. Or how it helps to teach a practice which no one applies in practice. How is it a threat to education when someone uses a new tool intelligently, so they can pass academic education exams? How does that make them any less valuable for working in that field? Assuming the exam reflects what working in that field actually requires.
I think we can also spin an argument in the opposite direction: More automation in education frees the students to explore side ideas, to actually study the field.
"I don’t see how it helps the humanities and other sciences to teach skills which are never used." - I can offer an unusual counter here, you're assuming the knowledge will never be used, or that we should avoid teaching things that are unlikely to be used. Were this the case, the field of graph theory would have ceased to exist long before it became useful in mapping - indeed Bool's algebra would never have led to the foundations of computer science and the machines we are using today.
"How is it a threat to education when someone uses a new tool intelligently, so they can pass academic education exams?" - Allow me to offer you the choice of two doctors, one of whome passed using AI, and the other passed a more traditional assessment. Which doctor would you choose and why? Surely the latter, since they would have also passed with AI, but the one without AI might not have passed the more traditional route due to a lack of knowledge. It isn't a threat to education, it's adding further uncertainty as to the outcome of such an education (both doctors might have the same skill levels, but there is more room for doubt in the first case).
"Wether their education was this or that way has no effect on the advancement of science in general, because these people don’t do science in their career." - I strongly disagree! In an environment where knowledge for the sake of knowledge is not prised, a lie is more easy plant and nurture (take for example the antivax movement). Such people can be an active hinderence to the progress of knowledge - their misconceptions creating false leads and creating an environment that distrusts such sciences (we're predisposed to distrust what we do not understand).
Not exactly. What I meant to say is: Some students will never use some of the knowledge they were taught. In the age of information explosion, there is practically unlimited knowledge 'available'. What part of this knowledge should be taught to students? For each bit of knowledge, we can make your hypothetic argument: It might become useful in the future, an entire important branch of science might be built on top of it.
So this on it's own is not an argument. We need to argue why this particular skill or knowledge deserves the attention and focus to be studied. There is not enough time to teach everything. Which in turn can be used as an argument to more computer assisted learning and teaching. For example, I found ChatGPT useful to explore topics. I would not have used it to cheat in exams, but probably to prepare for them.
Good point, but it depends on context. You assume the traditional doc would have passed with AI, but that is questionable. These are complex tools with often counterintuitive behaviour. They need to be studied and approached critically to be used well. For example, the traditional doc might not have spotted the AI hallucinating, because she wasn't aware of that possibility.
Further, it depends on their work environment. Do they treat patients with, or without AI? If the doc is integrated in a team of both human and artificial colleagues, I certainly would prefer the doc who practiced these working conditions, who proved in exams they can deliver the expected results this way.
I feel we left these lands in Europe when diplomas were abandoned for the bachelor/master system, 20 years ago. Academic education is streamlined, tailored to the needs of the industry. You can take a scientific route, but most students don't. The academica which you describe as if it was threatened by something new might exist, but it lives along a more functional academia where people learn things to apply them in our current reality.
It's quite a hot take to paint things like the antivax movement on academic education. For example, I question wether the people proposing and falling for these 'ideas' are academics in the first place.
Personally, I like learning knowledge for the sake of knowledge. But I need time and freedom to do so. When I was studying computer science with an overloaded schedule, my interest in toying with ideas and diving into backgrounds was extremely limited. I also was expected to finish in an unreasonably short amount of time. If I could have sped up some of the more tedious parts of the studies with the help of AI, this could have freed up resources and interest for the sake of knowledge.
Yes, within limits. Due to the information explosion, it became impossible to learn "everything". We need to make choices, prioritize.
How does your voting behaviour suffer because you lack understanding about how exactly potentiometers work, or how to express historic events in modern dance?
Both have inherent worth, but not the same for each person and context. We luckily live in a society of labor division. Not everyone has to know or like everything. While I absolutely admire science, not everyone has to be a scientist.
Because there is more knowledge available than we can ever teach a single person, it is entirely possible to spend a lifetime learning things with no use informing your ballot decision. I would much rather have students optimize some parts of their education with AI, to free up capacity for other important subjects which may seem less related to their discipline. For example, many of my fellow computer science students were completely unaware how it could be ethically questionable to develop pathfinding algorithms for military helicopters.
While all of what you say is true, we simply cannot teach everything since there is just too much knowledge and too little time in a human life.
And not everyone is equally interested or capable in learning everything.
This is necessarily the world we live in, even without adding capitalism or any evil intentions to the mix. Any education you can get or offer can only be a more or less well selected subset of the knowledge available.
In this light, I don't see it as a dramatic loss to remove educational emphasis from skills which can easily be replaced with modern technology. It would make sense to shift the focus to teaching a critical usage of said technology.
So ... I honestly think this is a problematic reply ... I think you're being defensive (and consequently maybe illogical), and, honestly, that would be the red flag I'd look for to indicate that there's something rotten in academia. Otherwise, there might be a bit of a disconnect here ... thoughts:
calculatorwas in reference to arithmetic and other basic operations and calculations using them ... not higher level (or actual) mathematics. I think that was pretty clear and I don't think there's any "fallacy" here, like at all.value of learning (actual) mathematicsis pretty obvious I'd say ... and was pretty much stated in my post about alternatives to emphasise. On which, getting back to my essential point ... how would one best learn and be assessed on their ability to construct proofs in mathematics? Are timed open book exams (and studying in preparation for them) really the best we've got!?Still forgetting with open book exams... seems like an obvious outcome as the in-exam materials de-emphasise memory ... they probably never knew the things you claim they forget in the first place. Why, because the exam only requires the students to be able to regurgitate in the exam, which is the essential problem, and for which in-exam materials are a perfect assistant. Really not sure what the relevance of this point is.Forgetting after coursework... how do you know this (genuinely curious)? Even so, course work isn't the great opposite to exams. Under the time crunch of university, they are also often crammed, just not in an examination hall. The alternative forms of education/assessment I'm talking about are much more long-form and exploration and depth focused. The most I've ever remembered from a single semester subject came from when I was allowed to pursue a single project for the whole subject. Also, I didn't mention ordinary general coursework in my post, as, again, it's pretty much the same paradigm of education as exams, just done at home for the most part.Rebuilding education toward employer centric training system... I ... ummm ... never suggested this ... I suggested the opposite ... only things that were far more "academic" than this and were never geared toward "productivity". This is a pretty bad staw man argument for a professor to be making, especially given that it seems constructed to conclude that the academy and higher learning are essential for the future success of the economy (which I don't disagree with or even question in my post).OK Mr Socrates how else would you assess whether a student has learned something?
Ha ... well if I had answers I probably wouldn't be here! But seriously, I do think this is a tough topic with lots of tangled threads linked to how our society functions. I'm not sure there are any easy "fixes", I don't think anyone who thinks that can really be trusted, and it may very well turn out that I'm completely wrong and there is not "better way", as something flawed and problematic may just turn out to be what humanity needs.
A pretty minor example based on the whole thing of returning to paper exams. What happens when you start forcing students to be judged on their ability to do something, alone, where they know very well that they can do better with an AI assistant? Like at a psychological and cultural level? I don't know, I'm not sure my generation (Xennial) or earlier ever had that. Even with calculators and arithmetic, it was always about laziness or dealing with big numbers that were impossible for (normal humans), or ensuring accuracy. It may not be the case that AI is at that level yet for many exams and students (I really don't know), but it might be or might be soon. However valuable it is to force students to learn to do the task without the AI, there's gotta be some broad cultural effect in just ignoring the super useful machine.
Otherwise, my general ideas would be to emphasise longer form work (which AI is not terribly useful for). Work that requires creativity, thinking, planning, coherent understanding, human-to-human communication and collaboration. So research projects, actual practical work, debates, teaching as a form of assessment etc. In many ways, the idea of "having learned something" becomes just a baseline expectation. Exams, for instance, may still hold lots of value, but not as forms of objective assessment, but as a way of calibrating where you're up to on the basic requirements to start the real "assessment" and what you still need to work on.
Also ...
OK Mr Socrates... is maybe not the most polite way of engaging here ... comes off as somewhat aggressive TBH.I think a central point you're overlooking is that we have to be able to assess people along the way. Once you get to a certain point in your education you should be able to solve problems that an AI can't. However, before you get there, we need some way to assess you in solving problems that an AI currently can. That doesn't mean that what you are assessed on is obsolete. We are testing to see if you have acquired the prerequisites for learning to do the things an AI can't do.
My theory prof said there would be paper exams next year. Because it's theory. You need to be able to read an academic paper and know what theoretical basis the authors had for their hypothesis. I'm in liberal arts/humanities. Yes we still exist, and we are the ones that AI can't replace. If the whole idea is that it pulls from information that's already available, and a researcher's job is to develop new theories and ideas and do survey or interview research, then we need humans for that. If I'm trying to become a professor/researcher, using AI to write my theory papers is not doing me or my future students any favors. Ststistical research on the other hand, they already use programs for that and use existing data, so idk. But even then, any AI statistical analysis should be testing a new hypothesis that humans came up with, or a new angle on an existing one.
So idk how this would affect engineering or tech majors. But for students trying to be psychologists, anthropologists, social workers, professors, then using it for written exams just isn't going to do them any favors.
I also used to be a humanities person. The exam based assessments were IMO the worst. All the subjects assessed without any exams were by far the best. This was before AI BTW.
If you’re studying theoretical humanities type stuff, why can’t your subjects be assessed without exams? That is, by longer form research projects or essays?
As they are talking about writing essays, I would argue the importance of being able to do it lies in being able to analyze a book/article/whatever, make an argument, and defend it. Being able to read and think critically about the subject would also be very important.
Sure, rote memorization isn't great, but neither is having to look something up every single time you ever need it because you forgot. There are also many industries in which people do need a large information base as close recall. Learning to do that much later in life sounds very difficult. I'm not saying people should memorize everything, but not having very many facts about that world around you at basic recall doesn't sound good either.
That's an interesting point I probably take for granted.
Nonetheless, exercising memory is probably something that could be done in a more direct fashion, and therefore probably better, without that concern affecting the way we approach all other forms of education.
It's an interesting point.. I do agree memorisation is (and always has been) used as more of a substitute for actual skills. It's always been a bugbear of mine that people aren't taught to problem solve, just regurgitate facts, when facts are literally at our fingertips 24/7.
Yea, it isn’t even a new problem. The exam was questionable before AI.
While I do agree with your initial point (that memorization is not really the way to go with education, I've hated it for all my life because it was never a true filter - a parrot could pass university level tests if trained well enough), I will answer your first point there and say that yes, it is important to know where Yugoslavia was, because politics was always first and foremost influenced by geography, and not just recent.
Without discussing the event mentioned itself, some points to consider:
The cultural distribution of people - influenced by geography - people on the same side of the mountain or river are more likely to share the same culture for example. Also were there places easily. Were they lands easily accessible to conquering armies and full of resources? Have some genocide and replacement with colonizers from the empire - and the pockets of 'natives' left start harboring animosity towards the new people.
Spheres of influence throughout history - arguably the most important factor - that area of Europe has usually been hammered by its more powerful neighbours, with nations not posessing adequate diplomacy or tactics being absorbed or into or heavily influenced by whatever empire was strongest at the time - Ottoman Empire, USSR, Roman Empire if we want to go that far into history. So I would say hearing 'Yugoslavia was in South East Europe' would immediately prompt an almost instinctual question of 'Oh, what terrible things happened there throughout history, then?' for one familiar with that area, thereby raising this little tidbit to one of the top facts.
We could then raise the question of what would have happened to the people had they been somewhere else? History is written by the victors and the nasty bits (like sabotage and propaganda to prevent a certain geographically nation from becoming too powerful) are left out.
My geopolitics game isn't that strong but I'm going to go out on a limb here and say that if the Swiss weren't in the place they are, they would probably not be the way they are (no negative nuance intended). Living in a place that's hard to invade tends to shape people differently than constantly looking over your shoulder.
And reading your second point, I'm understanding about what I wrote in this wall of text. Odd.
Yea ... we're on the same page here (I think). All the things you're talking about are the important stuff, IMO. "Yugoslavia is in south eastern Europe" doesn't mean much, even if you can guess something about the relatively obvious implications of that geography, as you say. But those implications come from somewhere, some understanding of some other episode of history. Or it could come form learning about Yugoslavia's and the Balkan's history. For instance, you might note from the location this it's relatively close to Turkey, but that wouldn't lead you to naturally expect a sizeable Islamic population in the region (well I didn't at first), unless you really knew the Ottoman history too. So there's a whole story to learn there of the particular cultural make up of the place and where it comes from and how that leads to cultural tensions come the Yugoslavian wars. In learning about that, you can learn about how far away the Ottoman empire was and where its borders got to over time, where the USSR was and the general ambit of Slavic culture etc. Once you've a got a story to tell, those things become naturally important and memorable.
And now I've added my own wall of text ... sorry. So ... yes! I agree! Both of our walls of texts are (loosely) about the important stuff, with facts sure, but motivated by and situated in history (though there's obviously a fuzzy line there too!)
Inflection? I used the present tense of
be/isfor past events ... that's a tense or conjugation. No inflections occur with irregular verbs like that. Are you not on top of your grammar?You are getting some flak, but imho you are right. The only thing an exam really tests is how well you do in exams. Of course, educators dont want to hear that. But if you take a deep dive into (scientific) literature on the topic, the question "What are we actually measuring here?" is raised rightfully so.
Getting flak on social media, through downvotes, can often (though not always!) be a good thing ... means you're touching a nerve or something.
On this point, I don't think I've got any particularly valuable or novel insights, or even any good solutions ... I'm mostly looking for a decent conversation around this issue. Unfortunately, I suspect, when you get everyone to work hard on something and give them prestigious certifications for succeeding at that something, and then do this for generations, it can be pretty hard to convince people to not assign some of their self-worth to the quality/value/meaning of that something and to then dismiss it as less valuable than previously thought. Possibly a factor in this conversation, which I say with empathy.
Any links to some literature?
Used only papers in german so far, sadly.
Here is something I found interesting in english:
Testing the test: Are exams measuring understanding? Brian K. Sato, Cynthia F. C. Hill, S. Lo Biochemistry and Molecular Biology Education
in general: elicit.org
really good site.
Hadn't heard of that
elicitcite ... thanks! How have you found it? It makes sense that it exists already, but I hadn't really thought about it (haven't looked up papers recently but may soon).Also thanks for the paper!!
Have found it relatively early after it was created, using it for getting a quick overview over papers when writing my own. It is sooo good for that.
Yea cool. Thanks for this!
In my experience, they love to give exams where it doesn't matter what notes you bring, you're on the same level whether you write down only the essential equations, or you copy down the whole textbook.