I had to do "write code on paper" stuff as part of french engineering school entrance exams.
It's fine (tho annoying when you lose points to "typos"), but it limits what kinds of problems you can reasonably put on the exam. You'll definitely lean a bit more into theoretical stuff than practicals. Which is fine for some courses, I think a bit less interesting in other courses.
Remember, the hand written code is also harder for reviewers to grade! You have to manually run the code in your head, for example
Having said all that... "we've booked the computer room, you don't have internet, go type up all your stuff in this VM we have set up" feels fine if you don't like this constraint IMO
GP is talking about every test being done on a computer presumably to avoid handwriting. Handwriting isn’t the usual “get with the times, grandpa” situation. It’s been proven again and again that writing by hand develops the brain and helps store that new knowledge better. It would be a disservice to the students to only use typing. Or TikTok shorts, or whatever is the tech of the day.
Exams shouldn’t probably be used for learning new information and typing is a lot more efficient and allows editing and revising so there are advantages to that.
> or whatever is the tech of the day.
Rote memorisation has been the standard technique probably for way longer that anything else so I’m not sure that every innovation is inherently inferior to what we had before
Handwriting is still “new information” by the time you are in university? How do you feel about reading when text to speech and LLM summarization are so advanced now?
No wonder some people are so easy to replace with an LLM.
>It's fine (tho annoying when you lose points to "typos"),
this would also end up valorizing particular ways of thinking or developing, so those whose development methodology is more exploratory until you get a certain point where you get an overarching view of how everything should interact and then go into a quicker iterative mode would be penalized.
I guess it's ok it does this, some things after all must be valorized whatever is chosen, although I've known many people who are quite good using this way of working who would then be downgraded.
Meanwhile I had to do VHDL to design a chip and then write code for it in paper, and get the actual solution the software that was meant to run in my imaginary processor to work, detailing both the code and what we thought our code was doing. Teacher and TAs would later make it run in a Xilinx to grade and make themselves available at a few hours if you wanted to discuss your grade/solution with them at a later time with the thing actually running in a board.
We had 3 hours to make the test. The papers the teacher delivered had differences between them (I think it was four different versions of the test?). The Teacher didn’t stay in the class also, you had to deliver the test to them later in their office.
If you're doing a good job, you have to do that anyway, or at least have enough of a spidey sense for broken code to know when to investigate and add an extra test case.
Something like 30% of the time at $WORK, interviewers report the candidate as having solved the problem when a closer inspection reveals UB, memory corruption, and other bullshit. The test cases pass, and I think that's part of the problem. You can't tune out and avoid deeply understanding the submission.
I'm fine with paper tests as a tradeoff, but in the real world where I am paid to program, I am a strong advocate of advanced type checkers/compilers and other static code analysis tools because they can keep all the code "in their head." They can keep far, far more code "in their head" than even the most gifted programming savants. Literally as I type out a mistake, the type checker alerts me to it.
So in some theoretical hiring example, if I have to choose between two candidates: one which is really good at keeping the code in the head and the other who is less good at that, but who is very skilled at using programming tools like advanced type systems or formal verification tools, then I'd prefer the latter.
Of course in practice, I would of course take a student that I knew had passed their paper test with their own knowledge over a student that likely ran their digital test through an LLM.
I would imagine every professor (or assistant) who graded any programming exam I wrote was doing just that and further expect them to let some things slide as long as the general direction was correct.
This would be HTR (Handwritten Text Recognition) not OCR and HTR is a lot harder than OCR especially for modern scripts (i.e. student's scrawls). And at every error the reviewer now needs to check if it was due to an error in the code or due to a bad HTR somewhere in the code.
I'd say compared to just typing the code on a disconnected computer: Not worth it.
Yes. That's exactly what college is. If you don't have the resources to effectively individually evaluate a student's output, you shouldn't hold a graded class.
Then let me spell it out for you, since you seem to be unaware:
There are many, many colleges that have no grad students. Period. They are undergraduate colleges.
Expecting any college or university to just be able to scale up the amount of manpower required to grade their classes/exams by a large multiple is absurdly unrealistic. I assure you that most existing colleges do have plenty of resources to effectively evaluate individual students' outputs, if they don't have to assume that anything submitted digitally is written by LLMs.
That new requirement is a massive change to how they operate, and it's incredibly rude and short-sighted to be dismissive and contemptuous of institutions that have trouble adjusting to it.
These colleges certainly have English, Philosophy, Art, and History classes, all of which require papers to be submitted and the exams consist of essay questions.
That Computer Science is somehow a privileged subject where the exam grading must be automated is elitism of the highest degree.
I graduated from a not so incredible university and we had multiple such rooms. Teaching assistants and some tutors helped supervising the exams and it wasn't easy to cheat without getting caught.
Contestant computers boot a Linux image that contains all the interpreters/compilers and documentation for the supported languages, some editors/IDEs(vim, Jetbrains, Eclipse, etc) and a web browser.
Network access is limited to the judging system only.
My experience in Europe is that most universities can support this.
Jfc, this is a solved problem at the community college near me. All of the computers in the testing room are thin clients that effectively remote into a vm, and you get checked with a metal detector before going into the room with the computers.
If you can escape the vm on the fly and manage to use an llm to cheat, you deserve that A.
IME, that kind of nitpicky grading mostly happens when the exam was designed too easy and does not depend on the medium.
> Remember, the hand written code is also harder for reviewers to grade! You have to manually run the code in your head, for example
Which you really need to do anyway because code that is broken due to a brainfart and code that is broken because of a fundamental misunderstanding should not receive the same grade.
I'm quite a bit older than you, old enough that I remember learning to touch type in elementary school on Apple IIe and IIgs desktop computers. It's not reasonable these days to expect people to hand write a 4, 6, 8 page length essay on paper and pen with a finite time limit in a classroom. Being able to edit and revise things in a word processor type interface is an essential part of writing an academic paper.
Additionally expecting whoever is reading the paper to comprehend everyone's (likely very sloppy, in this era) handwriting is an exercise in frustration for the person who would be evaluating the papers.
Not that tests/exams can't be given on paper, ever (multiple choice still works), but for something where people are expected to provide multiple pages of coherently written essay output, I would struggle to do it by hand. And I'm old enough that we did do a lot purely on paper when I was in school.
My child's high school is doing the same thing: all exams are now handwritten on paper in a supervised room. Phones and smartwatches are always banned during the school day, but laptops are banned during exams. This is standard at state-funded high schools in Australia.
There will likely be a period where those who went through high school with computers struggle with hand writing stuff, but the next generation will have done it all their lives.
If you did it before then, barring physical limitations which have occurred since then, you would struggle for a short while and then you would be fine. I also did coursework purely on paper, got out of practice, and then once my kids were in school got back into the habit of handwriting. I can even write cursive again. Cursive makes it much easier on the hand and wrist. There's a reason it was invented. :)
Being left-handed I've done left-handed cursive for far more than 5 months. I pushed myself to also try and learn to write right-handed. In a left-to-right writing style, its more than just slanting the letters slightly the other way. In the end, the right-hander is overall dragging the pen across the page while the lefty is overall pushing the pen across the page.
I have never been able to handwrite well and my family suspects I have dysgraphia. I can write, but slowly and unreadably. But I can type over 160 wpm. Typing for me is strictly superior, probably by an order of magnitude.
I think you're missing the point. This is a physical limitation. People don't write as much as they used to. I used to be able to write for hours, taking notes, writing papers, etc... Now, I try to take notes during a meeting and my hand cramps. And as a bonus, my handwriting never was great, but now it's illegible for me.
Yes, we all "could" re-learn this skill, but how many people will? If you asked me to type a paper on a locked-down computer, I could easily. If you asked me to write a 2000 word paper by hand in an hour, I doubt it would happen.
If we expect students to take in-person tests on paper, then you should also do the rest of the classwork on paper. I am completely in the camp that we learn more when we write something. The physical act is part of the learning process. But you can't expect students to write an in-depth exam without having the practice of doing it often.
From what I recall, doing handwritten CS exams in 2004, you aren't writing long essays. Long form was the course work. The exam was short form answers and it did not stress my writing hand. The one with most writing was the natural language exam (an extinct subject I should think) and I felt more time worry over having to flip back and forth between pages to cross reference than time spent writing.
> It's not reasonable these days to expect people to hand write a 4, 6, 8 page length essay on paper and pen with a finite time limit in a classroom.
Say what? Writing by hand is not only useful to write. It helps practice fine motor control. Something you need in many places in life, and most people have no practice of other than by writing.
I've sat exams a few years after university (for a tentative career change) and I can tell you I'd forgotten how genuinely tiring writing so much by hand was. I've made sure to write more regularly since, just in case I change from software engineering to something that requires more handwriting.
Well, how many times in that 9 years have you written on paper for 2 hours straight? Even as a kid who did it regularly, it sucked.
Doing it now I really don't think I could deliver my intellectual best while worrying about if anyone can read my handwriting and whether I'm gonna cramp up by the end of the exam.
Pen and paper is just not a very good way to produce text.
Well to be fair humans had to suffer through all kinds of miserable practices over those years. Modern style of writing as we understand it didn’t really develop until the 800s or so (if not way later).
So it’s like saying that clay tablets worked just fine for 1000+ years why would we switch to papyrus? Or who needs character for vowels? Or spaces between words, it’s not like anyone reads texts silently?
No? Some subsets of the world has papyrus I guess but for most people during most of that time people were pressing text into clay and wax and stuff, it must have fucking sucked.
Then we got paper and pens and that was a pretty decent interim for a short period. Then about 100 years ago we got typing. Then about 20 years ago we reached a world where almost everyone is better at typing than they are at writing.
Obviously it's still important that people can write by hand, but expecting people to do it for more than a few hundred words at a time is idiotic. Would you like your clothes to be hand sewn too? That also worked for thousands of years (much longer than writing) but we stopped doing it for a very good reason.
I'm much older. I've always been able to type faster than I can write by hand. Forcing me to handwrite an answer slows me down -- and produces something that's much harder to read.
I’m 40 and forcing students to do handwritten essays during tests has always been stupid. Typing is much faster, why bottleneck ideas by forcing handwriting?
I'm 60+. I'd be more concerned about the student's physical ability to write for several hours continuously. Writer's cramp used to be a problem, and that was when we were used to hand-writing everything. Legibility is also a consideration: I have to hand-write a lot(keyboards would not be socially acceptable for some of my work), and even with decades of practice and a hand that I designed for legibility, sometimes I have difficulty reading my own writing.
I have absolutely no concern over a 20 yr old's ability to weather the rigors of manipulating a 7 gram pencil. It's not like we're talking about getting them to spend a week on a roofing crew or swap their gaming mouse for a set of post hole diggers here. If someone needs an accommodation then that should absolutely be made available.
Nothing to do with weight. It's a cramp in the muscles of the hand from holding the pen and making fine movements for hours. I'm guessing you are from a generation after keyboard use became common, so you haven't encountered it.
Right and wrong. I am of a generation before common keyboard usage. As far as writers cramp goes, we all got it from time to time. At worst it was a minor nuisance compared to the skinned knees, bruises, and occasional broken bone that we racked up recreationally.
I'm all for handwritten tests, but it's more complicated than that. If you're actually writing for 2hr+ and haven't studied appropriate technique or bought some sort of crutch like a pencil holder then the repetitive motion will absolutely cause cramps for a fraction of the class regardless of being 20ish and healthy, and they might not find that out until they're forced to write for 2hr. The muscles manipulating a pencil (with poor technique) are much smaller than those manipulating a post hole digger, so that comparison isn't fair.
Not in my experience, nobody complained of that during my studies. 74 exams over 5 years ca. 2h long, also 2h long lectures, often without pause, back-to-back with another such lecture (but max 2 in a row)
A good exam also has breaks between writing because you need them to think about what you are going to write. An exam that is just about writing down words as fast as you can is going to suck whether that's handwritten or with a keyboard.
We have to notice that high stakes exams on paper worked for hundreds, possibly thousands of years, and high stakes exams on personal laptops have been tried for approximately six years and worked for none of them. With that framing, I don’t think the burden of proof is on the side saying that kids can take exams on paper?
They have, but antiquity is not necessarily a good argument. I attended a university which required examinees to wear a black suit, white bow tie, bands, and a gown. I don't recommend it.
But this is missing the point. The issue is not that people in general can't write by hand for hours, but that these particular people may have difficulty.
> We have to notice that high stakes exams on paper worked for hundreds, possibly thousands of years
Define “worked” there are reasons we have abandoned quite a few archaic practices which turned to be highly ineffective in hindsight (and often quite misery inducing). I’m not implying that handwriting is one of them just that it’s a terrible argument.
> burden of proof is on the side saying that kids
Smacking children who fail to give a correct answer with a stick worker for thousands of years as well..
You should definitely take speed into consideration. If your're writing an essay, being able to type it out and still have both the opportunity and time to edit it is great. If you're writing it on paper, you likely have neither. What comes to you first is what's submitted.
When I was in grade school the common practice was to use the back page of the exam booklet to do a quick outline (assuming there was no other scrap paper available) and just cross it out when you were done with it. Being able to organize your thoughts and maintain a clear direction in writing 500-1000 words seems like an important thing to test for.
And that's exactly the point! By making sure the student can't edit the entire text once its written, you force him to think about the essay's structure and force him to plan much more before writing :)
Absolutely! That's why they are students: you learn to walk before you learn to run. Students should be forced to plan their texts and practice getting better at that. Whenever they leave school, they'll be able to write even better texts than the essays.
Honestly, school essays, as a text genre, suck to read. They are not meant to be enjoyable, beautiful texts! Instead, they are meant to help gauge the student's ability at text planning, the quality of their text project, their grammatical knowledge, and orthography.
I must say I'd be open to seeing schools attempting other models, though, and measuring the results thereafter to see in practice what happens when students type their essays instead. In terms of text projects and planning, that could be interesting and train different skills.
If the goal is to assess the ability of the student to produce a professional product, then why prevent them from using AI in the first place? The vast majority of professionals have access to AI nowadays?
Most curricula should probably feature both forms of assessment, demonstrating your knowledge of the basics in a closed book assessment and your ability to produce high quality final products using all resources available to you in take home assignments.
It's not JUST about output, it's also about learning. The output is a way to measure the learning, it's not important of itself except as a reflection of the student's mastery of the curriculum.
You're thinking of the output as the goal rather than as the means to an end.
No, I’m not? As I said, you should assess whether people know the basics (I.e. are they learning) by testing without AI tools available. However, I believe you need to learn how to employ AI tools, just as you need to learn how to use a word processor. And the best way to do that is through experience.
If you’re suggesting that the test favors those capable of arranging their thoughts and words before putting pen to paper then.... I’m not sure there’s a problem
Sure but given any length of time, which does tend to be finitely allocated for a test (if for no other reason than the prof or proctor does have other places to be eventually), having to hand write is slower and harder to revise, which means it's harder to get that full, understanding-demonstrating essay, done and polished.
What? Suppose you want to fix the opening of your essay. Best case it's pencil and you can erase some, but worst case you have a longer sentence you want to put in there so you can't do it without scribbling all over and making a mess of the page. Word processors let you edit. How is this controversial?
> why is the speed it takes to write it at all relevant?
Okay, so from first principles:
1. Time is finite, we will all perish
2. Unless you are doing open book, needs to be supervised (proctored / or prof/TA is there)
3. That person is paid for a shift
4. That shift must end
5. Therefore, anything that enables people to write faster is good for the students, who can get more paper written, or the paper better revised, during the finite time available for writing.
> 5. Therefore, anything that enables people to write faster is good for the students, who can get more paper written, or the paper better revised, during the finite time available for writing.
I think you're missing some logic in there somewhere. If a student fills in as many pages as they can, each with the number 1 written on it as large as possible, was this a good conclusion? The quantity of writing is not a good metric.
EDIT: to give a closer to reality example, an essay that's 4 times longer than a competing essay does not make it better.
Not sure why you're getting so hyperbolic. The point isn't to write as much as possible, it's to be able to get all of your points across. I'm not talking about where the "answer" is 10 words long, I was thinking of a general essay. Longer isn't better, however, half an essay may be pretty useless to get a good grade, and if you can write twice as fast, you have a much less chance of running out of time, for any quantity of "time" without having to allocate hours, wasting more proctor time and making exams harder to schedule. Because like someone else said, they don't impose a time limit to screw the slowest-writing students, they do it for practicality (my above comment was trying to explain that).
It’s fine to erase or use scratch paper on an essay test.
You don’t write the test to fill the 60 minute slot. You time it so students are able to finish early if they’re really good. Slow ones need the whole time but can still do well if they understand the material being tested.
Well, to some extent, yes. Of course the literal number of works you can write per minute is not, but:
- someone with a good understanding will often come to a concise, clear answer while someone struggling will produce a convoluted paragraph.
- the way to get to the result will vary depending on your understanding (e.g., are you blindly applying some method or understanding what's going on). For instance, "hey, this is a vector field, I don't need to evaluate this complex integral, I just need to compute the difference between the start and endpoint of the curve!". Both answers will be correct, but one denotes a much better actual understanding (and will take way less time).
I meant to say they usually aren’t a rush for non-ace students, just a full hour. You have to work diligently, though. Competitive tests excepted, obviously.
I literally explained this above. No one is trying to rush the students or punish the slowest writers. It's because time is finite and having tests sized so that it's impossible for the slowest test-taker to even feel rushed means that the test time period needs to be much longer. That means higher costs for paying the proctor and makes them much harder to schedule since you need enough very long non-overlapping test periods that every standard class scheduling slot has a corresponding exam period (and think of midterms -- those often have to fit in a standard class period, or else if someone takes longer, that might blow through another class).
Re "hand writing is harder to revise", I never had an issue with erasing words or parts of the text and using asterisks, end footnotes, the margins and whatever free space available (with arrows or not) to do revisions in written exams. Nobody complained and afaik it was fairly standard to do where I studied, as long as your exam itself was actually legible. Granted, I refer to math-related exams not essays on literature or philosophy where form may have mattered more. On the other hand, I cannot imagine writing any math during an exam on a computer.
I would often do a bullet point summary/outline of my answer on the paper. That would have arrows and insertions and crossed-out stuff everywhere -- it was usually a mess.
But then I'd use that to write the actual text answer, and consequently, it was mostly clean. I'd still have to cross stuff out or add a phrase between the lines every so often, but once the basic structure was done, that was mostly unnecessary.
And if I was ever pressed for time, I'd usually get part marks for the outline anyway. Depending on the subject, I'd sometimes race through the questions doing just the outlines, and then come back to write the full text as a complete second pass.
To be fair, this wasn't creative writing. I think it'd be harder when the expression is the content.
That's a strawman. I said bottlenecking testing by requiring hand writing is stupid. Put another way - you're meant to be using the test to be thinking and re-thinking the problem and articulating approaches and solutions, not use your time, effort and energy on managing quill and paper.
It's not a strawman. If the test is designed so that you have spend most of the time thinking about your answer then even doubling the writing time has not significant impact.
>The point of the exercise is to demonstrate the student's understanding of the material, not their WPM.
Testing for understanding requires the fastest WPM possible. Regardless of the method used, students don't all write at the same speed. To minimize the influence of this variance, they need to use the fastest writing technique possible. If a fast writer spends 30 minutes writing in a two hour test, and a slow writer needs a full hour, the fast writer gets 50% more thinking time. But if you double both their writing speeds, the fast writer's advantage drops to 17%.
The faster the writing technique people are allowed to use, the better your test can identify poor understanding.
If you can't concentrate while people are working on computers near you, I don't think you'll do well in any workplace that is based around knowledge work.
This is why I prefer working alone at night and am massively more productive. In any case, what's being tested in an exam setting is one's understanding of the course material, not whether they are a good fit for your idea of a normal workplace.
Playing devil's advocate: one reason I'm good at typing on computers is that I learned on typewriters, which need much more force for each key press, especially the little fingers. Someone who learned on computer keyboards won't have trained the muscles on their fingers as much, and will have a hard time pressing the outer keys strongly enough (and if you press the keys too lightly, the mechanism won't hit the ink ribbon with enough force to transfer the ink to the paper).
Electric typewriters were pretty widespread by the 60-70s or so it’s just that they don’t have quite as much “retro” appeal these days ( eg like cassettes vs vinyl)
I saw plenty of mechanical typewriters by the 80s/90s, when I learned typing at school. A relative of mine also taught a typewriter course, with several different mechanical typewriter models. Everyone I know who had a typewriter had a mechanical one. I don't think I've ever seen an electric typewriter in person; I've only read about them in the Internet. They don't seem to have been that widespread to me.
Typewriters are an expensive and niche item these days, due to no longer being manufactured, and the good ones being collected by weird nerds. Sort of like buying a 40-year-old vinyl turntable that is in good and usable condition.
If I build you a custom device that receives questions from a central computer and lets the user plug in whatever keyboard they want via bluetooth or usb to only type and answer the questions (ability to edit the text and submit when ready). The central computer can receive all the questions submitted to any of these devices connected to it via wifi or cloud. How much would pay per device? Would you pay for a subscription?
I think this can already be implemented through a fairly mundane thin client approach, such as Dell/Wyse type thin clients, citrix stuff, or an ordinary x86-64 desktop PC setup with an absolute barebones OS that connects a graphical desktop session to something centralized over a LAN.
You mean on mechanical typewriters? It would be hard to learn to press keys slowly and avoid rollover unless using single finger typing: key travel was a long way on old typewriters.
