Suspecting AI cheating, Ivy League prof ordered an in-person final; scores fell 50%

nate anderson, so its probably slop that slipped by. at least, that’s easier for me to understand than a human being actually writing something like that since 1980.

Hot damn… Having issues because you’re used to the AI thinking for you is one thing … being incapable of counting to 10 (with repeated attempts!) is quite another thing entirely.

Not being able to go to bed because you have to read 20 AI slop papers…that’s hell

I was hired during the pandemic and it was quite an interesting experience to get there. They had an online assessment center, several hours (with breaks) of all sorts of quizzes, questions and tasks, mostly intelligence and memory-related, combined with math, general knowledge, language proficiency and a section on personality and personal preferences, with a relatively limited focus on IT questions at the end that felt tacked on for the job. There was no surveillance during it - no webcam requirements or anything. They just sent me a link and I could complete this timed test at a point in time of my choosing, in one go. Since it wasn’t very difficult at all and there was more than enough time, I had no issues with this one.

Before the interview, I was handed a 20 minute condensed version of largely the same type of questions to be completed in person (very small room, alone, under supervision, to eliminate any possibility of cheating), which I completed in no time, and finally an extensive interview (IIRC about 45 minutes) in front of what I can only describe as a tribunal of eight more or less relevant people sitting in a half-circle in front of a chair for the interviewee (at a distance due to COVID), most of which would just silently judge me with their eyes and take down notes as two to three of them switched between interrogating, I mean, interviewing me. They were actually very friendly, but it did feel quite intense. I was extremely nervous in the beginning, even stuttered a little, but as the interview went on, my nerves calmed down. None of the questions were surprising nor upsetting, but due to my initial nervousness, I still felt like I had failed.

Anyway, after they had hired me (I received a call about two weeks later), they later told me that they had done this twice with about 100 applicants each, just to hire a single person. In retrospect, I can’t believe I successfully made it through all of this. I was very well prepared for the interview and had previously spent a lot of effort (several weeks worth) on the application itself, tailoring it to the job and the organization, but still.

If you have the amount of resources this large org has, I think this is a very thorough way of doing things. The simple method of repeating a portion of the test in person (with different questions, but the same kind as online) eliminates cheaters and can be done by an org of virtually any size. The lengthy interview in front of a diverse group I’m not so sure about though. With shy and nervous applicants (as well as those on the spectrum), this can be very overwhelming, but since they all have to find a consensus in the end, discrimination and personal preferences influencing the hiring decision are less likely to happen.