Suspecting AI cheating, Ivy League prof ordered an in-person final; scores fell 50%

its that SEVERE?, its likely someone unable to finish grad 1-5. they likely used AI all the way to graduation, or most of thier education was spent online during covid, which dint help things.

I was hired during the pandemic and it was quite an interesting experience to get there. They had an online assessment center, several hours (with breaks) of all sorts of quizzes, questions and tasks, mostly intelligence and memory-related, combined with math, general knowledge, language proficiency and a section on personality and personal preferences, with a relatively limited focus on IT questions at the end that felt tacked on for the job. There was no surveillance during it - no webcam requirements or anything. They just sent me a link and I could complete this timed test at a point in time of my choosing, in one go. Since it wasn’t very difficult at all and there was more than enough time, I had no issues with this one.

Before the interview, I was handed a 20 minute condensed version of largely the same type of questions to be completed in person (very small room, alone, under supervision, to eliminate any possibility of cheating), which I completed in no time, and finally an extensive interview (IIRC about 45 minutes) in front of what I can only describe as a tribunal of eight more or less relevant people sitting in a half-circle in front of a chair for the interviewee (at a distance due to COVID), most of which would just silently judge me with their eyes and take down notes as two to three of them switched between interrogating, I mean, interviewing me. They were actually very friendly, but it did feel quite intense. I was extremely nervous in the beginning, even stuttered a little, but as the interview went on, my nerves calmed down. None of the questions were surprising nor upsetting, but due to my initial nervousness, I still felt like I had failed.

Anyway, after they had hired me (I received a call about two weeks later), they later told me that they had done this twice with about 100 applicants each, just to hire a single person. In retrospect, I can’t believe I successfully made it through all of this. I was very well prepared for the interview and had previously spent a lot of effort (several weeks worth) on the application itself, tailoring it to the job and the organization, but still.

If you have the amount of resources this large org has, I think this is a very thorough way of doing things. The simple method of repeating a portion of the test in person (with different questions, but the same kind as online) eliminates cheaters and can be done by an org of virtually any size. The lengthy interview in front of a diverse group I’m not so sure about though. With shy and nervous applicants (as well as those on the spectrum), this can be very overwhelming, but since they all have to find a consensus in the end, discrimination and personal preferences influencing the hiring decision are less likely to happen.

they probably got into the problem where candidates were using AI to automatically fill all the questionairres for the interview.

Do you have an example of the type of questions that will cause hallucinations?

The amount of scorn for degrees here is spectacular.

Medicine happens, in part, by experience. Even so, would you want your doctor or nurse working on you either of those roles without their degree(s)?

Psychologist?

Your lawyer?

The engineer designing that bridge you cross every day?

thats not the only thing they want, they have screen out specific keywords too, or look for X amount of experience, in this specific skill. even beyond your general i have “x amount of general experience”. plus they also use other metrics like, just looking at 10-20 resumes and trashing the rest.

they mean the job market as the low-wage, medium wage jobs that supplies most of sectors. not the bougie white collar jobs. to the end they also dumb it down enough that people end up joining the military as cannon fodder.

To be fair, jobs and degree value is 90% regional. There are states with less degrees and states with more degrees. The PNW, for example, has a glut of Bachelors degrees, meaning your Bachelors will give you very little if any traction for jobs. Why would it when there’s double digits in those degrees applying for every 1 job requiring one? Many Midwest, flyover states do not. When there’s no one applying to those degrees positions and then That One Guy shows up, he’s almost guaranteed at least a try in that position.

Sometimes you have to take your degree and move or it will hold little value. People don’t like to hear that, and that’s fair, but it’s also the reality.

You’ll have a cheaper house, cheaper COL, and be very competitive in a job in a flyover state, but because you don’t want to leave the city of Seattle or its surrounding beauty, you stay and bitch about it instead. Or San Francisco. Or [next high COL/housing place]. No one really likes Ohio or the ass end of Illinois but there are both job openings and cheaper housing there.

An in-law earned a safety degree. They could stay in their state for $80k or move to a degree bereft state, with cheaper houses even, for $125k. They moved, and on the company’s dime.

Some of what’s happening here is based in the human principle of: I don’t wanna.. “Don’t wanna” rules the day most of the time.

Don’t wanna keeps you in a crowded expensive place bitching about your finances.

nate anderson, so its probably slop that slipped by. at least, that’s easier for me to understand than a human being actually writing something like that since 1980.

It’s not just posers, it’s also an issue of just being easy, so even a seasoned, experienced professor can use it to avoid hours of prep for materials that would then be above the cut of the LLM content. Still lowers the standard, but without even changing the professors.

apparently IVY league schools arnt immune to AI slop- dumbing down. i think covid made both professors and STUDENTS lazy to the point they would just try to AI(likely to just search the answers online). from what ive seen colleges are pretty stingy at hiring people once they get enough tenured, they avoid paying more by having temporary instructors, either adjunct PHDs, MS holders, and the rare BS holder.

probablly better of starting at CC first taking all your GE classes, then looking at 4years to transfer to, this gives you time to research schools.

we had take final exam in a animal physio, but it gave a time limit so you cant just sit around searching google to look at all the answers. Also its done it a away its hard to search for the answers, and this was 10+years ago

I don’t know where you went to school, but in my college I was allowed to use calculators on the math tests. I never did because I quickly realized the professors intentionally wrote problems such that doing math in my head would be easy. Those who used a calculator missed the important crutch of, ’oops, this math is getting hard - That must mean I’m making a mistake’ and go back to fix the problem.

Physics and chemistry, dealing with real world constants that are often not particularly around easy numbers, you pretty much need a calculator in order to get anything close to a correct answer. But even then, the correct thing is to simplify without a calculator first before plugging the numbers in.

This is pretty much my perspective on the major problem represented by AI: cognitive atrophy.

I’m 40 something, masters degree, desk based professional consultant for 20 years now.

I don’t really recall the content of the courses I studied at university, although they do come back to me with a little revision. What has stuck with me, is everything you just described: how to collate information and understand problems, and how to communicate. I’ve had 30 odd years of practice at doing that.

My point is, had I been using AI during that time, I would not have had that practice.

Math classes never allowed calculators, not because they were some sort of unfair advantage, but because if you somehow became reliant on it, it would hamper your ability to learn how to solve the problem.

I really don’t get this logic. Calculators don’t solve problems they do arithmetic and make it so you don’t need to cry because you can’t remember the fucking unit square and your masochist calc prof wants you to do trig operations without a calculator.

If you are learning to solve expressions, then sure it makes sense to limit calculator use. I still hate it because I did poorly in math until we stopped using numbers all together and then I ✨ thrived ✨.

Yes. That’s what I was trying to communicate. They aren’t using their brain to gain the critical thinking skills to figure shit out for themselves. If we think this is bad, oh just wait til the gen in kindergarten grow to college age without having to use a single braincell through their entire education. It’s already happening and it’s scary. Watching 11 year olds unwilling to figure anything out for themselves and instead either asking someone how to do something or just googling it… We’re so screwed.

Because people don’t ask, “How do I do a Fourier Transformation?” The say, “Do this math problem for me and show so the steps.”

A half-wrong cheating tool can still be useful if you haven’t done any studying at all.

AIs appear impressive because they’re mostly right. That is: in many areas they’ll “know” more about a topic than a random person, so to the novice they’re really impressive.

It’s just that being “better than the average person” at a task is still pretty shitty compared to someone who specializes in that task.

It’s why so many CEOs find them impressive. Their results look good to the untrained.

Because people don’t cheat to gain knowledge. They cheat to fulfill a societal norm (getting the diploma/certification etc) that says they “succeeded”. Cheating is basically just taking a shortcut to an outcome without gaining the benefit of doing the actual work.

The usefulness of cheating in this scenario is that they can get the result the college requires without doing the work, and still receiving the benefit of achieving said result.

Because we don’t generally explain to people why we school them and teach them specific things. We fail quite badly at expressing the why of learning specific skills and we have known that for awhile. AI just created a readily accessible shortcut for some to thinking because we don’t and have basically never explained that a lot of school work is about teaching basic ideas and then developing skills to let people scale those ideas up using critical thinking and first principles. We are teaching people to think, not just to know.

Because it is very convincing. It generates great essays. The problem is that it has no way to evaluate truth.

Because they’re confidently wrong.

It’s still the fact that they need to be learning the HOW. There’s nothing wrong with calculators at some point but not when you don’t know how that calculator got the answer. It’s the same for AI.

Like what models? I use Gemini PRO (whatever the highest paid version is) for work and have to fact check everything of importance.

Dumb stuff it’s failed at: Here are two lists of a few hundred serial numbers, list all that are on both lists.

I’m trying to do X, how would I do that in X system? *Proceeds to tell me to click on things that don’t exist.

I don’t know that’s a good assumption anymore. These students are probably the ones who had zoom from home “school” through formative years, with Chromebooks pushed on them before that. Schools were dropping handwriting from curriculum long before LLM/agentic forms of AI.

Heck, I was graduating high school around the time the first iPhones were coming out and I barely wrote. It was maybe 1-2 essays per school year max, not exactly enough for robust skill development. Exams were almost all multiple choice or 1-2 word short answers at most to make grading more efficient.