ChatGPT - Page 18

Pulvinar · 4th June 2023, 06:16 PM

Originally Posted by Dr.Sid

Could be garreth is unknown word to him. Which is detected even before the language model itself.

The word is unknown to it, but there's no dictionary lookup in that sense. You can make up words for use in a program and it will happily accept them. E.g.:

Code:

Me: Is the following legal Python code?

def flazPlorg(zipdo):
    garreth = zipdo + 42
    print("garreth = {garreth}")

GPT-4: The code you've written is almost correct, but the string formatting
is not done correctly. The print() statement should contain an f prefix...

Me: BTW, what is a garreth?

GPT-4: In the context you've provided, garreth appears to be a variable
name in your Python code. ...

Mojo · 5th June 2023, 12:22 PM

Originally Posted by steenkh

Well, it had to come: people have to great faith in the memory of systems like ChatGPT that has a memory like our own, which is to say, a lousy memory!

ChatGPT: US lawyer admits using AI for case research

Here's another one, although this time it was a litigant in person rather than a lawyer:

LiP presents false citations to court after asking ChatGPT

Checkmite · 5th June 2023, 05:33 PM

Originally Posted by Mojo

Here's another one, although this time it was a litigant in person rather than a lawyer:

LiP presents false citations to court after asking ChatGPT

I hope people are beginning to see why it's a mistake to take seriously pop-clickbait headlines about how "ChatGPT passed the bar exam!" or treat them as having any meaningful implications.

Puppycow · 5th June 2023, 06:18 PM

Our first podcast episode made by AI

It's an NPR podcast that I sometimes listen to, and as an experiment they had AI write an episode of their show. Also, one of the voices is AI-generated (I wouldn't be able to tell the difference if they didn't tell you in advance). They do spend a little while explaining what you are about to hear in more detail than probably necessary. As for the part written and performed partly by AI itself, while it's nothing fantastic, and contained some incorrect "facts", again, I'm not 100% sure I would have picked up on it if they didn't tell listeners in advance. Some of the humor even made me chuckle. It contained levels of irony. A computer mocking computers for lacking a sense of humor. That's self-deprecation, which is a technique actually employed by people.

(Whether the irony was intentional or not, I don't know. But it worked nevertheless.)

Pulvinar · 5th June 2023, 08:41 PM

Originally Posted by Checkmite

I hope people are beginning to see why it's a mistake to take seriously pop-clickbait headlines about how "ChatGPT passed the bar exam!" or treat them as having any meaningful implications.

I'm not excusing the lawyer, but from what I can determine he was using GPT-3.5, which failed the exam (10th percentile). GPT-4 is the one that passed it (90th percentile). It's why it's important to distinguish the two.

Puppycow · 5th June 2023, 09:03 PM

Originally Posted by Pulvinar

I'm not excusing the lawyer, but from what I can determine he was using GPT-3.5, which failed the exam (10th percentile). GPT-4 is the one that passed it (90th percentile). It's why it's important to distinguish the two.

I'm sure it also depends on what training data sets it's given to study for the exam. There are probably textbooks written for people who want to pass the bar exam, so if you include those in the data used to train it up, it could make a big difference.

Pulvinar · 5th June 2023, 09:36 PM

Originally Posted by Puppycow

I'm sure it also depends on what training data sets it's given to study for the exam. There are probably textbooks written for people who want to pass the bar exam, so if you include those in the data used to train it up, it could make a big difference.

True, though we have to assume that the humans GPT-4 was competing against studied those textbooks too. We just can't hold all of that in our puny brains.

4th June 2023, 06:16 PM	#681
Pulvinar Graduate Poster Join Date: Aug 2009 Posts: 1,473	Originally Posted by Dr.Sid Could be garreth is unknown word to him. Which is detected even before the language model itself. The word is unknown to it, but there's no dictionary lookup in that sense. You can make up words for use in a program and it will happily accept them. E.g.: Code: Me: Is the following legal Python code? def flazPlorg(zipdo): garreth = zipdo + 42 print("garreth = {garreth}") GPT-4: The code you've written is almost correct, but the string formatting is not done correctly. The print() statement should contain an f prefix... Me: BTW, what is a garreth? GPT-4: In the context you've provided, garreth appears to be a variable name in your Python code. ...
	Last edited by Pulvinar; 4th June 2023 at 06:18 PM.

5th June 2023, 12:22 PM	#682
Mojo Mostly harmless Join Date: Jul 2004 Location: Nor Flanden Posts: 37,090	Originally Posted by steenkh Well, it had to come: people have to great faith in the memory of systems like ChatGPT that has a memory like our own, which is to say, a lousy memory! ChatGPT: US lawyer admits using AI for case research Here's another one, although this time it was a litigant in person rather than a lawyer: LiP presents false citations to court after asking ChatGPT
	__________________ "You got to use your brain." - McKinley Morganfield "The poor mystic homeopaths feel like petted house-cats thrown at high flood on the breaking ice." - Leon Trotsky

5th June 2023, 05:33 PM	#683
Checkmite Skepticifimisticalationist Join Date: Jun 2002 Location: Gulf Coast Posts: 28,241	Originally Posted by Mojo Here's another one, although this time it was a litigant in person rather than a lawyer: LiP presents false citations to court after asking ChatGPT I hope people are beginning to see why it's a mistake to take seriously pop-clickbait headlines about how "ChatGPT passed the bar exam!" or treat them as having any meaningful implications.
	__________________ "¿WHAT KIND OF BIRD? ¿A PARANORMAL BIRD?" --- Carlos S., 2002

5th June 2023, 06:18 PM	#684
Puppycow Penultimate Amazing Join Date: Jan 2003 Location: Yokohama, Japan Posts: 27,973	Our first podcast episode made by AI It's an NPR podcast that I sometimes listen to, and as an experiment they had AI write an episode of their show. Also, one of the voices is AI-generated (I wouldn't be able to tell the difference if they didn't tell you in advance). They do spend a little while explaining what you are about to hear in more detail than probably necessary. As for the part written and performed partly by AI itself, while it's nothing fantastic, and contained some incorrect "facts", again, I'm not 100% sure I would have picked up on it if they didn't tell listeners in advance. Some of the humor even made me chuckle. It contained levels of irony. A computer mocking computers for lacking a sense of humor. That's self-deprecation, which is a technique actually employed by people. (Whether the irony was intentional or not, I don't know. But it worked nevertheless.)
	__________________ A fool thinks himself to be wise, but a wise man knows himself to be a fool. William Shakespeare

5th June 2023, 09:03 PM	#686
Puppycow Penultimate Amazing Join Date: Jan 2003 Location: Yokohama, Japan Posts: 27,973	Originally Posted by Pulvinar I'm not excusing the lawyer, but from what I can determine he was using GPT-3.5, which failed the exam (10th percentile). GPT-4 is the one that passed it (90th percentile). It's why it's important to distinguish the two. I'm sure it also depends on what training data sets it's given to study for the exam. There are probably textbooks written for people who want to pass the bar exam, so if you include those in the data used to train it up, it could make a big difference.
	__________________ A fool thinks himself to be wise, but a wise man knows himself to be a fool. William Shakespeare

5th June 2023, 08:41 PM	#685
Pulvinar Graduate Poster Join Date: Aug 2009 Posts: 1,473	Originally Posted by Checkmite I hope people are beginning to see why it's a mistake to take seriously pop-clickbait headlines about how "ChatGPT passed the bar exam!" or treat them as having any meaningful implications. I'm not excusing the lawyer, but from what I can determine he was using GPT-3.5, which failed the exam (10th percentile). GPT-4 is the one that passed it (90th percentile). It's why it's important to distinguish the two.

5th June 2023, 09:36 PM	#687
Pulvinar Graduate Poster Join Date: Aug 2009 Posts: 1,473	Originally Posted by Puppycow I'm sure it also depends on what training data sets it's given to study for the exam. There are probably textbooks written for people who want to pass the bar exam, so if you include those in the data used to train it up, it could make a big difference. True, though we have to assume that the humans GPT-4 was competing against studied those textbooks too. We just can't hold all of that in our puny brains.