TechTakes

1427 readers

360 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 1 year ago

MODERATORS

[email protected]

254

Don’t use AI to summarize documents — it’s worse than humans in every way (pivot-to-ai.com)

submitted 2 months ago by [email protected] to c/[email protected]

110 comments fedilink hide all child comments

(page 2) 50 comments

sorted by: hot top controversial new old

[–] [email protected] 4 points 2 months ago (2 children)

Is it only me, or is the linked article not super long on details & is reaching a conclusion from 2 examples? This is important & I need to hear more, & I’m generally biased against AI at this point— but the article isn’t doing enough to convince me

load more comments (2 replies)

[–] [email protected] 3 points 2 months ago (17 children)

I had GPT 3.5 break down 6x 45-minute verbatim interviews into bulleted summaries and it did great. I even asked it to anonymize people’s names and it did that too. I did re-read the summaries to make sure no duplicate info or hallucinations existed and it only needed a couple of corrections.

Beats manually summarizing that info myself.

Maybe their prompt sucks?

[–] [email protected] 8 points 2 months ago

How did you make sure no hallucinations existed without reading the source material; and if you read the source material, what did using an LLM save you?

load more comments (16 replies)

[+] [email protected] -7 points 2 months ago* (last edited 2 months ago) (17 children)

You could use them to know what the text is about, and if it's worth your reading time. In this situation, it's fine if the AI makes shit up, as you aren't reading its output for the information itself anyway; and the distinction between summary and shortened version becomes moot.

However, here's the catch. If the text is long enough to warrant the question "should I spend my time reading this?", it should contain an introduction for that very purpose. In other words if the text is well-written you don't need this sort of "Gemini/ChatGPT, tell me what this text is about" on first place.

EDIT: I'm not addressing documents in this. My bad, I know. [In my defence I'm reading shit in a screen the size of an ant.]

[–] [email protected] 6 points 2 months ago

if the text is well-written you don’t need this sort of “Gemini/ChatGPT, tell me what this text is about” on first place.

And if it's badly written then the LLM will shit itself.

Now let's ask ourselves how much of the text in the world is "well-written"?

Or even better, you could apply this to Copilot. How much code in the world is good code? The answer is fucking none, mate.

[–] [email protected] 5 points 2 months ago (1 children)

@lvxferre @dgerard have you bumped your head?

[–] [email protected] 2 points 2 months ago (1 children)

No, it's just rambling. My bad.

I focused too much on using AI to summarise and ended not talking about it summarising documents, even if the text is about the later.

And... well, the later is such a dumb idea that I don't feel like telling people "the text is right, don't do that", it's obvious.

[–] [email protected] 9 points 2 months ago

You'd think so, but guess what precise use case LLMs are being pushed hard for.

load more comments (15 replies)

load more comments