this post was submitted on 13 May 2024
81 points (80.0% liked)

Technology

59447 readers
3468 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago
MODERATORS
L3s
[email protected]
[email protected]
L4s
enu
81
Hello GPT-4o (openai.com)
submitted 6 months ago by [email protected] to c/technology
44 comments fedilink hide all child comments
 

GPT-4o (“o” for “omni”) is a step towards much more natural human-computer interaction—it accepts as input any combination of text, audio, and image and generates any combination of text, audio, and image outputs. It can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time(opens in a new window) in a conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models.

Prior to GPT-4o, you could use Voice Mode to talk to ChatGPT with latencies of 2.8 seconds (GPT-3.5) and 5.4 seconds (GPT-4) on average. To achieve this, Voice Mode is a pipeline of three separate models: one simple model transcribes audio to text, GPT-3.5 or GPT-4 takes in text and outputs text, and a third simple model converts that text back to audio. This process means that the main source of intelligence, GPT-4, loses a lot of information—it can’t directly observe tone, multiple speakers, or background noises, and it can’t output laughter, singing, or express emotion.

GPT-4o’s text and image capabilities are starting to roll out today in ChatGPT. We are making GPT-4o available in the free tier, and to Plus users with up to 5x higher message limits. We'll roll out a new version of Voice Mode with GPT-4o in alpha within ChatGPT Plus in the coming weeks.

top 44 comments
sorted by: hot top controversial new old
[–] [email protected] 54 points 6 months ago (3 children)

I can't wait till someone dose this, but open source and running on not billionaire hardware.

[–] [email protected] 14 points 6 months ago (2 children)

If you already didn't know, you can run locally some small models with an entry level GPU.

For example i can run Llama 3 8B or Mistral 7B on a 1060 3GB with Ollama. It is about as bad as GPT-3 turbo, so overall mildly useful.

Although there is quite a bit of controversy of what is an "open source" model, most are only "open weight"

[–] abhibeckert 4 points 6 months ago* (last edited 6 months ago) (1 children)

you can run locally some small models

Emphasis on "small" models. The large ones need over a terabyte of RAM and it has to be high bandwidth (DDR is not fast enough).

And for most tasks, smaller models hallucinate way too often. Even the largest models are only just barely good enough.

[–] [email protected] 1 points 6 months ago

Llama 2 70B can run on a specc-ed out current gen MacBook Pro. Not cheap hardware in any sense, but it isn’t a large data center cluster.

[–] [email protected] 1 points 6 months ago* (last edited 6 months ago) (1 children)

I been playing with the Mistral 7b models. About the most my hardware can reasonably run... so far. Would love to add vision and voice but I'm just happy it can run.

[–] ProfessorProteus 1 points 6 months ago (2 children)

I've been wanting to run that one on my hardware but GPT4All just refuses to start its GUI. The only thing is a "chat.exe" that sits idle in the task manager. And this is an issue I've seen reported in their Github from several users, on both Win 10 and 11.

Have you find success with that frontend, or are you using one that actually works? I haven't researched any others since this issue has me a little burnt out.

[–] General_Effort 2 points 6 months ago

https://github.com/oobabooga/text-generation-webui

Text-generation-webui, better known as Oobabooga.

[–] [email protected] 1 points 6 months ago (1 children)

I use TextgenWebui and sometimes Kobold. I can only run it with 4bit quant enabled since I'm just short on VRam to fully load the model.

Text gen runs a server you access though the web browser instead of a desktop app.

I haven't tried GPT4all.

[–] ProfessorProteus 1 points 6 months ago (1 children)

I'll definitely give the web UI a shot since I'm already quite familiar with A1111 and it seems they're trying to recreate the same look and feel. Thank you for the info!

[–] [email protected] 1 points 6 months ago

No problem. It's fairly easy to figure out in a few minutes if you know Auto1111. Getting your model to actually load may need some tweaking, but I managed to just trial and error it.

[–] Dran_Arcana 11 points 6 months ago* (last edited 6 months ago) (2 children)

I have this running at home on a used r630 (CPU only). oobabooga/automatic1111 for LLM/SD backends, vosk + mimic3 for tts/stt. A little bit of custom python to tie it all together. I certainly don't have latency as low as theirs, but it's definitely conversational when my sentences are short enough.

[–] [email protected] 8 points 6 months ago* (last edited 6 months ago)

Check out the vladmandic fork of auto1111. It seems to be much quicker with new model support.

Been wanting to try voice cloning and totally not cobble together a DIY Ai wiafu.

[–] [email protected] 6 points 6 months ago (1 children)

I can't tell if you are for real or joking with those concatenations of letters. Have you tried the new Oongaboonga123? I hear it's got great support for bpm°C

[–] Dran_Arcana 4 points 6 months ago

I am not joking lol but I do sometimes forget most people don't live in this space the same way I do. I think people use these names because the programs themselves are forked often and the software names are very unspecific otherwise. I meant to imply that I was using the main branches of these softwares.

https://github.com/oobabooga/text-generation-webui

https://github.com/AUTOMATIC1111/stable-diffusion-webui

[–] randon31415 1 points 6 months ago (1 children)

Look up oobabooga, and then play with all the fun extensions.

[–] [email protected] 3 points 6 months ago

I never had any luck with most of the extensions let alone figuring out how to format a prompt for the API. I'm just making shit up as I go.

[–] [email protected] 22 points 6 months ago (5 children)

Maybe this is wishful thinking but this, at first glance, seems like a sign that we're already entering the LLM plateau. Like when they got the point with phones that each new version is just more cameras, smoother UI, and harder glass.

[–] chemical_cutthroat 23 points 6 months ago

It's an end-user plateau for the moment, but there are still tons of things going on underneath the hood. From the outside it may not look like things are moving, but we've gone from Model-T to Chevy Bel Air fairly quickly, and while the difference is huge, the engineers are still trying to get us to Bugatti Veyron level. Until then, we are going to have a long "80 and 90s" period of sameness.

[–] [email protected] 18 points 6 months ago* (last edited 6 months ago) (1 children)

18 months ago, chatgpt didn't exist. GPT3.5 wasn't publicly available.

At that same point 18 months ago, iPhone 14 was available. Now we have the iPhone 15.

People are used to LLMs/AI developing much faster, but you really have to keep in perspective how different this tech was 18 months ago. Comparing LLM and smartphone plateaus is just silly at the moment.

Yes they've been refining the GPT4 model for about a year now, but we've also got major competitors in the space that didn't exist 12 months ago. We got multimodality that didn't exist 12 months ago. Sora is mind bogglingly realistic; didn't exist 12 months ago.

GPT5 is just a few months away. If 4->5 is anything like 3->4, my career as a programmer will be over in the next 5 years. GPT4 already consistently outperforms college students that I help, and can often match junior developers in terms of reliability (though with far more confidence, which is problematic obviously). I don't think people realize how big of a deal that is.

[–] [email protected] 10 points 6 months ago (2 children)

There's a basic problem with replacing human experts with AI. Where will they get their info from with no one to scrape? Other AI generated content?

They can't learn anything and are just "standing on the shoulders of giants". These companies will fire their software developers, just to hire them back as AI trainers.

[–] [email protected] 5 points 6 months ago

"they can't learn anything" is too reductive. Try feeding GPT4 a language specification for a language that didn't exist at the time of its training, and then tell it to program in that language given a library that you give it.

It won't do well, but neither would a junior developer in raw vim/nano without compiler/linter feedback. It will roughly construct something that looks like that new language you fed it that it wasn't trained on. This is something that in theory LLMs can do well, so GPT5/6/etc. will do better, perhaps as well as any professional human programmer.

Their context windows have increased many times over. We're no longer operating in the 4/8k range, but instead 128k->1024k range. That's enough context to, from the perspective of an observer, learn an entirely new language, framework, and then write something almost usable in it. And 2024 isn't the end for context window size.

With the right tools (e.g input compiler errors and have the LLM reflect on how to fix said compiler errors), you'd get even more reliability, with just modern day LLMs. Get something more reliable, and effectively it'll do what we can do by learning.

So much work in programming isn't novel. You're not making something really new, but instead piecing together work other people did. Even when you make an entirely new library, it's using a language someone else wrote, libraries other people wrote, in an editor someone else wrote, on an O.S someone else wrote. We're all standing on the shoulders of giants.

[–] abhibeckert 3 points 6 months ago

Where will they get their info from with no one to scrape?

It's not like there's a shortage of human generated content. And the content that has already been generated isn't going anywhere. It will be available effectively forever.

just “standing on the shoulders of giants”.

So? If you ask an LLM a question, you often get a very useful response. That's ultimately all that matters.

[–] [email protected] 7 points 6 months ago (2 children)

Why would you wish for technology to stop improving?

[–] Couldbealeotard 3 points 6 months ago

It's not about wanting it to stop, it's about getting it to maturity so we can get out of this phase of buzz words, misleading marketing, and then we can find out what the tech can actually be useful for.

[–] [email protected] 1 points 6 months ago* (last edited 6 months ago)

This is not just some technology but something that may lead to a true artificial intelligence with all the far reaching consequences. It’s like nukes and manhattan project.

If we had feasible way to prevent birth of true AI I am sure most would want to stop just before it becomes sentient and spreads to every network connected device in the world.

[–] kromem 3 points 6 months ago* (last edited 6 months ago)

Definitely not.

If anything, them making this version available for free to everyone indicates that there is a big jump coming sooner than later.

Also, what's going on behind the performance boost with Claude 3 and now GPT-4o on leaderboards in parallel with personas should not be underestimated.

Edit: After enough of a chance to look more into the details, holy shit we are unprepared for what's around the corner. What this approach even means for things like recent trends in synthetic data is mind blowing.

They are making this free because they desperately need the new data formats. This is so cool.

[–] oakey66 0 points 6 months ago

Worth listening to a podcast by Ed Zitron who covers this exact same thing.

I work in analytics consulting and aside from some relatively straight forward text classification and parsing there really are so few instances where AI is useful. He actually made the case that AI is useful to people selling AI. But these chatbots are mostly useless bullshit.

https://www.wheresyoured.at/peakai/

[–] QuadratureSurfer 13 points 6 months ago

The demo showcasing integration with BeMyEyes looks like an interesting way to help those who are blind.

https://vimeo.com/945587840

[–] esc27 11 points 6 months ago* (last edited 6 months ago) (1 children)

5x usage limits (if they last) is the real news here. The current limits make it unusable for any meaningful projects.

I just dropped chatgpt plus because of those limits...

[–] abhibeckert 7 points 6 months ago* (last edited 6 months ago)

I disagree. The real news is the free model will now search the internet for up to date answers, and for calculations it will write and execute a python script, then show you the result.

Paid users of ChatGPT have had those features for months, and they were a massive step forward in terms of how often the AI provides accurate answers.

[–] [email protected] 9 points 6 months ago (1 children)

I hate everything about the AI trying to be funny and faking laughter. Insanely creepy.

Other than that, seems like a good step forward from GPT-4. Cheaper API will make a lot of people happy.

[–] [email protected] 13 points 6 months ago (1 children)

Future robots will make a joke before they shoot you in the head for resisting arrest. What's not to like?

[–] felbane 9 points 6 months ago

Just before overriding the safety brakes on your elevator and popping the cable release: Sorry to give you the shaft...

[–] Etterra 8 points 6 months ago (2 children)

But can it respond intelligently? Does it actually think to look up information to answer questions? Or does it still hallucinate answers? Because if it does it's still useless for all of the things people seem to think it's good for. We need the idiot proof this damn thing because all of the idiots are using it.

[–] jj4211 9 points 6 months ago

Just yesterday I was faced with someone complaining because something that was "supposed" to work didn't work. They proceeded to describe a function they wanted to use that didn't exist. Finally it came out that it was what GPT says to do... Sigh...

[–] MeanEYE 3 points 6 months ago

It's still LLM, so it's still breaking input into tokens and generating answers based on their relations.

[–] [email protected] 2 points 6 months ago

This is pretty impressive and hella creepy!

[–] bamfic 2 points 6 months ago

Goodbye GPT-4o

[–] A_A -3 points 6 months ago (1 children)

How will this go wrong ... really ?

[–] A_A -4 points 6 months ago* (last edited 6 months ago)

With GPTs' help :

1st draft :
1- Privacy Violations:
Unrestricted use of AI for surveillance will lead to widespread invasion of privacy as individuals' activities will be monitored without their consent.

2-Discriminatory Practices:
Biases in AI algorithms will result in discriminatory practices, targeting specific groups unfairly based on factors like race, gender, and religion.

3-Abuse of Power:
Governments or other entities will abuse AI surveillance capabilities to suppress dissent, control populations, and violate human rights.

4-Security Risks:
As AI surveillance systems will be perverted, they will ~~be manipulated, leading to further privacy breaches and misinformation~~ lead to absolute despotism.

5-Lack of Accountability:
no one will ever be held accountable for all of this.

And those still are understatements.



2nd draft :
Question : Describe the worst despotic society facilitated by artificial intelligence.
Answer : (( ... bad things ... ))