this post was submitted on 24 Jun 2023
30 points (85.7% liked)
Programming
17670 readers
308 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities [email protected]
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I wrote a fairly detailed spec for some software and told it what dependencies to use, what it should do, and what command-line options it should use. The base was a decent starting point, but after several hours of back-and-forth, after actually reading the code, I realized it had completely misinterpreted my spec somehow and implemented a similar feature in a completely broken way, as well as making a few mistakes/redundancies elsewhere. I tried to coach it to fix these issues, but it just couldn't cope.
I spent about 3 hours getting this base code generated, and about 5 hours re-writing it and implementing the features properly. The reason I turned to ChatGPT is because I needed this software written by the end of the day, and I didn't have time to read all the different docs for the dependencies I needed to use to write it. It likely would have taken me at least 2 days to write this program myself. It was an interesting learning experience, but my only ChatGPT usage in the future is likely to be with individual code blocks.
You really need to pay attention if you're using LLMs to generate code. I've found it usually gets at least one thing wrong, and sometimes multiple things horribly wrong. Don't rely on it; look for other sources to corroborate all of its explanations. Additionally, please do not feed proprietary, copyrighted code into ChatGPT. The software I was writing was released under a free license. OpenAI will use it as training data unless you use their API and opt out of it. ChatGPT isn't really a tool; it's a service which is using you as much as you're using it.
I had similar experiences and nowadays I just ask for sample exemples of how to do stuff in isolation then I piece them together myself.
One example was trying to create some hooks for git to avoid copy-pasting something every commit. After trying to often correct it again and again, I just decided to start fresh and ask for a generic sample. It finally gave me a correct one. But I did the work to customize if for my needs and test it.
The nature of the software I was writing was inter-dependent on other functions, so it needed to understand the surrounding context, which was difficult to coach it on. Its strength is definitely in isolated examples.
I've been using Kagi's Discuss Document feature for some things, like understanding documentation or an API. It's pretty useful. Also works on videos/files like PDFs.