ChatGPT – What it can do and where it fails
Recently I took some time to explore if and where ChatGPT could be useful for my work.
I noticed very fast that it´s easy to have some fun with ChatGPT but it´s not that easy to come up with real usecases for my work.
My first idea was to have an app where I can choose a category and inside the category an option and based on my choice a command will be executed.
This can be very helpful for snippets and other often used commands.
I went to ChatGPT and explained what I want to have, ChatGPT said “no problem” and gave me a Bash script for a basic app with GUI that did exactly what I asked for.
Then I came to the conclusion that it probably would be a good idea to use ChatGPT for coding related tasks.
This could definitely be helpful if it works.
If…
The script that I got from ChatGPT is made with Bash and it uses Zenity for the GUI.
Zenity is a program to create GUIs for Bash scripts.
Then there is Yad, Yad is similar to Zenity but it´smore powerful.
When I asked ChatGPT to complete the task with Yad it always had at least small errors.
When I asked ChatGPT to add new functionality to the existing script it also built in errors.
ChatGPT solved the initial task so good that I thought it probably only needs few small changes to get the same good results for other tasks.
Then I started testing.
I tested ChatGPT with GPT 3.5 and I tested ChatGPT with GPT 4.
I went to the OpenAI Playground and tested complete mode with text-davinci-003, I tested chat mode with GPT-3.5-Turbo, I tested edit mode with text-davinci-edit-0 ...