Hackathon 2023:
The possibilities and limits of ChatGPT integration

AIWorkshopSoftware Development

These 6 obstacles we ran into

ChatGPT has been live for a while now and everyone sees that the possibilities are endless. The expectation is therefore that more and more companies will embrace this supertool in the coming years. We reserved a full day to get to work with the whole team on three ideas around the ChatGPT Glossary · In briefAPIAn API is a defined way for software to exchange data or call functions in other software without needing to know how that software works internally.Read more⁠. Goal: put a working app on the table in one day and at the same time try out new techniques together and gain experience.

Product design first

Over the past weeks we have gathered ideas and today we dive in. We split into 3 groups of 4 or 5 people and work out three of those concepts. For some it feels a bit unfamiliar. Normally we get the product design delivered ready-made from clients, so inventing something completely from scratch is new for most. The sketches are still a bit vague at first. They take shape quickly.

Finally building

Each group knows what they want to build. Tasks are divided based on what everyone feels like and coding can begin. One dives into the documentation to check what a specific tool can all do, another sets up a Glossary · In briefbackendThe backend is the part of an application that processes data, applies business rules and communicates with other systems on the server. Users usually access it through a frontend or API.Read more⁠ and yet another is busy with the latest version of Node. This runs smoothly.

Outsourcing to AI

In the spirit of this hackathon there is vigorous outsourcing to Artificial Intelligence. Of course you no longer need to invent a name yourself if you can ask ChatGPT for a list of suggestions. A slab of text for your website is written in the blink of an eye, you generate a logo via Logomaster, and you can even ask ChatGPT to produce a prompt that you later send back as input (a kind of ChatGPT-ception). It works.

The 'hack' part of the hackathon

Where we normally worry about the stability of software, security standards and solving problems as cleanly as possible, that plays no role whatsoever today. Solutions may be dirty and hardcoded, as long as it functions in the short term. No tests need to be written either to safeguard quality. That helps.

Solutions may be dirty and hardcoded today, as long as it works.

Some love that freedom. "It finally does not have to be neat!" Others struggle with it. "This really should not be like this." Still it is a welcome change for everyone and it helps that we know we do not have to keep building on this mess later.

These 6 specific things we ran into

ChatGPT often stops answers midway through a long slab of text. Does not announce it. If you ask for something that must follow exactly one format, you immediately have a problem: your application simply trips over it. Look up the LangChain OutputFixingParser online (or ping one of us) and this is usually fixable.
There is no built-in way to retrieve chat history with the API. Truly zero. Everything you want remembered, you therefore remember yourself somewhere else.
The default gateway timeout is 30 seconds. That is the period a server waits for a response. ChatGPT regularly (read: hard) takes more time for its response. That makes standard deployment unworkable, because the timeout hits hard on every request. So we had to craft a few workarounds.
ChatGPT has no sense whatsoever of its own limits. You can ask it for things the tool says it can do, but what you get back is then often illogical, does not line up, or simply stays away entirely.
Combining fiction with intelligent creativity remains difficult. We had ChatGPT invent a riddle, and that riddle was honestly really beautiful, but when we asked for the solution the chat simply had no answer. That link between making a clever riddle and being able to solve it yourself is completely missing. The slowness of ChatGPT means that as a user you have to wait long for a result. We are by now so used to everything being available immediately that wait times of 2 minutes or longer are quite a letdown. One of our solutions was to put Davinci in the loading screen to generate facts, which generally came back well within 10 seconds. That worked a lot more pleasantly.

The result

At a quarter past six the moment was there: the presentation of the various apps that had been put together that day. Of the three groups, two had delivered a working end product. Below you see what each team managed to achieve. The conclusion among the developers today stood as a pillar above water: a hackathon is fun, extremely intense, and it is wonderful to just 'ram production' once without really worrying about quality. As a bonus we learned a lot about what ChatGPT can and cannot do, and we are eager to apply that knowledge in our upcoming projects.

Interactive story - RPG-style

Group members: Bauke, Adriaan, Martijn, Remco, Rick

You can deploy GPT as an interactive storyteller. Players (one or more) invent a character, choose a setting, and maybe a genre. Based on what they do, GPT determines the next scene. At least, that is the idea.

On this case we slammed hard into the limits of ChatGPT, especially when it came to consistency. Building on a story invented earlier simply did not work, riddles the chat itself had invented had no solution, and every story stayed vague, full of gaps. The group discovered that you have to stuff enormous numbers of restrictions and requirements into your prompt, and even then you do not get a decent story out of it. What did run nicely was creating player cards via a text-to-image API.

Travel agency for round trips

Group members: Roland, Raymond, Robin, Siebe

GPT can also propose a round trip. As a user you choose a destination and how long you are away, and GPT puts together a travel schedule with places and the main highlights of those places.

The website for 'JournAI' looked surprisingly slick quickly, the team had asked ChatGPT to spit out content for a site, with USPs of the product and three reviews included. They then worked long on hardcoded trips, so you could only go to Vietnam. Not ideal, but at least you could tinker with different parts without the API integration having to run at full speed.

A solid first foundation. You put what you are looking for in a text field and a map rolls out with a decent trip. Although ChatGPT did think a "beach holiday" is something where in six days you cover the entire Mediterranean coastline.

E-learning

Group members: Ewout, Bren, Merel, Olaf, Ted

Take Duolingo too. Well, not really. But with GPT you try that same principle: serving a user a learning programme for an arbitrary text or topic. Say you throw in the Wikipedia page about elephants, and GPT builds a whole course, including quiz questions. Just automatically.

Because you stand waiting a while until ChatGPT puts everything together, that group put something in the loading screen early. That way you do not only have to stare at that screen. Within 10 seconds the end user already has something in hand via Davinci: quirky facts about exactly that filled-in topic.

You wait quite a while, but then you also get a solid e-learning back. Three levels, and with every text block a quiz question. Sometimes those questions are really hilarious, like: "what is your favourite dinosaur?" (we got that wrong by the way). And multiple choice answers are filled in just as creatively. With a question about what an elephant's ears are for, "to fly" stood as one of the options.

Key takeaways

ChatGPT can do a lot, but it is not truly reliable answers stop midway, contradict themselves or drop away entirely, even when the model firmly claims it can do something.
Context memory? You have to arrange that entirely yourself storing chat history and being able to get back to it later does not happen via the API, you really have to build something around it.
Wait times are a real problem for your product those can sit solidly above 30 seconds, and preparing your UX with matching technique explicitly for such slowness must be part of your design.
Standard infra wrecks you here that 30-second limit is often too short, so you quickly work with alternative deployments or with asynchronous flows.
Results from prompting stay elusive without tight, crystal-clear requirements it goes wrong, and even then quality stays uneven, especially with creative fiction that must hold tone and line.
Just quickly cobbling something working together can yield a lot such a hackathon setup with everything hard-pinned, neat code set aside for a moment, helps experiment quickly and expose technical blind spots.
Dead time while waiting should not be left lying push something active into it: mini content that rolls out of a faster model, or a separate flow, so someone really gets something in hand.