Hackathon 2023: The capabilities and limitations of ChatGPT integration

These are the six obstacles we encountered

ChatGPT has been live for a while now. The possibilities are clearly limitless, and it is expected that more and more companies will start using this powerful tool in the coming years. We scheduled a day in our agendas to work together on three ideas using the ChatGPT API. The goal is to create a functional app in just one day (while also experimenting with new techniques and gaining valuable experience in the process).

  • Linda

Written by Linda

Let's start with product design

Over the past few weeks, we've collected ideas, and today we are starting to work on three of those ideas in three groups (consisting of 4 or 5 people). It feels a bit unfamiliar for some of us since we usually receive product designs from our clients, and coming up with something completely from scratch is new to many of us. We went through some rough sketches, but the ideas are quickly taking shape.

The building phase

The groups know what they want to create, and tasks have been assigned based on individual interests. Coding can now begin. While one person delves into documentation to explore the possibilities of a specific tool, another sets up the backend, and someone else searches for the latest version of Node.

Outsourcing to AI

In the spirit of this hackathon, various tasks are being outsourced to Artificial Intelligence. For instance, you can ask ChatGPT to generate a list of options instead of coming up with a name yourself. The body of your website can be written in no time, and you can generate a logo using Logomaster. You can even ask ChatGPT to generate a prompt that can be later sent back to ChatGPT as input (a kind of ChatGPT-ception).

The "hack" part of the hackathon

In our usual work, we focus on aspects such as software stability, security standards, and clean problem-solving. However, none of that matters today. Solutions can be "dirty" and hardcoded, as long as they work (in the short term). For example, there's no need to write tests to ensure quality. Some people enjoy this freedom: "Finally, we don't have to worry about being dry!" However, others struggle with it, saying, "This is not how things should be done." Nevertheless, it's a welcome change, and it helps that we know we won't have to continue developing on top of this mess.

These are the six specific challenges we encountered

  1. ChatGPT tends to truncate long responses without warning, especially when requesting a specific response. This can break your entire application. To address this issue, search online for "Langchain OutputFixingParser" or ask one of us for help.
  2. There is no built-in way within ChatGPT to store chat history and retrieve it later using the API. So if you want to save chat history, you need to store it using alternative methods.
  3. The default "gateway timeout" (the maximum time a server waits for a response) is set to 30 seconds. ChatGPT often takes (waaaaaaaaaaay) longer than that to respond. Standard deployments don't work well because you have to wait longer. Therefore, we had to come up with alternative solutions.
  4. ChatGPT lacks awareness of its own limitations. You can ask for specific things that ChatGPT claims to be able to provide, but the answers may be illogical, inconsistent, or even completely absent.
  5. Combining fiction with intelligent creativity is challenging. We asked ChatGPT to create a riddle (which turned out to be a beautiful riddle), but it turned out that ChatGPT itself didn't have an answer to that riddle.
  6. Due to the slowness of ChatGPT, end-users have to wait a long time for results. We are accustomed to immediate results, but with ChatGPT, there can be a waiting time of 2+ minutes. One of our solutions was to use Davinci for generating fun facts during the loading screen, as it often provides a response within 10 seconds.

The results

At quarter past 6, the moment has arrived: the presentation of the various applications built today. Out of the three groups, two have managed to produce a functional end product. Below, you can see what each team has created. The conclusion among the developers today was clear: a hackathon is fun, incredibly intense, and it's great to "go all out" in production without worrying too much about quality. As an added benefit, we have learned a lot about the possibilities (and limitations) of ChatGPT, and we can't wait to implement this knowledge into one of our future projects.

Interactive story - RPG-style

Team members: Bauke, Adriaan, Martijn, Remco, Rick

The idea is to utilize GPT as an interactive storytelling tool. One or more players should be able to create a character, choose a setting, and potentially select a genre. The goal is to have the story respond to player input and generate subsequent scenes based on that input.

In this case, the limitations of ChatGPT became apparent, particularly in terms of answer consistency. It proved impossible to build upon a previously generated story, riddles created by the chat had no solution, and the stories remained vague and disjointed. The group discovered that providing extensive restrictions and requirements in the prompt was necessary, but even then, a cohesive story did not emerge. However, a successful aspect was the generation of "player cards" using a text-to-image API.

Travel Agency for Round Trips

Team members: Roland, Raymond, Robin, Siebe

The idea is to utilize GPT to propose a round trip. The user selects a destination and the number of days. GPT then generates an itinerary with locations and highlights of those locations.

By initially asking ChatGPT to provide content for a website (including product USPs and three reviews), the website for "JournAI" quickly became visually appealing. The team worked extensively with hardcoded trips (for a while only Vietnam was available) to ensure that they could address different aspects, even if the API integration was not optimal.

A beautiful initial foundation for an application to prepare travel itineraries. Users can enter their preferences in the text field, and a map with a suitable guide is generated (although according to ChatGPT, a "beach vacation" entails traveling the entire Mediterranean coast in 6 days).

E-learning

Team members: Ewout, Bren, Merel, Olaf, Ted

The idea is to use GPT to automatically present a kind of DuoLingo e-learning program to the user based on a text or topic they provide. For example, the user enters the Wikipedia text about elephants, and then GPT turns it into a course including quiz questions.

Because there is a wait time for ChatGPT to assemble the e-learning content after entering the original information, this group decided to add "something" to the loading screen. They chose to fetch facts about the entered topic from Davinci, providing the end user with something to engage with within 10 seconds instead of just staring at the loading screen.

After a considerable wait, a beautiful e-learning course is generated. It includes subjects at three levels and a quiz question for each section of the text. The questions can be humorous, such as "What is your favorite dinosaur?" (which we got wrong, by the way). Additionally, the multiple-choice answers can be creatively filled in; for example, one of the options for a question about the function of an elephant's ears was "to fly."

Linda
Tips for our next hackathon?

Let me know! Then we can organize another one quickly ;)