Review of OpenAI's Text-Completion AI (Playground / ChatGPT)

Feel free to discuss anything related to the creation of new KotC 2 adventures using the toolset.

Review of OpenAI's Text-Completion AI (Playground / ChatGPT)

Postby BlueSalamander » Wed Dec 28, 2022 6:28 pm

Recently, I’ve been testing the latest natural-language AI (GPT3 text-davinci-003) from OpenAI to see whether it can help with coding or creative writing. Since it may help with module creation, I am posting this in the KotC 2 Module-Creation forum.

You can either ask questions to the AI on the ChatGPT page at or on the Playground page at

You do need to create a free account and you will have to provide a phone number.

I’ve done most of my testing with the Playground, as the ChatGPT page was unavailable to me for a while. But today I was able to test the ChatGPT page.

The Playground allows you to set a number of parameters, such as a reply-randomness factor, and it lets you know exactly what’s included in the input given to the AI, as there is a strict limit on the size of the input. That limit also applies to the output.

The Chat does not have parameters for you to adjust. There is only a Regenerate Response button, which can be used to produce a different reply to the same query. On the other hand, there is no apparent limit on the input size, and that’s great.

When asked for C++ code, the Playground seemed to go into more detail than the Chat, but the quality of the response was reduced because of the strict limit on input and output.

Aside from that, I've found the AI to be a competent assistant for providing lists of Non Player Characters (NPCs) with their names and descriptions matching the context of your module.

It can also suggest factions and reasons for conflict between them. It can create NPC dialogues if you're sufficiently precise in what you're asking for.

It can create riddles for the player to solve. It can write rhyming poems about your adventurers and villains – good stuff that would take me a really long time, or perhaps forever, if I wanted to do it manually.

It can help design adventure systems, for example if you're creating a scenario with a spirit meter, alignment meter, or one that tracks the virtues of the player. You can ask the AI to provide examples of situations that will affect the player’s alignment.

There's no question that it's a great assistant for creative writing.

However, I've found it to be quite lazy when asked to create the draft of a whole D&D adventure module with x locations, the AI then resorting to outputting uninspired same-structure sentences such as:

<< Location 1 is the cave. The monsters here are goblins. The reward is a key granting access to the next location. Location 2 is the forest. The monsters here are orcs. The reward is a key granting access to the next location. Location 3 is the tower, etc. >>

Still, if you’re running out of inspiration, the AI will at least provide some ideas and you won’t have to start from scratch.

By the way, I really like the Youtube channel The DM Lair as a source of inspiration and ideas for creating interesting encounters.

The AI failed when I asked it to create mutually-exclusive mission choices. Apparently, the AI considered that << do you accept the mission yes/no >> is a sufficient answer.

The AI shares some defects with another chat AI named Replika, which I tried a while back.

While GPT3 has a tonne of real-world knowledge, it doesn't differentiate between real-world facts and imaginary creations (fiction). That's why it's best for non-factual fantasy writing.

It doesn't have any memory of what you told it previously beyond the current conversation. However, it will take into account the whole current conversation when replying, creating the illusion of it having memory.

It provides no guarantee of quality or truthfulness. It has just one job: to answer something to your input. For the AI, it doesn't matter if the reply provided is nonsense or uninspired.

When asked (through the Playground) about coding a chess AI and the evaluation of a move, it provided code based on a function that returns the score from taking a piece from the opponent and a function that returns a positional score.

When asked to expand the positional score function, it provided code that distinguishes between pieces (Pawn, Rook, etc) but for each piece the positional score given was the same, as if the King and the Queen had the same interest as the Pawn in reaching the top of the chessboard, which is nonsense. A similar question on ChatGPT provided a more interesting answer in text form (see image).

You can use the Playground to write an essay, but not a complete book. However, you might be able to write a book with ChatGPT by asking enough follow-up questions.

The AI can write software but the size limit prevents it from writing or reviewing an extensive programme.

At the moment, the AI is not connected to Internet, so it cannot use Internet to confirm facts and it has no knowledge of recent events.

All in all, despite the flaws, it's pretty impressive and useful. Although perhaps not as impressive, in my opinion, as AI drawing. See this Kickstarter update for a review of AI image generation.

In time, I'm sure that the AI will be able to write entire books on its own and in the style of an actual author, such as the next Harry Potter book.

I guess we'll be flooded with books, even more so than now, and people will only read the very best ones. Curating may then be more important than writing. Thank You For Reading! Best Regards :)
'Say there is a chunk of meat. Pirates will have a banquet and eat it! But heroes will share it with other people. I want all the meat!!' - Luffy in One Piece
User avatar
Master Conjuror
Posts: 1879
Joined: Sun May 18, 2008 6:20 pm

Review of OpenAI's Text-Completion AI (Playground / ChatGPT)


Return to KotC 2 Forum for Adventure-Module Creation

Who is online

Users browsing this forum: No registered users and 1 guest