Introduction to PNGTuber-GPT
This is a custom C# action for Streamer.bot and Speaker.bot to add a GPT-based PNGTuber to your stream!
Getting Started Video
Purpose
This document intends to show how to write a bot that connects to Twitch Chat and allows people to either redeem channel points or give bits to ask the bot questions. The question or prompt will be relayed to ChatGPT to generate a response, and then the response will be played back on-stream using a PNGTuber sidekick. The bot will respond in-character using a lore file that provides directives to the bot on the character it should play and how it should respond. The prompt will also include parameters such as the length of the response. You will also provide directions on how creative or literal the bot should be in its responses. The bot will connect to the moderation endpoint of ChatGPT to prevent hate or harmful prompts from being passed through. The bot should be able to remember things about users that they can redeem via points or bits to “train” the bot, and the bot should only reference specific users in this database when they are mentioned in a prompt.
Components
There are a variety of software and cloud components that enable this to work. This is one path of many, and it is beyond the scope of this document to teach you how to configure each one. You can utilize ChatGPT to answer questions about how to do this for other providers, or research their documentation. The overall methodology shouldn’t change too much until new innovative features are added.
• Streamer.bot (Supercharge your Live Stream | Streamer.bot) – used to execute custom actions based on events that happen on Twitch
• Speaker.bot (Speaker.bot · Speaker.bot) – used to playback TTS on stream
• Google TTS (Text-to-Speech AI: Lifelike Speech Synthesis | Google Cloud) – Used to convert the responses from GPT to dialog on the stream
• OpenAI GPT 3.5 Turbo, Optimized for Dialog (Models - OpenAI API) – Used to take our prompt and convert it to a generative response that can be spoken on stream, also used to moderate for hate/harmful speech
• Custom C# Code for Streamer.bot - This will execute the call to the OpenAI API and return the response to the Speaker.bot
• Veadootube Mini (https://olmewe.itch.io/veadotube-mini?download) – Used to animate the character on the stream when talking
• VB Audio Cable (https://vb-audio.com/Cable/) - Used to isolate the voice from the bot into it's own audio stream that can be captured by Veadootube Mini