Speaker A
I've got a question for you. Why do you watch me? Is it because I'm Teimu Vidal or even Teimu Vidal? Is it because you're sadistic and like to watch me suffer, even though 90% of you probably don't even know what you're looking at?
Limit Cant Code shares his struggles and lessons learned training an AI VTuber, emphasizing the complexity beyond simple fine-tuning.
Ask about this video. Answers come from its transcript only — with the timestamp, so you can check them.
Generated from the transcript and can be wrong — check the timestamp.
Key Takeaways
- Training AI VTubers requires more than simple fine-tuning; data quality and training methods are critical.
- Continued pre-training (CPT) is a key technique for teaching domain-specific knowledge effectively.
- Creating bespoke data sets tailored to the character is essential for personality and lore development.
- Misunderstanding training concepts like RLHF and synthetic data can lead to wasted effort and poor results.
- Persistence and deep research are necessary due to the complexity and evolving nature of AI model training.
What the video covers
- The creator discusses his extensive, frustrating experience training AI VTuber models, spending over 200 hours coding and nearly 1000 hours overall.
- He challenges the common notion that fine-tuning alone is sufficient for creating a high-quality AI VTuber personality.
- The importance of bespoke, high-quality data sets over generic or cheap data sets is emphasized for achieving a unique AI character.
- He describes failed attempts including evolving model generations, handwriting data sets, synthetic data, and RLHF misunderstandings.
- The creator explains the difference between supervised fine-tuning and continued pre-training (CPT) and how CPT better suits learning domain-specific knowledge.
- He highlights the difficulty of teaching the AI VTuber lore and personality naturally without forcing topics into conversations.
- The video covers technical challenges such as overfitting, chat template misuse, and model incoherence due to poor training strategies.
- Limit Cant Code stresses the need for proper research and understanding of training methods and data preparation.
- He shares insights on the iterative process and mental toll of AI training, including migraines and frustration.
- The video ends with encouragement for viewers to try training models themselves despite the challenges.
Chapters
- 00:00Introduction and Why You Watch Me
- 00:55The Reality of Training AI Models
- 02:04Emotional Toll and Data Challenges
- 02:58Failed Attempts and Chat Templates
- 04:54Realizations and Proper Training Techniques
- 05:57Challenges Teaching Lore and Personality
- 07:04Continued Pre-Training Explained
- 08:10Technical Issues and Model Improvements
- 11:22Data Preparation and Collaboration
- 12:37Final Thoughts and Encouragement
Full Transcript — Download SRT & Markdown
Speaker A
Or do you watch me because I seem to have the answers to making AI Tuber?
Speaker A
Well, if you're in the last camp, open your freaking eyes. I'm Limit Code. I'm not trying to be humble. I'm legit stupid. Maybe it's my fault with how I present these videos. But this time, I just need to crash out because after
Speaker A
spending 200 hours training models, I don't even know if I'm even close. And that's 200 hours directly in front of my computer coding. That's not time spent training models. That's not time spent actively researching. That's not even time spent dreaming of solutions. I've
Speaker A
probably spent close to a thousand hours across all of this, and I'm still mid.
Speaker A
And I hear you say, "Damn, Limit. You really are stupid." And to that I say, come. We can be stupid together. Go try for yourself because training models is hard. If you've ever watched videos about how to train a model locally or
Speaker A
how to train a model with your own data, the answer is always just fine-tune, bro. It's easy. Download Unsloth and use any conversational data set and you're fine-tuning, bro. No. God, please no.
Speaker A
That might work if you're just trying to change the tone, but my AI girlfriend needs to be perfect. Damn it. There's this saying that as an AI researcher, 80% of your job is just making data sets. Usually, the models stay the same,
Speaker A
but the data changes what it learns and how. And what pisses me off the most is how misinformed I've been because of it's just fine-tuning, bro. You aren't going to get the AI waifu of your dreams by going to the dollar store to buy a
Speaker A
cheap data set made in China. No, you need to suffer a little. The journey I took from taking Jay Eisen out back end to preparing Irma V's debut after reviewing all the footage.
Speaker A
Oh my god, I want to cry. I saw a doctor for all the migraines I've had just from viewing this footage. And I might need a lobotomy to forget all the trauma. I'm not only trying to shift the tone, and
Speaker A
I'm not even just trying to have it respond to the name Irma V. I want her to understand her world. I want her to have a personality and creative quips. I want her to have her own opinions.
Speaker A
It's five letters. I want her to be Irma V. And so yeah, I just need bespoke data sets and character of Irma. It's not that simple.
Speaker A
What I said, it's not that simple. Do you want to know what I had to do to get all of this data? Do you? My initial 50 models looked something like this. Oh, I'll talk to a prompted GPT model. Nope.
Speaker A
Oh, how about handwriting? Nope. Um, ooh, but a custom chat template was not supposed to do. Talk to a fine-tuned GPT model.
Speaker A
Getting colder, but I can fine-tune on better conversation. I said colder. Ooh, I just thought of another chat template. N quit. Handwriting data sets.
Speaker A
It's iffy. I'm not a story writer. And even if I was, I'd be forcing myself to write hundreds of long conversations at a minimum. And I'd have to be very deliberate with the conversations to make sure the model can actually learn
Speaker A
the persona, the trivia, be everything. We'll see the results in a year, and I doubt it'll be any good. In fact, it's getting even worse, and I cannot tell why. I was also recommended to try making synthetic data sets and RLHF. And
Speaker A
my dumbass assumed I knew what those were. I assumed synthetic data sets were just me talking to a model instead of writing the model's responses myself.
Speaker A
And I assumed RLHF was just me taking the best response of several responses generated by the model. It would take me months to learn what they really were.
Speaker A
But I'll get to that in a moment because I've committed an even greater sin.
Speaker A
Model cannibalism. I had this god-awful strategy to create evolving generations of GPT models by using the conversations from a fine-tune to continue fine-tuning the same model. The hope was that with every evolution, the conversations would focus on some other aspect, whether it
Speaker A
be lore or relationships or banter or whatever. But it just kept getting worse and worse and worse. Eventually, the models became incoherent, lost their sense of self, and the learnings weren't stacking like I hoped to lose my [ __ ] and ignoring the fact that I'm feeding the model the same slot that came out of its ass. These conversation responses were just bad and I didn't know what I was doing with the training hyperparameters and ended up overfitting
Speaker A
with all of my tiny ass data sets. And oh my god, don't even get me started on chat templates. Man, every time I watch myself bring it up, I get this Pavlovian response. Okay, no, let's I want to make
Speaker A
my own chat template. You know what? What I thought the chat template was a fancy formatted system prompt and a user input. Then I did a little bit of research, but clearly not enough because I spawned these abominations. And eventually I finally
Speaker A
bashed my head into the wall enough times to accumulate enough brain damage to stop thinking and finally see what was in front of me. And that's when I grew up enough to at least use the proper atomic delimiters, but clearly
Speaker A
not grown up enough to realize templating is done by Ginga and not my own [ __ ] ass prompter. In these one to two months alone, I turned through five different chat templates. And today I'm coming back around and saying that you
Speaker A
don't need a custom template. But who knows, maybe next week I'm going to come back and say you don't even need a chat template at all. Like man, just put the context in the bag. Thank God my stubbornness has limits though. After
Speaker A
two months of all of this, I decided the whole evolving model strategy just wasn't going anywhere. Not just for personality, but especially lore.
Speaker A
Whenever I would teach lore, the model would get it half right. It knew that it was an AI YouTuber with a creator, but it would often attribute Valle as our creator, especially when leaving the context of straight Q&A. It seemed more
Speaker A
like the model was remembering how to respond to who is your creator as opposed to actually learning Irma's creator is me. And it wasn't just her facts either. It was so frustrating to teach her anything new because I would
Speaker A
need to turn those facts into a conversation, but it needs to be incorporated naturally because if I stick it in in Q&A, then she only learns to get it right during Q&A. But then it's like I'm forcing the topic into
Speaker A
random dialogue. And the model learns to force the topic when I just wanted to have the understanding in the back of its tiny little brain. I just need some easy way to teach the model something without having to force it into
Speaker A
conversation. Then one night it was like I was ascending. No, I wasn't high, but I was having a brain wave. Conversations go through a chat template to become a single block of text. But what if, what if we just remove the chat template?
Speaker A
What if it's straight facts? Then I realized, wait, I didn't realize until now, but I was already thinking of fine-tuning in stages. I first taught a chat template, then personality, and so on. But training an LLM is also done in
Speaker A
stages, with the first stage being pre-training. What if I take a base model and continue pre-training, and I'll call it continued pre-training.
Speaker A
Yeah. So apparently that's a thing that was already well established and also just so happens to be called continued retraining. And those exact words totally weren't in front of my face for hours on the Unsloth wiki. God, I really
Speaker A
need to take a step back sometimes and properly do some research. But why do these tutorials keep talking about fine-tuning for learning domain-specific knowledge instead of continued pre-training?
Speaker A
What was that? CPT is actually a specialized form of fine-tuning, while the fine-tuning that I specifically knew was supervised fine-tuning. Seriously, how is CPT not supervised? Why the [ __ ] is self-supervised unsupervised? They're
Speaker A
What was that? CPT is actually a specialized form of finetuning, while the fine-tuning that I specifically knew was supervised fine-tuning. Seriously, how is CPT not supervised? Why the [ __ ] is self-supervised unsupervised? They're both the next token prediction. Except
Speaker A
there's one key distinction that I overlooked, resulting in so many problems. With CPT, your data is literally anything. It doesn't have to be a conversation. It can be a blog, notes, script, anything. And in training, you train to predict every
Speaker A
token. But with supervised fine-tuning specifically, your data is conversation in a chat template. And I mistakenly trained on every single token in that chat template. Thinking if the model could learn to infinitely continue the chat template without messing up, then
Speaker A
it quote unquote understood the chat template and that by the same nature, regurgitating facts also meant that it understood the facts.
Speaker A
That's wrong. Regurgitating is not necessary for understanding and learning to reproduce the parts you'll never need to reproduce can interfere with the true learning objective. You can modify the data collators while training and Unsloth even provides a function for
Speaker A
this which is what you're supposed to do. So you can spend those precious weights learning how it should actually respond and not how other users should respond. That's why my models kept misidentifying as random other people and have a hard time learning to
Speaker A
identify as Irma. The fact in SFT data sets we chose the assistant responses that was makes it supervised and it would be again months before I actually realized this. Like seriously, how am I supposed to know all of this [ __ ] from
Speaker A
the start? Don't give me that. I'm borderline illiterate. There's a lot of other key differences between continuous pre-training and supervised fine-tuning.
Speaker A
And honestly, each deserved their own video because they require very different strategies and considerations I'm still in the process of discovering.
Speaker A
Even then now would it differ from person to person and even each lab has their own secret recipes. At the end of the day, CPT works on any random old text to actually teach the model new concepts and relationships between these
Speaker A
concepts. Whereas SFT requires task specific text to mold how it returns that information. Then there's also the differences in hyperparameters and target modules and blah blah blah. You get the point. Go read a book. Okay. So I've created a bunch of corpus. I did
Speaker A
CPT on that corpus. I have a base model that can recall facts about Irma. Surely we're good now. Surely all that's left to do is to SFT with my existing chat data set and I'll be done. Nope. Sit
Speaker A
back down. We've still got 100 hours to go. This model is fine, but it's not in GGF, which is needed if I want to run it in Llama CPP as I had planned. So, I need to convert this using the many
Speaker A
examples provided by Unsloth or even directly using the scripts from Llama CPP. Except I can't. What? runtime error. Waits cannot be opened. What does that mean? I was following their notebook. I trained again in case it was corrupted. I made sure to load in for
Speaker A
bit was disabled. The only things that were kind of different were my model and my data set, which was strange because I used the llama models in the past, but not this one. I don't know what BMBB 4bit means. I'm Limit and I don't know
Speaker A
how to Google things. You see, models are normally shipped as 16- bit. And Llama CBP expects 16 bits for conversion, even if quantizing to a smaller bit count. So, it was reading four four-bit parameters and training it as one 16- bit parameter. And it
Speaker A
consistently fails here once it runs out of parameters to read. Okay, I'll just train on a nonBMBB 4-bit model like Quen and try again. Success.
Speaker A
Not. Sure, the model could now merge with the Laura, and sure, I could convert it to GGUF, but I encountered two even bigger problems. Firstly, after doing the supervised fine-tuning and looking at the results, these responses were the most incoherent and bland
Speaker A
responses I've ever heard. At this stage, I was using my god-awful chat template, but it didn't get much better after I changed it to use Coin's atomic delimiters. It was still failing drastically, and I thought I just didn't
Speaker A
have enough data to teach it to converts using the chat template. And that was a given because this model was only ever pre-trained and never had the labs finetuning done on it. The bigger related problem though is that no matter
Speaker A
what, the model wouldn't generate the atomic delimiters to begin with. I was thinking to myself, oh yeah, the model still doesn't understand the chat template and kept on training make a data set with a higher concentration of delimiters thinking that will help. I
Speaker A
even spent days making a web app to directly build and modify samples and spend even more days with the web app to create my data sets. I spent days downloading and cleaning other data sets from online writing all of those scripts
Speaker A
to clean them by hand. And after weeks of just creating data sets, I tried training again and I still haven't solved the problem. What the actual hell? You know what? At least I didn't try to scale it further and instead I
Speaker A
took a moment to investigate further because no matter how far I scaled, I probably wasn't ever going to get this model to generate the proper delimiters.
Speaker A
I checked the token IDs for each sample to see if the model was even trained to generate these tokens. And sure enough, they are. But the tokens they output instead after training are non-delimiting tokens with similarly high IDs. I tried training on nothing
Speaker A
but delimiters, but the model never learned. I asked around on different forums and even the onslaught discord to no avail. And that's when I finally decided to check the embedding weights table and compared delimiter to a simple hello token and found the probability of
Speaker A
generating a delimiter was orders of magnitude smaller. And since SFT is not meant to shift token distribution like that, I was never going to get to the point where this model could statistically generate the delimiting tokens. I compared this to Wama 3 and
Speaker A
those weights seem completely fine. So, it seemed to be by design that this base model with only pre-training done to it would never generate these delimiters because it was never trained to do so.
Speaker A
It took damn near a month, but I finally learned I could apply CPT to an already instruct tuned model and get the Instruct model with my custom knowledge integrated. So, after 120 hours, I finally have an Irma model that can
Speaker A
embed its lower accurately into conversations. And I don't think it sounds all that bad. Certainly good enough for a debut, right?
Speaker A
Are you tired yet? Because I sure am. A month later, I stress tested her model with my mods, and the responses were just so bad. Thinking slop, system prompt leakage, identity confusion, death loops, the list just goes on. My
Speaker A
god, at this point, I was stressing. I already spent 3 months getting to that point, and my list of problems just got even larger. And now there's less than one month till her debut. But thankfully, Luxinator came back from
Speaker A
last video to save my ass. And we hopped in a call to discuss two different things, synthetic data sets and base models. He was the one that originally told me about synthetic data sets, which I originally assumed I understood, but
Speaker A
obviously that wasn't the case. Seeing how hard I failed, I decided to finally ask him to show me his synthetic data setup. We both agreed that using an AI to generate the entire sample end to end just sloppy. And so to have an AI help
Speaker A
me in creating a data set, my idea was to sit there and reroll messages to create the ideal data set. It works, but it's super timeconuming. Lux on the other hand uses existing conversational data sets and replace the conversation.
Speaker A
Filling the assistant responses with an AI prompted with his persona prompts. It doesn't matter if the user's follow-up message doesn't make any sense. We just choose to naively replay. That's the basics of it at least. And so I took the
Speaker A
Olympus data set, ran it through four different uncensored role playing models, produced gigabytes of conversational data and manually sat there reading and filtering for usable examples. I had to do this because honestly even these role playing models are in struck slopp and don't portray
Speaker A
Irma all that well. I also tested like 10 other models and they were all pretty bad. And so there I was sitting through reading thousands of conversation full of the most depraved and degenerate [ __ ] But in the end, I came out with
Speaker A
500 samples of like four to 10 kernes each. Cool. What about the model? I thought Quinn was good.
Speaker A
Why is working with new models so damn hard? I've said it before, but I'll say it again. Model makers are concerned with accuracy and benchmark maxing, and that makes it harder to train for personality. The Quen series is verbose
Speaker A
in general, but it's especially heavy on thinking. And since I'm using my own custom chat template, the response section gets interpreted as a thinking section. This messes up its behavior that I learned prior with my new fine-tune causing a lot of the issues I
Speaker A
was seeing. Lux was evaluating a bunch of these base models and he avoided the Quen series for these exact same reasons, and he instead recommended I try Mistral Nemo. And that did seem good at first, but I ran into issues merging
Speaker A
my Laura since for some reason it shipped as 8bit and a dequantization process to merge tanks my CPT model's accuracy. And with less than a week left, I just had to switch to a model that I was actually familiar with. Good
Speaker A
old llama. Needless to say, this all took a lot of compute to try and get this done on time. Way more compute than I owned at the time. So, in the 3day span, I spent 200 USD renting GPUs and
Speaker A
training like 15 different models while synthetically generating all the data sets in parallel. A bargain considering how much I've spent on everything else so far. Can you believe I still haven't talked about half the stuff I tried during this time, let alone everything I
Speaker A
tried after. What about data set balancing? Amending short conversations for long context, knowledge graphs to data sets, stage fine-tuning, preference optimization monitoring loss e valves. Do you want me to stop? Because I can keep going. Point is, once you
Speaker A
actually try it for yourself, it's a neverending rabbit hole, and I clearly still have a lot to learn. As much as I would love to keep ranting, that's all the time I have for today. If you've made it this far, like, comment,
Speaker A
subscribe. Help me get this video out there because I've spent way too long on this. But anyways, thank you all for watching and I'll see you next time.
Topics:AI VTuberfine-tuningcontinued pre-trainingAI model trainingdata setsRLHFsynthetic datachat templatesLimit Cant CodeAI personality
