mk30<p>"One thing I would like to highlight here is is the sheer computational resource intensity of systematically testing an AI model’s behavior. Each permutation test required thousands of forward passes through the model. Rather than keeping my existing instance running continuously, I wrote an orchestration layer which allowed me to parallelize these tests at about 30% of the standard cost.</p><p>Even with this optimization, the full suite of validation tests I described cost around $3,500 in compute resources and represented almost a week of continuous computation. This is one reason why rigorous validation of AI models is often shortchanged in both research and industry—the compute costs of thorough testing often rival or exceed the training itself.</p><p>In general, the computational demands of modern AI are staggering and often overlooked. When researchers talk about “training a model,” they’re describing a process that can consume as much electricity as a small household uses in months. The largest models today (like GPT-4) are estimated to cost millions of dollars just in computing resources to train once. For context, the model I built for this experiment used a tiny fraction of the resources needed for commercial AI systems (about 0.001% of what’s needed for the largest models), yet still cost thousands of dollars." - <a href="https://tarakiyee.com/training-an-ai-on-ancient-undeciphered-texts-what-i-wish-i-didnt-learn/" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">tarakiyee.com/training-an-ai-o</span><span class="invisible">n-ancient-undeciphered-texts-what-i-wish-i-didnt-learn/</span></a> by <span class="h-card" translate="no"><a href="https://mastodon.online/@tarakiyee" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>tarakiyee</span></a></span> </p><p><a href="https://tilde.zone/tags/ML" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ML</span></a> <a href="https://tilde.zone/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>AI</span></a> <a href="https://tilde.zone/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>tech</span></a></p>