Campus Technology Insider Podcast February 2024
David Wiley 31:46
Well, I think, I think there's still so many technical advancements that need to be made, that maybe, maybe they're not the most fun trends to follow, because they are kind of technical, but just, you know, two that come to mind just in the past, quite recently. The first is the kind of model architecture that powers models like Chat GPT is called a transformer. It's very powerful, obviously, in a bunch of ways, because Chat GPT blew all of our minds the first time that we tried it. But there's, just in the last, last little bit, I want to say, two months, there's been another technical architecture called Mamba that's been proposed, that looks to have a lot of the same kind of benefits as the transformer, but to be faster, less expensive to run, cheaper to train, things like that. That could lead to some very exciting things happening in the future. There's a technique for reinforcement learning with human feedback — RLHF is the way that models have, for the, for the last year, the way that we've been kind of fine-tuning models and teaching them to behave more the ways that we want them to behave. It just recently, there's been a new technique proposed called direct preference optimization that looks like it works as well as RLHF, but again, faster, cheaper, easier. So whatever the thing is that comes out next that's powered by Mamba and is kind of fine-tuned using DPO, and whatever these other new techniques are, people might not know about all the things under the hood, but they'll know, Oh, my gosh, this is even better and even faster and even cheaper than Chat GPT was. It's still kind of early days for, I know it's not early days for AI broadly, but for what we think of as being large language models, for example, it's early days. And even though GPT-4 blows all of our minds, it could be so much more powerful. And so I think there's still a lot of kind of fundamental under-the-hood, kind of basic research to go on that will be really exciting this year — and will end up, it will result in new products that all of us get really excited about.
Rhea Kelly 34:14
Thank you for joining us. I'm Rhea Kelly, and this was the Campus Technology Insider podcast. You can find us on the major podcast platforms or visit us online at campustechnology.com/podcast. Let us know what you think of this episode and what you'd like to hear in the future. Until next time.