Reference prompting

- As we start talking about advanced prompt engineering, it's important to understand what actually goes on when we talk to these AI models, because while it feels as if we're talking to an entity at the other end that is actually conversing with us, what's actually happening is we're sending off requests to a data model that puts together things that look like sentences and words. And that data model is trained on our own language, meaning when it returns content to us, it looks to us as if it is language, and therefore we imbue meaning into it. Which means, when we look at the responses, it's very hard not to think that this is coming from an intelligent being that actually is answering our questions. But in reality, it's just a machine putting together small pieces of letters that turn into words, that turn into sentences, that statistically match something that sounds like a human would say if they were trying to answer the question you were asking, which is a huge mouthful to say, and it's really hard to wrap your head around. But that is actually what's going on. So, as we start this journey into advanced prompt engineering, I want to start off at the very basic level to show you kind of how it works, and where the cracks are, and also give you an indication of how to just get the system to work better right off the bat. So, let's dive into ChatGPT. The very first thing I want to show you is a sort of proof that this is not an intelligent machine, and this is a well-known example. If you go to ChatGPT or any other system and you give it a very specific task, which is, "Spell the word lollipop backwards," it will never be able to do it properly. Like, you can see here, it says "pilollop." And no matter how much you try to get it to do this, it will never get it right. That's because these systems are not actually seeing the word as a set of letters. Instead, they're seeing the word as a set of tokens, and a token is one or two or three letters combined. So, when you're telling the system to spell a word backwards, sometimes, instead of seeing the word as a word, it sees it as a set of tokens, and then it tries to reorganize the tokens and everything falls apart. And for whatever reason, the word lollipop really triggers this. So, you can try for yourself. Try to get the system to spell lollipop backwards, and you'll discover the only real way of doing it is by telling it, "Split it into separate letters first, and then do it." So, that's an indication of how this is not an actual person or actual intelligent being we're talking to. And it also gives you an indication of what happens when you start asking it questions that it might have some information about, but doesn't necessarily know the true facts about. So, let's dive in again and see what that looks like. I want to ask ChatGPT about something that's relatively esoteric, but that it should still have some knowledge of. So, I'll go in and say, "In the Capability Approach, there are five general principles that are agreed upon, of which two seem necessary." Yeah, that's all misspelled; it doesn't matter. The system will figure that out. So, let's see what it says now. Now, because ChatGPT was trained on the internet, and there is quite a lot of information about this philosophy, called Capability Approach, on the internet, it'll be able to give me a coherent answer here. And this answer, just by scanning over it, looks like it is correct, but it's not actually answering specifically what I'm asking for. You see, in discussions about the Capability Approach, we often see this commitment to five principles: treating each person as an end, and then focus on choice, and there's a bunch of things. And then, it says, "However, it's been argued that only two of these principles are genuinely essential," and that's the information I want to surface. So, how do I get ChatGPT to surface specific information, when, when I ask it for things, it surface more general information? The simple answer is: ChatGPT and these other language models are language models. They're language transformers. So, if you provide a bunch of texts and you ask a question about it, it'll go up into that text and pull out the pieces you want, and then give you those pieces. So, if I go in here and I copy out a huge chunk of this, and then paste that in to give context to my question, and then copy my actual question at the bottom. Send off the request. And what happens now is the system goes up into the text and finds the answer I'm looking for. So, here you see it's pulling out treating each person as an end; focus on choice of freedom; and then it adds additional context to it. It even also pulls out just those two main values. What you see here is, once you give these AI systems context in the form of text or a spreadsheet, or something you can look up on the internet, they're far more likely to give you the correct answer because you're giving them reference material to work with. So, anytime you're trying to get these systems to respond to you about a real factual thing where context exists, give it the context, give it a reference to start with, and your prompts will always return better results.