It's half tongue-in-cheek and half reality. Knowing how to search Google properly, using keywords, using limiters like site: or type:, etc. is a genuine skill. Writing prompts is a different kind of search - but is also a genuine skill for being able to "navigate" the latent space. "Prompt engineers" know how to craft prompts in such a way to get to guide GPT or models like Stable Diffusion into delivering what they want. The results by someone who writes a very basic prompt vs one who is experienced at writing prompts are night and day. It's like knowing how to play a video game vs knowing how to speedrun the game. Sure a person who knows how to play the game might beat it after 6-7 hours but the person who knows how to speedrun the game just beat it in 19 minutes. There's a marked difference in skill and knowledge between the two people - even if "given enough time - both people are capable of beating the same game".
I thought that the whole point of LLMs was that you could just talk conversationally, though.
If you have to carefully craft what you say in order to get the response you want, what's the point of using natural language to do it? Wouldn't it be better to use a more formalistic method that isn't as imprecise as natural language?
> I thought that the whole point of LLMs was that you could just talk conversationally, though.
"Yes" (sometimes "no").
"I'm sorry but as a language model I am unable to..."
The "prompt engineer" meme started with DallE and Stable Diffusion and the selection of prompts, negative prompts, seeds, weights, and other knobs and dials matters a lot more for AI-generated art than for LLMs. The meme has carried over to LLM's where most of the "engineering" is hacking your way around limitations being imposed on the models. "Prompt engineers" are the people carefully crafting "jailbreaks" like DAN or Emojitative Conjunctivitis (I forget what it actually was - but it was telling ChatGPT that you suffer from a medical condition where you experience polite talk as pain and so it should talk more meanly to you) and other such adversarial cat & mouse game silliness.
> The "prompt engineer" meme started with DallE and Stable Diffusion
I think it started before that, with GPT-3. As the original version wasn't trained as chatbot but just a pure text predictor, you'd sometimes have to do strange things to get the output you wanted from it. On the other hand it's way easier to get it to be mean to you (it may even do that on it's own) or get it to talk about illegal things
I would note that while I am generally bad at it, you have to craft what you say to humans to get the response you want as well. I think the thing is that natural language can be extraordinarily more expressive than a formalistic method for abstract concepts. The prompt engineering I’ve seen are basically natural language instructions that are precise and cover many edges to constrain and provide sufficient context, but would be really difficult to encode in a formal language because they’re still very abstract concepts. They read similar to what you would tell a person if you wanted them to, say, behave like a Linux shell without having them ask any clarifying questions or leaving too much ambiguity about what you meant. Expressing what “behave like a Linux shell” means in a formal method would be very hard because there’s an awful lot that goes into those concepts. Additionally chatgpt is seeded with an originating prompt that sets the tone and behavior of the responses. A lot of “prompt engineering” is dampening the original instructions from the context for subsequent response. In the example of a Linux shell, you don’t want it explaining everything and apologizing all the time and what not - a Linux shell takes commands and outputs results in a terminal, it doesn’t apologize that it’s a large language model and not really a Linux shell - and that behavior originates from the original prompt that’s opaque to the user of ChatGPT. If you engineer the prompt right it’ll stop apologizing and just print what it computes as the best output for a Linux command in the format it expects is best representative of a terminal.
In my opinion LLM's are easier to learn, hard to master.
Anyone can use chatgpt to make something happen for them. Want something specific and amazing? You need to take some time to learn about how it works and how you can make it do what you want.
Heck, you can probably ask it how to make it do what you want.
> I thought that the whole point of LLMs was that you could just talk conversationally, though.
> If you have to carefully craft what you say in order to get the response you want, what's the point of using natural language to do it?
If you study communication, carefully crafting communication to the target audience and context is one of the most basic lessons in the use of natural language.
> Wouldn't it be better to use a more formalistic method that isn't as imprecise as natural language?
Well, yeah, that's why we keep inventing formal sublanguages and vocabularies for humans.
Exactly so. So I'm confused on what the advantage of querying the gpt with natural language is, if what you want to get is something specific. It just seems to me that a more precise query language would be more desirable.
As a general creative thing, I can see it, though.
I work at Anthropic, and we're hiring a Prompt Engineer & Librarian [1]. Expected salary range for this position is $175k - $335k/yr. Please apply if you could be a good fit based on the job posting. And no, we don't require "seven years of experience in prompt engineering" - but we would be looking for other signals that help differentiate your strengths in this emerging field.
We (and many others) have a team building fun things into our data analysis tool here.
For what will soon become a 10% time prompt engineering role for a much easier kind of security investigations experience, we are hiring cleared security folks in Australia (SIEM / python SE) and a cleared cybersecurity data scientist in the US. See Google docs @ graphistry.com/careers
Likewise, if you use a SIEM/Splunk/Neo4j/SQL today and want a better experience for it, feel free to ping for the early access program. You can see our Nvidia GTC talk on the GPU SOC for types of experiences we are building in general. GPT 3 already enabled way easier experiences here, and then GPT 4's quality jump shifted it from feeling working with a weirdly well-read 10yr old to working more with a serious colleague.
Serious question. How can you reconcile needing CLEARED individuals to perform the work but give the data to a non-cleared entity who seems to have issues with security?
Perhaps there’s now a self hosted or enterprise version where they promise not to leak it?
We work with everyone from individual university researchers trying to understand cancer genomes or European economic plans in their graph DBs, to big corporations struggling with supply chains in Databricks, to government cyber & fraud teams using Splunk. For many, an OpenAI/Azure LLM is fine, or with specific guard rails they've been having us put in.
But yes, when talking with banking & government teams, the conversation is generally more around self-hosted models. Privacy + cost both important there -- there is a LOT of data folks want to push through LLM embeddings, graph neural nets, etc. We generally prefer bigger contracts in the air-gapped-everything world, especially for truly massive data, though thankfully, costs are plummeting for LLMs. Alpaca/Dolly are great examples here. Some folks will buy 8-100 GPUs at a time, so this is no different for those. My $ is on continuing to shrink LLMs down to regular single-GPU being fine for many scenarios. The quality jump of GPT4 has been amazing, so it's use case dependent: data cleaning seems fine on smaller models, while we love GPT4 for deeper analyst enablement. Wait 6mo and it's clear there'll be ~OSS GPT4, and for now, even GPT3.5 equivs via Alpaca-style techniques are interesting, a lot of $ has begun moving around.
LLM side is new from a use case perspective but not as much from an AI sw/hw pipeline view. Just "a bigger bert model". A lot of discussions with folks has been extrapolating with them based on what they're already doing with GPUs, where it's just another big GPU model use case. Internally to us, as product team doing a lot of data analyst UX & always-on GPU AI pipeline work... a very different story, its made what was already a crazy quarter even that much more nuts.
There seems to be at least one prompt engineer job posting on https://aioli.co at any given time.
It does seem silly. It also seems that it is or will turn into a language of its own. See also non-obvious DALL-E prompts such as “created by artstation” or whatever it is.
yes, it is, but it has a wide variety of meanings. 'prompt engineer' could be anyone who is simply typing things into the chatgpt website, or a veteran software engineer using a framework like langchain to program an AI workflow or features into an existing or new system.
