Text-to-* Model Prompt Programming Task
(Redirected from prompt engineering task)
Jump to navigation
Jump to search
A Text-to-* Model Prompt Programming Task is a programming task that requires the creation of a AI model text prompts (for a text-to-* model) to solve a prompt-based text-to-* model inference task.
- AKA: LLM Prompt Engineering.
- Context:
- measure: improve their accuracy and relevance for a given task or application.
- It can (often) apply a Prompt Engineering Technique.
- It can be supported by a Text-to-* Model Prompt Programming System.
- It can range from being Text-to-Text Prompting, to being Text-to-Code Prompting, to being Text-to-Image Prompting, to being Text-to-Multi-Modal Prompting.
- It can range from being Manual Prompt Engineering (by a prompt engineer) to being Automated Prompt Engineering (by a prompt engineering system).
- It can be associated with Text-to-* Model Prompt Optimization.
- ...
- Example(s):
- Text-to-Text Prompt Engineering such as:
- ...
- Text-to-Code Prompt Engineering, such as:
- [[]].
- Text-to-Image Prompt Engineering, such as:
- ...
- Text-to-Text Prompt Engineering such as:
- Counter-Example(s):
- See: Prompt Tuning, Prompt-based Learning, Train of Thought, Few-Shot Learning, Transfer Learning.
References
2023
- (Amatriain, 2024) ⇒ Xavier Amatriain. (2024). “Prompt Design and Engineering: Introduction and Advanced Methods.” In: arXiv preprint arXiv:2401.14423. doi:10.48550/arXiv.2401.14423
- QUOTE: Prompt design and engineering has become an important discipline in just the past few months. In this paper, we provide an introduction to the main concepts and design approaches. We also provide more advanced techniques all the way to those needed to design LLM-based agents. We finish by providing a list of existing tools for prompt engineering.
- NOTES:
- Prompt Engineering can elaborate on advanced prompt engineering techniques such as Chain of Thought (CoT), Tree of Thought (ToT), and Retrieval Augmented Generation (RAG) to overcome inherent limitations of LLMs like transient state, probabilistic nature, and outdated information.
- Prompt Engineering can discuss the integration of external knowledge through RAG to enrich LLM outputs, making them more informed and contextually relevant by dynamically incorporating real-time or domain-specific information.
- Prompt Engineering can survey a variety of tools and frameworks developed to aid prompt engineers, such as Langchain, Semantic Kernel, Guidance library, Nemo Guardrails, and LlamaIndex, highlighting their contributions to streamlining prompt engineering processes.
- Prompt Engineering can address the inherent challenges in prompt design, including the need to understand the AI model's capabilities, the context of its application, and the creative and domain knowledge required to craft effective prompts.
- Prompt Engineering can underscore the significance of continuous innovation in prompt engineering to keep pace with the rapid evolution of LLMs and generative AI, suggesting that emerging techniques like Automatic Prompt Engineering (APE) may become standard practices.
- Prompt Engineering can conclude by emphasizing the growing criticality of prompt design and engineering in the AI field, encouraging practitioners to engage in this burgeoning area to shape the future trajectory of LLMs and generative AI technologies.
- Prompt Engineering is highlighted as crucial for maximizing the potential of Large Language Models (LLMs), underpinning the development of effective Artificial Intelligence applications through the craft of prompts.
2023
- (Wikipedia, 2023) ⇒ https://en.wikipedia.org/wiki/Prompt_engineering Retrieved:2023-6-17.
- Prompt engineering is a concept in artificial intelligence, particularly natural language processing. In prompt engineering, the description of the task that the AI is supposed to accomplish is embedded in the input, e.g. as a question, instead of it being explicitly given. Prompt engineering typically works by converting one or more tasks to a prompt-based dataset and training a language model with what has been called "prompt-based learning" or just "prompt learning".
2023
- (ChatGPT-OpenAi, 2023) ⇒ https://chat.openai.com Retrieved:2023-4-23.
- Q: What is Text-to-* Model Prompt Engineering?
- A: Text-to-* model prompt engineering refers to the process of designing and refining prompts or inputs for text-based models such as natural language processing (NLP) models, text classification models, or text generation models. The goal of prompt engineering is to improve the performance of these models by providing them with more effective inputs or prompts.
In text-to-* models, the prompt is the initial text or query that is given to the model, which it uses to generate a response or output. The quality and specificity of the prompt can have a significant impact on the accuracy and relevance of the model's output.
Prompt engineering involves several steps, including:
- 1. Data preprocessing: This involves cleaning and organizing the data that will be used to create the prompts.
- 2. Prompt selection: This involves choosing the most appropriate prompt or query for the given task or application. This may involve analyzing the input data and selecting prompts that are most likely to elicit the desired response from the model.
- 3. Prompt refinement: This involves iteratively refining the prompt or query to improve its effectiveness. This may involve adjusting the wording, structure, or format of the prompt based on the model's performance.
- 4. Evaluation: This involves testing the performance of the model using the refined prompt and making further adjustments as needed.
- Overall, prompt engineering is a key component of developing and optimizing text-to-* models, as it can help to improve their accuracy, relevance, and effectiveness for a given task or application.
2023b
- (ChatGPT-OpenAi, 2023) ⇒ https://chat.openai.com
- ... Another term that more specifically reflects the AI and NLP-focused nature of prompt engineering is “prompt programming". Prompt programming refers to the process of creating prompts or queries that are used to elicit specific responses from NLP models. The term "programming" emphasizes the technical nature of the task and suggests a more structured approach to designing prompts tailored to the needs of specific NLP models. ...
2023c
- (Liu et al., 2023) ⇒ Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. (2023). “Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing.” In: ACM Computing Surveys, 55(9).
- QUOTE: ... Now, as of this writing in 2021, we are in the middle of a second sea change, in which the “pre-train, fine-tune” procedure is replaced by one in which we dub “pre-train, prompt, and predict.” In this paradigm, instead of adapting pre-trained LMs to downstream tasks via objective engineering, downstream tasks are reformulated to look more like those solved during the original LM training with the help of a textual prompt. For example, when recognizing the emotion of a social media post, “I missed the bus today,” we may continue with a prompt “I felt so ” and ask the LM to fill the blank with an emotion-bearing word. Or if we choose the prompt “English: I missed the bus today. French: ”), then an LM may be able to fill in the blank with a French translation. In this way, by selecting the appropriate prompts we can manipulate the model behavior so that the pre-trained LM itself can be used to predict the desired output, sometimes even without any additional task-specific training (Table 1(d); e.g., Brown et al. [9], Petroni et al. [100], Radford et al. [105], Schick and Schütze [120]). The advantage of this method is that, given a suite of appropriate prompts, a single LM trained in an entirely unsupervised fashion can be used to solve a great number of tasks [9, 131]. However, as with most conceptually enticing prospects, there is a catch — this method introduces the necessity for prompt engineering, finding the most appropriate prompt to allow a LM to solve the task at hand. ...
2023d
- (Wikipedia, 2023) ⇒ https://en.wikipedia.org/wiki/prompt_engineering Retrieved:2023-2-26.
- Prompt engineering is a concept in artificial intelligence, particularly natural language processing (NLP). In prompt engineering, the description of the task is embedded in the input, e.g., as a question instead of it being implicitly given. ...
2022
- (Zhou et al., 2022) ⇒ Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. (2022). “Learning to Prompt for Vision-language Models.” International Journal of Computer Vision 130, no. 9
- ABSTRACT: Large pre-trained vision-language models like CLIP have shown great potential in learning representations that are transferable across a wide range of downstream tasks. Different from the traditional representation learning that is based mostly on discretized labels, vision-language pre-training aligns images and texts in a common feature space, which allows zero-shot transfer to a downstream task via prompting, i.e., classification weights are synthesized from natural language describing classes of interest. In this work, we show that a major challenge for deploying such models in practice is prompt engineering, which requires domain expertise and is extremely time-consuming—one needs to spend a significant amount of time on words tuning since a slight change in wording could have a huge impact on performance. Inspired by recent advances in prompt learning research in natural language processing (NLP), we propose Context Optimization (CoOp), a simple approach specifically for adapting CLIP-like vision-language models for downstream image recognition. Concretely, CoOp models a prompt’s context words with learnable vectors while the entire pre-trained parameters are kept fixed. To handle different image recognition tasks, we provide two implementations of CoOp: unified context and class-specific context. Through extensive experiments on 11 datasets, we demonstrate that CoOp requires as few as one or two shots to beat hand-crafted prompts with a decent margin and is able to gain significant improvements over prompt engineering with more shots, e.g., with 16 shots the average gain is around 15% (with the highest reaching over 45%). Despite being a learning-based approach, CoOp achieves superb domain generalization performance compared with the zero-shot model using hand-crafted prompts.