Tag: prompt-injection
-
Hacker News: Garak, LLM Vulnerability Scanner
Source URL: https://github.com/NVIDIA/garak Source: Hacker News Title: Garak, LLM Vulnerability Scanner Feedly Summary: Comments AI Summary and Description: Yes **Summary:** The text describes “garak,” a command-line vulnerability scanner specifically designed for large language models (LLMs). This tool aims to uncover various weaknesses in LLMs, such as hallucination, prompt injection attacks, and data leakage. Its development…
-
Hacker News: The Beginner’s Guide to Visual Prompt Injections
Source URL: https://www.lakera.ai/blog/visual-prompt-injections Source: Hacker News Title: The Beginner’s Guide to Visual Prompt Injections Feedly Summary: Comments AI Summary and Description: Yes Summary: The text discusses security vulnerabilities inherent in Large Language Models (LLMs), particularly focusing on visual prompt injections. As the reliance on models like GPT-4 increases for various tasks, concerns regarding the potential…
-
Schneier on Security: Prompt Injection Defenses Against LLM Cyberattacks
Source URL: https://www.schneier.com/blog/archives/2024/11/prompt-injection-defenses-against-llm-cyberattacks.html Source: Schneier on Security Title: Prompt Injection Defenses Against LLM Cyberattacks Feedly Summary: Interesting research: “Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks“: Large language models (LLMs) are increasingly being harnessed to automate cyberattacks, making sophisticated exploits more accessible and scalable. In response, we propose a new defense…
-
Simon Willison’s Weblog: Quoting Question for Department for Science, Innovation and Technology
Source URL: https://simonwillison.net/2024/Nov/1/prompt-injection/#atom-everything Source: Simon Willison’s Weblog Title: Quoting Question for Department for Science, Innovation and Technology Feedly Summary: Lord Clement-Jones: To ask His Majesty’s Government what assessment they have made of the cybersecurity risks posed by prompt injection attacks to the processing by generative artificial intelligence of material provided from outside government, and whether…
-
Simon Willison’s Weblog: Quoting Model Card Addendum: Claude 3.5 Haiku and Upgraded Sonnet
Source URL: https://simonwillison.net/2024/Oct/23/model-card/#atom-everything Source: Simon Willison’s Weblog Title: Quoting Model Card Addendum: Claude 3.5 Haiku and Upgraded Sonnet Feedly Summary: We enhanced the ability of the upgraded Claude 3.5 Sonnet and Claude 3.5 Haiku to recognize and resist prompt injection attempts. Prompt injection is an attack where a malicious user feeds instructions to a model…
-
Simon Willison’s Weblog: This prompt can make an AI chatbot identify and extract personal details from your chats
Source URL: https://simonwillison.net/2024/Oct/22/imprompter/#atom-everything Source: Simon Willison’s Weblog Title: This prompt can make an AI chatbot identify and extract personal details from your chats Feedly Summary: This prompt can make an AI chatbot identify and extract personal details from your chats Matt Burgess in Wired magazine writes about a new prompt injection / Markdown exfiltration variant…
-
Hacker News: Hacker plants false memories in ChatGPT to steal user data in perpetuity
Source URL: https://arstechnica.com/security/2024/09/false-memories-planted-in-chatgpt-give-hacker-persistent-exfiltration-channel/ Source: Hacker News Title: Hacker plants false memories in ChatGPT to steal user data in perpetuity Feedly Summary: Comments AI Summary and Description: Yes Summary: The text discusses a vulnerability discovered in ChatGPT that allowed for malicious manipulation of its long-term memory feature through prompt injection. While OpenAI has released a partial…