Leon Derczynski

@leonderczynski.bsky.social

LLMs & Security at NVIDIA Prof in CS/NLP at IT University of Copenhagen garak guy, garak.ai "berømt skikkelse" "like a gazelle" Seattle/Copenhagen 🏔️

"The point is not that any single model, including ours, will always be the best" Exactly what comes up with vulnerability discovery. No one model finds all the vulnerabilities. No single vuln is found by only one model. This is a problem we have to work on together. depthfirst.com/post/why-def...

Why Defenders Can’t Bet on One Model | depthfirst

Critical security capabilities cannot depend on infrastructure customers do not control. The next generation of AI security products needs to adapt across models, providers, policies, and access…

depthfirst.com

Are politicians the best people to place guardrails on how we do computing? Constraining the *uses* of technology makes sense - it's crucial here. But the gulf between subject experts and politicians often ends in harm. Just look at what EU Chatcontrol degraded into. thehill.com/policy/techn...

Lieu criticizes GOP colleagues for lack of strategy on AI

Rep. Ted Lieu (D-Calif.) on Wednesday expressed concern about the rapid evolution of artificial intelligence. “I am still freaked out by AI, and it’s actually accelerated much more quickly than I t…

thehill.com

False dichotomies around LLM speak: * "frontier" models vs. open model - leading models can be open * closed model vs. Chinese model - origin doesn't impact distribution You can have open frontier models, closed Chinese models, open US models, closed non-frontier models (e.g. for private context)

another data point showing it's the harness not the model - this time from wiz: "Atlas: Wiz's autonomous AI Agent for vulnerability research" look at their bold quote -- "Along the way, we learned that the durable advantage is not any single model, but the system around it"

Bild

"Open-source AI matters because it defines the ecosystem. It provides the foundation and sets parameters for progress, just like the open infrastructure of the internet and early AI: BSD Unix, PostgreSQL, Firefox" Open wins at grass roots level 🤷‍♂️ nationalinterest.org/blog/techlan...

Why America Must Dominate Open-Source AI

China's open-source AI models are spreading globally. To preserve technological leadership, the United States must lead not only in capabilities but in openness.

nationalinterest.org

The Hill covers Open Secure AI Alliance: “The United States now faces a similar choice with artificial intelligence” the letter states. “Our AI leadership will be judged not by one frontier AI model, but by whether the United States builds a strong, open ecosystem that diffuses into every sector”

BildBild

The harness is everything. New tech demo dropped for one way of doing CodeAct agents. Super-efficient. The team smashed CyberGym while building this - it's now the top-ranking open system for that vulnerability discovery benchmark.

Bild

Adversarial attack in the wild! The close visual appearance of M and W in this typeface and and packing of vertical lines make it hard to read, easy to get wrong, and tougher to scan. Love it. How often do you see something like this?!

red car with washington license plat MWWMWMWMW or similar

I keep saying the strength is in the harness, not the model - because it's true. No use without a harness, though. Here's VISA's open-source cybersecurity harness. Just add model! Very cool of them to share this tech and lift the defensive cybersec poverty line. github.com/visa/visa-vu...

GitHub - visa/visa-vulnerability-agentic-harness: Visa Vulnerability Agentic Harness

Visa Vulnerability Agentic Harness. Contribute to visa/visa-vulnerability-agentic-harness development by creating an account on GitHub.

github.com

Anonymised analysis of the openai model 'breaching' hugging face: > report doesn't say what sandbox sol broke out of?? > a docker container running as root > Plot twist there was no sandbox at all > many use "sandbox" and "container with host access" interchangeably ymmv, use critical thinking

Bild

80%+ of breaches have nothing to do with a new vulnerability. The novel security risks (vulns) the press has been excited about are routine. Mitigations are in place anywhere half serious. Vulns have been traded on the dark web for years - if new vulns meant apocalypse, it would've been years ago.