Artificial intelligence is reshaping both sides of cybersecurity: attackers are using it to move faster and fool people more convincingly, while defenders are racing to put AI to work before adversaries do. This page tracks the AI developments that matter most from a security point of view. Every trend below is sourced from published industry research and verified reporting, not speculation. Trends marked as ongoing research are areas Professor Fehr is actively studying, including his doctoral work on AI-agent liability and least-privilege supervision of AI agents.
1. AI Agents Are Becoming the Newest Privileged Insiders
AI agents can now act on your behalf: sending emails, running scripts, moving data between systems, and executing transactions through APIs. That makes them powerful assistants, and it also makes them a new class of identity to secure. Gartner identifies agentic AI oversight as a top cybersecurity priority for 2026, warning that rapid and often unsanctioned adoption of AI agents is expanding enterprise attack surfaces and raising the risk of rogue automation.
Akamai's September 2026 State of the Internet security report describes a fundamental shift: enterprise risk is moving from managing human identities to governing autonomous nonhuman entities, and the difficulty of setting guardrails for autonomous agents is now a central challenge. The report also found that more than 6% of enterprise AI chatbot conversations contain sensitive corporate data, with 47% of interactions happening through unmonitored personal accounts.
Why it matters for security: An agent acts with credentials but no judgment. A 2026 survey reported that 37% of organizations experienced AI agent-caused operational issues in the past twelve months, and only 9% could intervene before a harmful agent action completed (Cybersecurity Insiders). Unauthorized access, unmanaged automation, and limited visibility into agent behavior are the core risks, per Gartner.
What this means for you:
- Keep an inventory of every AI agent operating in your environment, including shadow and employee-installed ones.
- Give agents the least privilege they need, never standing admin access.
- Require human approval before any agent action that moves data externally, changes permissions, or creates accounts.
- This is the territory of Professor Fehr's research into "PAM for agents": supervising AI agents like employees, with least privilege by default and elevated authority only on request, scoped, and time-bound.
2. AI-Generated Phishing Is Now the Norm, Not the Exception
The classic signs of phishing are gone. Poor grammar and obvious fakes have been replaced by polished, personalized messages written by AI. Hoxhunt's 2026 research, based on millions of real attack observations, found a 14-fold end-of-year surge in AI-generated phishing, which now accounts for roughly half of all observed attacks (Hoxhunt). KnowBe4's 2026 threat trends report found that 86% of phishing attacks in the observed period were AI-driven (KnowBe4).
Attacks have also spread beyond email. KnowBe4 reported a 49% increase in calendar-invite phishing and a 41% escalation in Microsoft Teams attacks, describing a shift from single-channel attacks to coordinated multi-channel campaigns. Barracuda's 2026 Email Threats Report, analyzing more than 3.1 billion emails, found that one in three messages was malicious or unwanted spam, and that 90% of high-volume phishing campaigns used phishing-as-a-service kits (Barracuda).
Why it matters for security: Attackers can now generate thousands of unique, convincing lures in minutes, personalized with real names, transaction details, and your organization's writing style. Verizon's 2026 breach report noted that 62% of breaches involved the human element, up from 60% in 2025 (Hoxhunt press).
What this means for you:
- Train people to verify unusual requests through a separate channel, not by replying to the message itself.
- Extend phishing defenses to Teams, calendar invites, and collaboration tools, not just the inbox.
- Consider phishing-resistant authentication methods, since credential theft remains the goal of most campaigns.
3. Deepfakes and Synthetic Media Are Defeating Human Verification
Synthetic audio and video have crossed the line from novelty to fraud weapon. In February 2026, fraudsters reportedly caused approximately 95 million euros to be transferred out of Fideuram, the private-banking arm of Intesa Sanpaolo, by impersonating senior executives and advisers and using an AI-generated voice clone of a well-known lawyer. Reporting indicates there is no sign the bank's core systems were technically compromised; the attackers manipulated people inside the authorization chain instead (FinTelegram).
The nsKnox 2026 Treasury Dragons Payment Fraud Index, surveying 104 corporate treasury professionals, found that 53% had faced a confirmed or suspected deepfake-related fraud attempt, most often in multi-channel attacks that pair a legitimate-looking email thread with a voice call, messaging exchange, or video interaction. Yet only 44% have formal policies covering AI-related fraud (nsKnox).
Why it matters for security: A voice call or video meeting can no longer be treated as proof of identity. Even four-eyes controls offer limited protection when attackers socially engineer both pairs of eyes, as the Fideuram case illustrates.
What this means for you:
- Never approve a payment change or data release based on a call or video alone. Verify through a pre-established out-of-band channel.
- Put a written verification policy in place that names AI-related fraud explicitly.
- Consider pre-agreed verification phrases or callback procedures for executive payment requests.
4. Prompt Injection and Data Leakage: the AI Vulnerability You Cannot Patch Away
Prompt injection is when hidden instructions inside content an AI reads (a webpage, a document, an email) steer the AI to do something its owner did not intend. OWASP ranks prompt injection as the top security vulnerability for large language model applications, and its Top 10 for Agentic Applications, released in December 2025, catalogs agent-specific risks including goal misalignment, tool misuse, and delegated trust failures (OWASP AISVS research).
Real incidents have moved past theory. In mid-2025, the EchoLeak vulnerability (CVE-2025-32711) demonstrated a zero-click prompt injection against Microsoft 365 Copilot that enabled enterprise data exfiltration without any user interaction (Cybersecurity Insiders). In 2026, researchers disclosed the Copilot Studio "ShareLeak" flaw (CVE-2026-21520), where indirect prompt injection in untrusted SharePoint fields drove the agent to query SharePoint lists and exfiltrate personal data through Outlook. Microsoft patched it, but a near-identical attack still succeeded against the patched system, showing that fixing one payload syntax leaves the underlying weakness intact (OWASP AISVS research).
Why it matters for security: No known input filter reliably prevents prompt injection, because the attack uses the model's core ability to follow natural language. It is a privilege problem, not a filtering problem: the damage comes from what the AI is allowed to do once steered.
What this means for you:
- Constrain what AI agents may do so a successful injection cannot cause material harm: limit tool access, permissions, and data scope.
- Treat anything an AI reads from external sources as untrusted, even if it looks like normal content.
- Keep humans in the loop for consequential actions like sending data outside the company.
5. AI Regulation and Liability Are Now Enforceable Law
The era of voluntary AI guidelines is ending. On August 2, 2026, the European Commission's AI Office gained formal enforcement powers over providers of general-purpose AI models under the EU AI Act. It can now request documentation, evaluate models, order corrective measures, and impose fines of up to 15 million euros or 3% of worldwide annual turnover, whichever is higher. Prohibited practices, banned since February 2025, carry penalties up to 35 million euros or 7% of turnover (Wilson Sonsini).
The same enforcement date activated Article 50 transparency obligations: users must be told when they are interacting with AI, and AI-generated or altered content must carry provenance signals such as watermarks or metadata (Volkov Law). Obligations for high-risk AI systems were extended to December 2027 under the Digital Omnibus package, but the current enforcement powers apply now. The Act also reaches beyond Europe's borders: it applies to any provider whose AI is made available to EU users.
Why it matters for security: Compliance is now a live legal exposure, not a future planning exercise. The liability questions raised by autonomous agents, who is responsible when an agent's action causes harm, are exactly the territory Professor Fehr is researching on AI-agent liability and agency. Regulatory pressure will increasingly demand the inventories, audit trails, and human-oversight controls that good security already requires.
What this means for you:
- Build a complete inventory of the AI systems you run, who owns each one, and what data feeds it.
- Classify each system by risk tier and confirm your transparency disclosures on any customer-facing AI.
- Verify that your vendors comply with the foundation-model obligations upstream of you.
6. AI on Defense: the Agentic Security Operations Center
Security teams are adopting AI to keep up. The 2026 SOC-CMM survey data shows AI adoption growing aggressively: AI co-pilots up 145% and AI agents up 118% year over year. Yet only 10% of SOCs report getting excellent value from AI; two-thirds are deploying off-the-shelf AI inside an existing security stack without changing anything around it, and that cohort reports the least value (The Hacker News).
Adoption is creating its own detection problem. Intezer's review of roughly 16.9 million enterprise SOC alerts found about 73,000 tied to AI tools and agents, a volume that grew 685% between February and June 2026. A coding agent working normally spawns shell processes, reads credential stores, and opens outbound connections, behavior indistinguishable from an intruder in the first hour of a compromise. Detection logic written before coding agents existed describes this activity in the vocabulary of ransomware and reverse shells, generating false alarms (CapTech/Intezer).
There is real upside when the architecture changes with the tools: early data shows AI-driven playbooks can reduce average incident response times by 34% (SecureWorld).
Why it matters for security: AI can multiply a security team's capacity, but only if detection logic learns to distinguish legitimate agent behavior from intrusions, and only if humans retain oversight of consequential actions. Buying AI without redesigning the processes around it compounds the problem instead of solving it.
What this means for you:
- Expect legitimate AI agent activity to trip legacy detection rules; tune and rebaseline accordingly.
- Keep a human in the loop for automated containment decisions that could disrupt the business.
- Measure outcomes (time to detect, time to respond), not just AI adoption, when judging value.
7. Social Media and AI: What Is Real and What Is Not
Your feed has become a contest over belief. AI can now produce photographs, video, and audio that most people cannot tell apart from the real thing, and social platforms spread them faster than any verification can follow. A fabricated statement by a candidate days before an election, a synthetic eyewitness image from a war zone, a cloned voice in a viral clip: each is built to trigger a reaction, outrage, fear, a vote, a share, before anyone checks. Microsoft's Threat Analysis Center documented Russian, Iranian, and Chinese influence operations using generative AI in the 2024 U.S. election, including deepfake videos of candidates. The NSA, FBI, and CISA warned in their joint deepfake guidance that synthetic media can spread false information about political, social, military, and economic issues and cause public unrest, and that producing it at scale now takes little more than a laptop.
One of these images is a real photograph. The other was generated by AI. Decide before you read the answer.
Answer: Image A is the real photograph, Image B is AI generated. Credits: photograph by Paulrudd (Wikimedia Commons, CC BY-SA 3.0), Adobe Firefly Image 3 prompted by Jiri Vedral (Wikimedia Commons, public domain).
Why it matters for security: In wartime the stakes turn lethal. When footage emerges of a bombed hospital or a struck convoy, each side now claims the other side's video is staged or AI generated, and sometimes that claim is the only defense offered. Verification takes days; public judgment takes minutes. The result is a second effect researchers call the liar's dividend: once the public knows anything can be faked, real evidence can be dismissed as fake too. The FBI saw the domestic version in July 2026, when its IC3 warned that scammers were impersonating the IC3 itself with AI generated videos of senior FBI officials, fake social media profiles, and a spoofed website, contacting victims through Facebook Messenger and Telegram. The Bureau's position is blunt: IC3 has no social media presence and never contacts individuals through social platforms. Case in point: in May 2025, a dramatic explosion video circulated on X presented as real footage of an Israeli bombing in Gaza. A fact check by D-Intent Data traced it to a TikTok account that posts AI generated visuals for engagement; the video depicted no real event. Read the fact check (D-Intent Data) Watch the original video (TikTok). The account's own bio states that all of its videos are AI generated. The clip still traveled as real.
What this means for you:
- Treat emotionally engineered media as a delivery mechanism. If a clip is built to make you furious or terrified, verify it before you share it.
- Check provenance, not plausibility. Find the original source, reverse search the images, and favor verifiable origin over realistic appearance.
- In an election or a crisis, wait for confirmation from more than one independent, named source before acting on dramatic footage.
- Do not trust official looking contact on social platforms. Agencies like the IC3 state they do not operate there; go to the official site by typing the address yourself.
- Organizations: rehearse your response to a fabricated video or statement about your leadership now. The joint agency guidance recommends planning and tabletop exercises for exactly this.
Sources: Microsoft Threat Analysis Center on nation state election influence, NSA, FBI, and CISA deepfake guidance, FBI IC3 alert on IC3 impersonation.