Chapter 6: Using AI for Cyber Defense and Securing Organizational AI
Artificial intelligence is changing both sides of cybersecurity. Organizations can use AI and machine learning to analyze activity, identify threats, prioritize work, and support security teams. At the same time, generative AI applications, models, data pipelines, and autonomous AI agents introduce risks that organizations must understand and manage.
A meaningful AI cybersecurity achievement may involve using AI to strengthen cyber defense, securing the organization’s own AI systems, or doing both. Recognition should be based on verified implementation and measurable results—not the novelty of an experiment or the use of the term “AI.”
Using AI for Cyber Defense
AI and machine learning can analyze large volumes of security information and identify patterns that may be difficult to detect through fixed rules alone. Potential uses include identifying unusual account behavior, detecting suspicious network activity, recognizing malware, prioritizing alerts, and connecting related events across systems.
AI-assisted security monitoring can help analysts review alerts and investigate incidents. A system might summarize relevant activity, develop a timeline, recommend investigative steps, or help search technical records using natural language. Generative AI may also support cybersecurity analysis and documentation by producing preliminary incident summaries, converting technical findings into leadership reports, or helping teams draft response procedures.
These capabilities should assist qualified professionals rather than replace accountable decision-making. AI-generated conclusions may be incomplete, inaccurate, or influenced by low-quality data. Organizations should define when human review is required, particularly before disabling accounts, isolating systems, notifying affected parties, or taking other consequential actions.
Prioritizing Alerts and Vulnerabilities
AI can support alert prioritization by considering severity, affected assets, user behavior, threat intelligence, and organizational context. It may also enrich alerts automatically or recommend an incident classification.
In vulnerability management, AI may help connect technical findings with asset importance, exploit information, system exposure, and potential business impact. It can also assist teams in reviewing remediation guidance or identifying recurring weaknesses.
A credible AI cyber defense achievement should show more than faster processing. Organizations should measure whether prioritization became more accurate, high-risk cases were addressed sooner, analyst workload improved, or remediation decisions became more consistent. Automation that closes alerts quickly but overlooks important activity is not a successful outcome.
Securing Generative AI and AI Agents
Organizations must also protect the AI systems they develop, purchase, or integrate. Securing artificial intelligence includes safeguarding models, training and reference data, applications, interfaces, infrastructure, credentials, and connected services.
Generative AI applications may be exposed to prompt injection, in which malicious or misleading instructions attempt to alter system behavior. Other risks include sensitive-data leakage, unauthorized model access, poisoned or manipulated data, insecure external integrations, excessive permissions, and adversarial inputs intended to produce incorrect results.
AI agents create additional concerns because they may be authorized to retrieve information, use software tools, communicate with other systems, or take actions with limited direct involvement from a person. Agentic AI security should include clearly defined identities, limited permissions, strong authentication, action logging, approval requirements, and controls over which data and systems an agent may access.
An AI agent should not receive broad privileges merely because it performs work on behalf of an authorized employee. Its access should reflect the specific tasks it is permitted to complete, and high-impact actions may require human confirmation.
Managing Shadow AI
Shadow AI occurs when employees use AI applications or features without organizational review or authorization. They may unintentionally enter confidential information, customer data, source code, internal documents, or regulated records into systems that have not been evaluated.
Organizations can respond by establishing acceptable-use policies, providing approved alternatives, applying access controls, educating employees, and monitoring for unauthorized services where legally and operationally appropriate. The objective should not be to discourage beneficial innovation. It should be to enable responsible use while protecting organizational and stakeholder information.
A shadow AI achievement might involve discovering the extent of unauthorized use, introducing an approved platform, reducing risky data transfers, and increasing employee understanding of AI-related obligations.
Establishing AI Governance and Risk Management
Effective AI risk management begins with an inventory. Organizations should identify AI models, applications, features, agents, providers, data sources, system owners, users, and dependencies. Risk assessments can then consider the system’s purpose, information involved, possible failure modes, access levels, external connections, and potential consequences.
Governance measures may include:
- AI security and acceptable-use policies
- Defined ownership and approval processes
- Data-handling requirements
- Identity and access controls
- Vendor and supply-chain assessments
- Testing before deployment
- Continuous monitoring and logging
- Incident-response procedures
- Periodic reviews and retirement processes
Testing should examine not only whether the AI system performs its intended function, but also whether it resists misuse, protects information, follows authorization boundaries, and behaves reliably under unusual conditions.
Preparing for AI-Enabled Cyberattacks
Attackers may use AI to improve phishing, impersonation, social engineering, malware development, vulnerability discovery, or the speed and scale of other malicious activity. Organizations should update employee education, identity-verification procedures, fraud controls, monitoring, and incident plans accordingly.
Claims about AI-enabled threats should remain evidence-based. Not every sophisticated attack is necessarily powered by AI. Preparedness should focus on observable risks and effective defenses rather than speculative or sensational descriptions.
Measuring AI Cybersecurity Achievements
Useful measures for AI in cybersecurity may include improved detection quality, shorter investigation times, reduced repetitive analyst work, faster remediation, increased monitoring coverage, or better documentation consistency. For AI security, measures may include the percentage of systems inventoried, applications assessed before deployment, high-risk weaknesses corrected, unauthorized tools reduced, or agent actions covered by logging and approval controls.
Organizations should distinguish pilots from achievements. An experiment may demonstrate potential, but verified achievement requires an implemented capability, defined scope, appropriate governance, evidence of adoption, and measurable results.
As AI evolves, cybersecurity recognition should remain grounded in responsible implementation. The strongest AI cybersecurity success stories will demonstrate not merely that an organization used artificial intelligence, but that it applied human oversight, protected its systems and stakeholders, and produced a credible improvement in security or resilience.