AI Safety Argumentation Platform Research Engineer
At a glance
Mid-level AI Research role at Future of Life Organizations. Remote, United States · full-time · $160,000–$210,000 base.
aimlroles summary, based on the employer's posting.
What you'll do
- Expand schemas for claims, evidence, arguments, rebuttals, and confidence across ontologies and knowledge graphs
- Build defeasible reasoning models that represent argument logic and show where objections can undermine it
- Run and check LLM-assisted research pipelines, using cross-checks, source tracking, and expert review
- Coordinate agents for multi-step research, handle failures, and turn structured arguments into audience-specific communications
- Map strong counterarguments and update the graph when developments in AI safety, capabilities, or governance warrant changes
What you bring
- You have practical knowledge of formal or semi-formal argumentation, including defeasible reasoning or structured argument models
- You have built ontologies or knowledge graphs, using OWL/RDF, property graphs, or a comparable approach
- You have operated LLM agent systems, including coordination, scaled prompting, and quality checks on outputs
- You can prototype and ship with LLM-assisted coding, and bring strong coding skills plus experience verifying automated processes
- You understand AI safety, governance, and frontier-AI developments, and can assess evidence, locate authoritative sources, and communicate uncertainty
Who this fits
This mid-level role suits someone who can combine argumentation methods, graph engineering, and hands-on LLM system work. The position is fully remote with occasional travel. U.S.-based employees must already be authorized to work in the country; candidates elsewhere would work as independent contractors, and visa sponsorship is unavailable.
From the employer
The case that AGI and ASI pose catastrophic risks is strong but poorly systematized: fragmented across literatures, inconsistently formalized, and vulnerable to motivated dismissal. CARMA is building an evidentiary infrastructure to fix this. It combines ontologies, knowledge graphs, defeasible argumentation frameworks, and LLM-assisted population pipelines under expert curation, feeding structured argument content into communications flows that reach policymakers, technical audiences, journalists, and the public.
In this role, you'll develop and operate that system. You'll work where argumentation theory meets agentic AI tooling, building machinery that is both formally tractable and persuasive in practice, the epistemic backbone that will help stakeholders elucidate why good arguments for prospective expectations are good, and why bad arguments are bad.
This position is 100% remote but requires occasional travel.
About CARMA
The Center for AI Risk Management & Alignment (CARMA) works to help society navigate the complex and potentially catastrophic risks arising from increasingly powerful AI systems. Our mission is specifically to lower the risks to humanity and the biosphere from transformative AI.
We focus on grounding AI risk management in rigorous analysis, developing policy frameworks that squarely address AGI, advancing technical safety approaches, and fostering global perspectives on durable safety. Through these complementary approaches, CARMA aims to provide critical support to society for managing the outsized risks from advanced AI before they materialize.
CARMA is a fiscally-sponsored project of Social & Environmental Entrepreneurs, Inc., a 501(c)(3) nonprofit public benefit corporation.
Responsibilities
- Extend ontologies and knowledge graph schemas representing claims, evidence, argument structures, defeaters, and confidence
- Implement defeasible argumentation frameworks (e.g., ASPIC+, Dung-style, argumentation schemes) that capture both logical structure and vulnerability to rebuttal
- Operate and quality-control LLM-driven population pipelines, with cross-check scaffolds, provenance tracking, and human-in-the-loop curation
- Architect agent coordination patterns for multi-step research and population tasks, with robust error handling and graceful degradation
- Pre-harden argument structures by mapping the strongest counterarguments, steel-manned objections, and known defeaters
- Build export pipelines that translate structured argumentation into diverse communications formats across audiences and registers
- Maintain current awareness across AI safety, capabilities, and governance sufficient to know when new developments require graph updates, and to know where to find authoritative further detail
- Collaborate with communications staff and researchers to ensure outputs serve real persuasive needs
Required Qualifications
- Working familiarity with formal or semi-formal argumentation theory (abstract or structured argumentation, defeasible reasoning, dialectical models, or argumentation schemes)
- Experience with ontology engineering or knowledge graph development (OWL/RDF, property graphs, or equivalent)
- Operational experience with LLM agent systems: agent coordination platforms, prompt engineering at scale, and QC regimes for LLM outputs (adversarial probing, consistency checks, calibration)
- Fluent vibecoding practice: rapid prototyping and shipping with LLM-assisted development in production-adjacent contexts
- Substantive grounding in AI safety, AI governance, and current frontier-AI dynamics, with the literacy to locate authoritative sources on any sub-topic or human expertise in the space
- Familiarity with philosophy of science concepts bearing on evidence: defeaters, burden of proof, inference to the best explanation, underdetermination
- Good coding skills; comfort with graph databases or query languages
- Experience designing cross-check and verification scaffolds for unreliable automated processes
- Sound judgment about when a claim is well-supported versus when it needs hedging, further substantiation, or withdrawal
- Self-directed; strong written communication
Preferred Qualifications
- Graduate work or equivalent depth in argumentation theory, computational argumentation, epistemology, or philosophy of science
- Familiarity with AIF, Carneades, or comparable computational argumentation tools
- Track record in AI safety or governance (publications, policy work, or substantive community contributions)
- Background in argument mining, claim extraction, or stance detection
- Experience with debate formats or structured deliberation methods
- Understanding of motivated reasoning, belief change, and cognitive biases as they bear on communications strategy
- Open-source contributions in any relevant area
CARMA/SEE is proud to be an Equal Opportunity Employer. We will not discriminate on the basis of race, ethnicity, sex, age, religion, gender reassignment, partnership status, maternity, or sexual orientation. We are, by policy and action, an inclusive organization and actively promote equal opportunities for all humans with the right mix of talent, knowledge, skills, attitude, and potential, so hiring is only based on individual merit for the job. Our organization operates through a fiscal sponsor whose infrastructure only supports persons authorized to work in the U.S. as employees. Candidates outside the U.S. would be engaged as independent contractors with project-focused responsibilities. Note that we are unable to sponsor visas at this time.