QA Analyst – AI Solutions (Portugal-based Remote/Hybrid)

Sword Health is on a mission to free two billion people from pain. With 67% of members achieving a pain-free life and a 70% reduction in surgery intent, at Sword, we are using AI Care to change lives, and save millions for our 25,000+ enterprise clients across three continents. Today, we hold the majority of industry patents, win 70% of competitive evaluations, and have raised more than $400 million from top venture firms like Founders Fund, Sapphire Ventures, General Catalyst, and Khosla Ventures.Recognized as a Forbes Best Startup Employer in 2025, this award highlights our focus on being a destination for the best and brightest talent. Not only have we experienced unprecedented growth since our market debut in 2020, but we’ve also created a remarkable mission and value-driven environment that is loved by our growing team. With a recent valuation of $4 billion, we are in a phase of hyper growth and expansion, and we’re looking for individuals with passion, commitment, and energy to help us scale our global impact. Joining Sword means committing to a set of core values, chief amongst them to “do it for the patients” every day, and to always “deliver more than expected” on behalf of our members and clients.This is an opportunity for you to make a significant difference on a massive scale as you work alongside 1000+ (and growing!) talented colleagues, spanning three continents. Your charge? To help us build a pain-free world, powered by AI, enhanced by people — accessible to all.About Sword Intelligence: At Sword Health, we spent years perfecting AI systems across our own patient population of millions, learning what works in real healthcare environments. Now, through Sword Intelligence, we’re taking those battle-tested technologies and productizing them for health systems, governments, and insurance companies worldwide. See our website for more information.About the role: Hospitals are hemorrhaging staff, insurance claims are drowning in bureaucracy, and governments can’t keep up with demand. While everyone else talks about “digital transformation,” we’re actually doing it – deploying human + AI solutions that work in the real world, not just in demos.As our QA Analyst, you’ll be the guardian of quality for AI Agents that are already changing healthcare. You’ll own the complete quality lifecycle—from pre-launch validation to continuous post-deployment improvement. This means you don’t just catch problems; you solve them, monitor for new ones, and proactively strengthen our AI Agents.Think of it as being a quality engineer for the future. You’ll identify issues, implement solutions, measure the impact, and continuously evolve our AI to perform better. When you bolthires a problem with how our AI handles a medical scenario, you’ll trace it to the root cause, modify the underlying prompt/logic, validate the fix works across different contexts, and establish monitoring to prevent similar issues.This is quality assurance that actually improves the product, not just documents what’s wrong with it.What you’ll be doing: Ensure AI agents perform consistently before and after going live;Monitor performance across different healthcare environments and languages;Establish quality standards that work for AI systems, not traditional software;When the AI Agent does not follow the defined protocol, trace why it happened;Fix the underlying prompt logic and reasoning chains;Create new testing methods to prevent similar issues;Figure out how to test systems designed to be intelligent and adaptive;Build frameworks for testing across infinite healthcare scenarios;Define what “good enough” means for AI in healthcare;Manage compliance with MDR (Medical Device Regulation) and AI Act requirements for AI systems together with the compliance team;Document quality processes and verification and validation procedures for regulatory submissions;Help ensure the algorithms perform with the expected accuracy and do not lead to bias;Work with compliance teams to achieve and maintain certification of the relevant algorithms;Help establish industry standards for AI quality assurance;Become one of the few experts in AI healthcare testing methodologies.What you need to have: Comfortable speaking and understanding English (doesn’t need to be perfect);2-6 years of experience in QA, testing, or analytical roles;Strong analytical and problem-solving skills;Comfortable learning new technologies and methodologies;Curious about the evolving AI landscape.What we would love to see: Familiarity with secure handling of health data (GDPR, HIPAA);Experience with medical device regulations (MDR, 21 CFR 820, ISO 13485, IEC62304);Comfortable speaking and understanding Portuguese;Experience with AI tools or prompt engineering;Background in healthcare, enterprise software, or regulated industries;Familiarity with quality metrics and monitoring systems;Basic programming or scripting experience.Portugal – Sword Benefits & Perks:• Health, dental and vision insurance• Meal allowance• Equity shares• Remote work allowance• Flexible working hours• Work from home• Discretionary vacation• Snacks and beverages• English classNote: Please note that this position does not offer relocation assistance. Candidates must possess a valid EU visa and be based in Portugal.Sword Health complies with applicable Federal and State civil rights laws and does not discriminate on the basis of Age, Ancestry, Color, Citizenship, Gender, Gender expression, Gender identity, Gender information, Marital status, Medical condition, National origin, Physical or mental disability, Pregnancy, Race, Religion, Caste, Sexual orientation, and Veteran status. Apply tot his job Apply tot his job