--- title: "The Ethics of the Tools We Build" description: "Systems that rank, predict or make decisions about people can cause harm even when nobody intends it. Applied ethics examines who is affected, which values guide the design, and how different moral fr" canonical: https://lightmysky.com/learn/personal-and-social-development/the-ethics-of-the-tools-we-build-mt_eZh-vIoluR source: https://lightmysky.com/learn/personal-and-social-development/the-ethics-of-the-tools-we-build-mt_eZh-vIoluR.md retrieved: 2026-11-12 --- > **Agent view.** This is the Markdown twin of the page, for tools and assistants. > When to use this site, and the call that answers each job: https://lightmysky.com/agent-instructions.md > API description (OpenAPI 3.1): https://lightmysky.com/openapi.json · Authentication: https://lightmysky.com/auth.md > Pricing: https://lightmysky.com/pricing.md · Catalog: https://lightmysky.com/llms.txt · Full catalog: https://lightmysky.com/llms-full.txt > Every machine-readable file on this domain: https://lightmysky.com/.well-known/ai-catalog.json > Ask for Markdown with `Accept: text/markdown`, a `.md` address, or `?mode=agent`. # The Ethics of the Tools We Build Systems that rank, predict or make decisions about people can cause harm even when nobody intends it. Applied ethics examines who is affected, which values guide the design, and how different moral frameworks judge the same technology. Subject: Personal & Social Development · Area: Ethics & Philosophy · Ages 16 to 18 Page: https://lightmysky.com/learn/personal-and-social-development/the-ethics-of-the-tools-we-build-mt_eZh-vIoluR ## Ready when they can - Take one automated decision system and say who bears the cost when it is wrong - Apply two of the three moral frameworks to the same technology case and compare their verdicts - Explain a challenge in arguing for duties to future people or other species, without assuming those duties must be weaker ## Lesson: Who is affected by a tool? An error rate reports what happens. Calling a tool fair also needs a value judgment. A hiring filter trained on past decisions may repeat discrimination. Its history identifies a risk, not proof that every result is biased. Design choices, such as the score required to pass, can affect who gets an opportunity. Expressing a decision as maths does not settle whether it is fair. Before launch, identify affected people, compare errors across relevant groups, and provide a challenge route. Define errors by the tool's task. If it only checks eligibility, rejecting someone who qualifies is a mistake; eligibility alone need not guarantee selection for a scarce place. A mistaken housing rejection can cost a rental opportunity without causing eviction. In the fictional chart, each group has 100 eligible applicants, and the bars count mistaken eligibility rejections. *(drawing: Each group has 100 eligible applicants. The bars show mistaken rejections, not all rejections. Investigate the gap.)* Consequentialism judges choices by their consequences. A triage tool might save more lives overall yet miss urgent cases in one group more often. Compare expected benefits and harms for everyone affected, including trust and future effects. Counting only the extra lives saved leaves out other consequences. Different consequentialist theories may value outcomes differently. Compare frameworks by applying each rule to the same options. A greatest-net-welfare rule selects the best expected balance of benefits and harms for everyone. A duty rule rejects options violating its stated requirement, such as fair treatment or informed consent. The rules may choose differently when the welfare rule favours an option that violates the duty. These are specified views, not fixed verdicts for every theory. Consent does not excuse every harm. Future people cannot join today's decisions, and long-term effects may be uncertain. These difficulties concern representing their interests; they do not prove weaker duties. For other species, arguments may concern individual animals' suffering, species survival or ecological relationships. These are different grounds for protection. Inability to speak or consent does not by itself settle moral importance. **Recap.** Identify affected people, compare group errors, and explain the consequences or duties behind your judgment. Difficulty representing future interests does not prove they matter less. *This lesson, quiz and flashcards were written for this dataset. Responsible Computing and Introduction to Philosophy informed the concept checks. The test data and trial comparison are fictional.* ## Practice 8 questions on this page, each with its working shown. ## Needs first - [Ethical Frameworks and Moral Reasoning](https://lightmysky.com/learn/personal-and-social-development/ethical-frameworks-and-moral-reasoning-mt_LPYPuSaxv_) ## Optional connections - [Bias in AI Systems](https://lightmysky.com/learn/computing/bias-in-ai-systems-mt__BbOjiY5A5) - [Justice: Who Deserves What, and Why](https://lightmysky.com/learn/personal-and-social-development/justice-who-deserves-what-and-why-mt_umi3pKeHa1)