A case to look through
A firm
How can an organization remain redirectable without sharing all of the purposes of the people who depend on it?
A stylized profit-seeking firm illustrates an alignment architecture based less on shared values than on structured dependencies, institutions, and continually adjustable control handles.
Start with the puzzle
The firm does not have to care about everything its environment cares about.
Consider the simplified economic model of a firm pursuing profit. Its customers may care about useful products, workers about wages and working conditions, governments about taxes or pollution, and owners about returns.
The firm need not internalize all of those values as its own objective. Prices, contracts, law, ownership, regulation, and resource dependencies can instead alter which profit-seeking actions are available or attractive.
This makes the firm useful for separating value agreement from the architecture that makes a capable system redirectable.
Try looking at this case through
Alignability
The firm's behavior can be redirected through changes in its environment without requiring corresponding changes in its objective.
LensInstitutional / procedural alignment
Contracts, law, ownership, and procedures structure what the organization can do and how it interacts with others.
How it happensIncentives and payoffs
Changes in prices and other consequences alter which actions satisfy the firm's existing local objective.
DissociationAlignability ≠ value alignment
Redirectable behavior does not imply shared values.
Try breaking it
Remove the control handles.
Imagine that harms cannot be measured or priced, contracts cannot be enforced, ownership conveys no control, regulations cannot alter permissions, and customers cannot redirect spending.
The firm's internal capabilities may remain intact while much of its external alignability disappears.
Important limit
A well-steered system can still be steered toward the wrong thing.
Institutions and incentives do not choose their own targets. A price can omit an externality. A regulation can encode a bad objective. A governance structure can serve the wrong constituency.
The existence of effective control handles therefore does not settle the upstream question of what they should be used to accomplish.
Look again