An AI agent that acts for you has to distinguish what you will never accept from what you would simply prefer. Aarik Gulaya, who built Base Layer, draws the line this way:
A rule can't be broken, a preference can. Preferences can turn into rules, but then a preference that becomes a rule and didn't start out as a rule is like a second-class rule.
Rules and preferences for an agent
A rule marks the edge of what an agent may do for you. A preference guides its choices inside that edge and can give way. A preference you hold can also become a rule for the agent that acts for you:
If you have a preference as a user, that could become a rule for the agent acting on your behalf.
Base Layer's research describes these limits as overlapping behavioral envelopes, one each for the AI provider, the organization and the individual. In his view, an agent is out of bounds only when it breaks a rule. Everything else inside the envelope is allowed:
An envelope is only broken when a rule is broken, otherwise all actions within that envelope are permissible.
The research page on behavioral envelopes works through one example. It is a work in progress.
An example
Take a rule against flights before eight. If you later override it, that raises a question about what kind of limit it was:
If you have a rule, no flights before eight, and that rule is overridden, was it ever really permission? Or was it always a preference linked to a particular permissive act?
What is still open
The line itself is not settled. He is still asking how to tell a policy from a preference and when a preference turns into a rule:
how do we define the difference between a policy and a preference? ... When do preferences become rules?
His answer above treats a rule that started as a preference as second-class. What that should mean in practice is still open.