Is it okay to lie to the AI Agent?
One of the hardest parts about building AI agents in your org is that you cannot rely on them to keep private information hidden across boundaries.
For example if one agent has access to both your DM's and a public channel on slack (or Zulip) and it's able to read/write to both places, then you face a real risk of your private messages getting leaked into the public channel. It could be by pure mistake of the model. Even with good prompting you just can't really guarantee it won't get confused.
This confusion problem is solvable by better models and better security frameworks. But even when...