I think this is true even in what seems like their natural use case I’m computer programming. In the hands of an inexperienced programmer these tools can feel empowering but you run a lot of risks around how software works, how maintainable it is, how it can continue to iterate, how secure it is.
It occurs to me that this is key to breaking down one of the ways that pro-AI and anti-AI people tend to talk past each other. There are things that AI can do *for* us, and that list of things is likely to grow. But I'm not seeing any serious examples of things that AI can do *with* us.