Thoughts on starting new projects with LLM agents

My advice for new projects is to always do the first scaffolding yourself. LLMs are great if they don't have to make decisions. But setting up a new project is both simple and full of decisions. If you do it manually, then you will have much better experience with LLM filling the blanks.

My big recommendation is to have the LLM map out the project structure up front, and then have the LLM come up with a phased plan for each feature, and implement it on a branch. Ask the model to do a code review after each phase is done as well. Then you end up with a set of small focused PRs which each do one thing and can be reviewed in a sane fashion. These two papers are also very much worth a glance, they explain how up front planning helps significantly improve the competency of models:

• https://arxiv.org/abs/2509.16198
• https://arxiv.org/abs/2409.03733