Weekly Coding • 2026-07-19

Comparing AI Models in a Specific Scenario

I decided to conduct a code review for my project using LLMs. This time, the experiment involved a project I wrote as part of my paid work. The task for the LLM was to check every file—and if necessary, read related files to better understand the context. A separate agent was launched for each file because, in this local setup, there’s no need to save tokens 👌 I wasn’t sure what to expect at first, but after completing the process, one specific piece of code caught my attention: shown 🖼️ in the attached screenshot.

The code is generally correct: we try to perform an action, and if it fails, we throw the last Exception. We only call onSuccess() for successful operations. However, some AI models stumbled over this:

• Qwen 3.6 27B 🔴
  • Immediately issued a critical remark without even reading the code of the retryWithCatch() function.
• Qwen 3.5 27b Claude Opus 4.6 reasoning distilled 🔴
  • Read the retryWithCatch() code but still issued a critical remark. Did it read it and just not understand? Hmm…
• Gemma 4 31B QAT 🟢
  • Read the retryWithCatch() code; everything is OK. It works slowly but thoroughly.
• Qwen 3.6 35B A3B 🟢
  • Read the retryWithCatch() code; everything is OK. It works fast, and it makes mistakes just as fast…

As we can see, Qwen 27B has some issues understanding code… I think a human in this situation could easily make the same mistake due to inattention: glancing at the code and assuming that onSuccess() after retryWithCatch() marks all operations as “successful” without looking closely 🥴.

At the same time, Qwen 27B has a high score in the Artificial Analysis Intelligence Index 🤔💭 But in my opinion, the logic of the other two models on this list is better. Though it’s nothing surprising—after all, they have more parameters: 27B < 31B < 35B.

Other Coding in Spare Time

Besides testing LLMs, I did a few other things for my projects last week during my free time:

1. New refactoring of the personal website: I removed the intermediate web server used for generating static HTML content. There will be less redundant code—which is good…
2. Setting up a sandbox for pi.dev: Running without internet access. Access to the local file system is also restricted: only one project folder is allowed. The agent can no longer reach other files, even if it wanted to…

P.S.: The lead developer in the client’s team said that AI code review is now a mandatory development stage for every feature. Moreover, this must be done immediately, even before opening the PR: you write the code yourself, you run the LLM yourself, and you read and fix the remarks yourself 💻🧐 Only after that do you open the PR; the human reading your code only has to check if the “passed LLM review” checkbox is ticked ☑️🛫