commit bf1b2d7502136d8615dfe0cd4ea6fdd3c7a96173
parent 1f86baf8f30b0cdbe1da9a757f87fe29b95739e6
Author: Eamon Caddigan <eamon.caddigan@gmail.com>
Date: Sat, 15 Mar 2025 20:46:03 -0700
Update post with new evidence and reflections
Diffstat:
1 file changed, 17 insertions(+), 0 deletions(-)
diff --git a/content/posts/coding-assistants/index.md b/content/posts/coding-assistants/index.md
@@ -3,12 +3,14 @@ title: "Programmers should reject LLM-based coding assistants"
description: "Even without the ethical issues present in the coding assistants
that exist, these tools are fundamentally unfit for the job."
date: 2024-02-22T20:36:52-08:00
+lastmod: 2025-03-15T20:38:22-07:00
draft: false
categories:
- Programming
- Data Science
tags:
- R
+- LLMs
---
The complexity of our world is beyond the limits of human comprehension. In
@@ -175,6 +177,21 @@ as spaces in need of creative solutions. Encourage junior programmers to
develop a deeper understanding of the tools they use currently, and insist
that your colleagues at all levels imagine something better.
+## Update 2025-03-15
+
+In the year since I wrote the above, I’ve spent time with the LLMCA features
+in the Databricks platform, and I indeed find them to be a nuisance which I
+have to continually disable. Rather than suggesting the autocompletions I’d
+expect from (e.g.) a language server---variables and functions in the current
+scope, stuff like that---it suggests non-existent tokens and would (if I let
+it) produce broken code. By replacing functionality I expect with something I
+don’t want, it’s literally worse than useless,
+
+[This blog post from Rob
+Bowley](https://blog.robbowley.net/2025/02/01/the-evidence-suggests-ai-coding-assistants-offer-tiny-gains-real-productivity-lies-elsewhere/)
+arrives at a similar conclusion, now backed by data that weren’t available
+when I started thinking about this stuff.
+
[^science]: For its part, science can be a great tool for exposing the
limitations of these mental models. But at the end of the day, it’s
still only producing different, hopefully better models, operating at
