I have always assumed that it would be aging, ultimately, that pulled me back down into the fat part of the thinking distribution. Recently, I have started to worry that there is another effect that could pull me down much faster - cognitive fitness erosion due to AI.
It's ironic that objectively speaking, I can now absolutely smoke my younger self with the help of AI. Combining the knowledge, experience and intuition that I have developed over the years with the amazing processing power of current generation LLMs, I feel awesomely smart today. I can do so much more than I ever could have before. But there is this nagging feeling that it is all about to fall apart because my knowledge, intuition and reasoning ability are about to rapidly decline due to declining fitness and depth of understanding.
When I changed majors from math to philosophy in college, I had the good sense to keep taking math, math econ and physics classes to "stay in shape." Even after graduating I kept working on things that exercised the math muscle. That helped a lot when, 10 years after taking my last math class, I entered graduate school in math. That was a pretty good workout.
Over the many years since then, I have always made sure to "get my workouts in" - starting an OSS math library; staying very hands-on as a tech leader; working on math-econ problems and continuing to read papers in math, stat, CS and other technical fields; leaning into hard technical problems at work. Every weekend I do something hard technical and if I don't get a good "workout" by mid-week at work, I try to squeeze in a workout at night during the week.
I am sure this regimen has helped keep me able to penetrate hard problems. Recently, however, I feel like I have done the running equivalent of substituting easy jogging for the intervals in my workouts and I am getting worried about the fitness consequences of that.
Here is a concrete example. The following would have counted as a "mid-week workout" were it not for the jogging in place of intervals. At work, I needed to define and specify an algorithm for an order-preserving transformation of a set of scores so that selected values in the range of the scores ended up hitting specific quantiles. I knew that this could be done easily with a piecewise linear transformation and some simple CDF inversion. I drew pictures and waved hands, but did not see comprehension, so I decided to spec it out fully. Pre-AI, this would have been a nice little workout. Derive the formulas, fuss with the LaTex, create some worked out examples and write a little starter Python. But last night I could just ask Claude to do it. After a two-sentence prompt, Claude created a nice .md with embedded LaTex, worked examples and starter code. I reviewed carefully and made some edits. But I did not derive the formulas myself. I did not work out the examples myself. I just jogged through the review.
A lot has been written about the benefit of "the struggle" in working hard problems out for yourself and how LLMs are taking that away. I agree with that, but I also think even just the simple, easy crank-turning stuff like deriving the formulas above is critical to maintaining cognitive fitness.
I don't think that the answer is to stop using LLMs or to force myself to do everything by hand. The trick is how to get a really good workout working with the LLM. What I have started to play with is a technique that I used to use when reading math papers which is to read a section and then try to generate what is about to come in the next section by myself. So when reviewing the LLM work product, I select some key sections and when I get there in the review, I do just that part by myself, validating against the LLM output. That is like putting a little fartlek into the jog through the review. In the example above, I did that in the section where the CDF of the scores is inverted and the parameters of the transformation are derived. Not a full workout, but more than a jog.
Another thing that I am focusing on is trying to spend more time in cognitively demanding conversations, often using LLM artifacts as context. Amazon realized early that technical conversations are much more productive when they start with documents. A good way to leverage the AI surplus is to create documents in advance of technical meetings. Preparing in advance and answering questions in real time about docs that you create with LLMs is a good way to get to and sustain an elevated cognitive heart rate for a while. It also ensures that you really understand the content.
I guess that I am ultimately optimistic here. I just need to modify my workouts to make sure they are hard enough and I can fit them into my routine. It's kind of like when my knee eventually says "no mas" for running I will need to pick up rowing or something. Anybody have a good cognitive rowing machine they can recommend?
No comments:
Post a Comment