Is Your Writing Yours?
How To Think About The Risks And Benefits Of AI Authorship
The more I write the more paranoid I get about whether or not my writing sounds like AI or whether the ideas I’m sharing are truly mine and to what extent.
I keep asking myself, what is the right amount of friction in my process that would allow me to exercise my writing and thinking muscle and still take advantage of what AI can provide in the research and editing process.
Can we truly use AI and be good writers? Can we truly get better at producing insightful and original pieces while still using AI as an assistant?
When it comes to knowing your contribution and learning through the process, I have bad news. The answer is closer to a “no” and it is complicated.
It turns out we are very poor judges of our thinking process.
When you decide to use AI to plan, research and write an essay, you may feel you’re in control because you are doing the thinking, writing the prompts, and evaluating the output. That sense of control feels very real. Unfortunately, the literature so far indicates that it is all an illusion.
The introspection illusion as a concept has been studied for several decades (Nisbett & Wilson, 1977; Pronin, 2009). The insight is simple: we cannot reliably judge our own bias, nor can we trust the introspective analysis of our own thought process. And when we compare our judgment to others’, we fail to compare them fairly.
Most recently, Jakesch et al. (2023) developed a co-writing persuasion experiment in which 1,506 participants wrote about whether social media was good for society with the help of an opinionated AI assistant. The findings show the assistant shifted both what people wrote and what they later reported believing, with the AI influence opaque to them.
There is another psychological quirk that makes it even harder to assess whether what we are writing has been influenced or modified by AI. The Anchoring Effect is a cognitive bias in which we rely too heavily on the first piece of information we receive when making decisions.
For example, it is easy to find evidence of people using AI to generate the outline of whatever they are trying to write. In their words, this outline helps them deal with the terrifying prospect of a blank page. The issue is that once you get the outline, that outline is your anchor, and the research supports that it is extremely hard to deanchor yourself from that information (Epley & Gilovich, 2005; Wilson et al., 1996).
But what if you use AI in a way that constantly pushes back?
Maybe you create excellent system instructions or just magnificent prompts. If the AI is pushing back and helping shape your thoughts, then it is impossible to separate your thinking from the influence of AI.
Psychological ownership is a thing. When you use these models to help you produce better outcomes, you may feel like you own the thinking, but as you continue to use it you will also feel that it is not completely yours, and that duality is unsettling.
Should I stop using AI for my writing?
It depends on how the AI is designed to interact with you. Most of the research I’ve been reviewing points to major risks of cognitive offloading. However, there is also early evidence that if you can manage to force the model to only give you hints, forbidden from giving you final answers, let’s call this tutor mode, you can minimize harm (Bastani et al., 2025) and, in other studies with similar guardrails, even increase your rate of learning (Kestin et al., 2025).
The problem I have with the tutor use case is that writing is not always a guided learning process. Writing is a messy process of research, curiosity, asking what we really believe in, looking at the evidence and what we want to communicate. That is a messy process in which we are not always trying to learn concepts but connecting the ones we already know in new ways.
The common advice popularized by AI academics, practitioners and researchers is that maintaining friction where it matters is helpful. For instance, allowing AI to execute repetitive tasks associated with research or data structuring can be beneficial to the writing process. However, the consensus is that this AI application would be most dangerous for a novice: the premise is that without latent knowledge you don’t know the nuance around the insights you are evaluating, and therefore you can’t truly maintain epistemic integrity.
Then the question becomes, how do you know you are truly an expert or have enough experience to judge the results? As we established before, we don’t seem to be good at this, and we definitely don’t have good and objective ways to measure experience and expertise.
I have an MSc in Applied Neuroscience. Does that mean I can just assume I have enough expertise to judge the AI-assisted research process on subjects like cognition? Where does my expertise start and end?
Side note, my heart breaks for the millions of students and recent graduates who haven’t been given the chance to truly form experience and implement knowledge in the real world in a way that can help them protect themselves from offloading their cognitive capabilities. This is going to bite us in the butt in a couple of years, in ways that we, as a society, will feel ashamed of.
But… everybody is doing it anyway, right?
There is a massive gap between what organizations expect from AI-assisted authorship and what people are doing with AI. The convergent position across publication-ethics bodies and major publishers mostly focuses on different levels of disclosure based on the extent of the assistance. The general consensus among academic publishers is that LLM use should be disclosed when it reaches significance, with a line drawn between stylistic copy-editing, which does not require disclosure, and actual text generation, which does; some publishers are stricter and require disclosure of any use.
For non-academic writers the picture is thinner. For instance, the Authors Guild has pushed model contract clauses on AI disclosure, Amazon KDP requires sellers to disclose AI-generated content to Amazon (not to readers), and some fiction venues banned AI submissions outright after being flooded.
The reality is that detected AI writing has increased in both academic and non-academic fields, and disclosure has become harder to validate. The disclosure regime asks authors to self-report a quantity of AI use that, as we already discussed, they cannot accurately know even if they want to be transparent.
Today, when we read a post on Substack and identify well-known patterns like “It’s not X, it is Y” we immediately assume the post is likely to be AI generated, or at least heavily modified. But how do you verify that? There is no SynthID for text, because inherently it cannot contain any metadata or invisible embedded watermarks.
I used that pattern many times in my writing long before ChatGPT arrived on the scene. I have “vintage” writing to prove it. I always felt immense satisfaction when I was able to craft that type of sentence, because it told me I understood something about the subject deeply enough to clarify it in one sentence. Now I’m nervous about writing it. That pleasure is gone. Good job people.
So should you stop using AI for your writing?
Unfortunately, this is a subjective decision. Only you can determine whether or not you are comfortable with the type of delegation you do, and whether or not you feel you can defend the arguments in your writing at any point. So far, nobody has invented a scientific way to know how much effort is good and how much is yours.
There is some consensus and evidence that by intentionally “handicapping” the AI’s ability to give you a final answer, you transition the tool from a “ghostwriter” (which can lead to skill atrophy and what Fan et al. (2025) call “metacognitive laziness”) into a “scaffold” that supports learning and retention. That’s where we stand today.
What does all of this mean in practical terms?
Based on the current research, we can have some confidence that:
Using unguarded AI tools (free tiers with default settings) with no subject matter experience means you will be atrophying your capabilities. The question is, how do you know you have enough experience on a topic at any given time, and how do you know you have the right guardrails if models keep changing?
We are poor judges of our own thinking process, so determining the right level of friction to avoid offloading too much is hard. There is no scientific way right now to measure how much of your writing was your thinking and how much was AI bias.
So far, the only way we assess whether a piece of writing is good even though AI was used in the process is moral judgment: a sense of how much effort something took. For example, if I told you it took me a week to write this essay between researching, analyzing and organizing thoughts, but I only used AI for gathering data and to explain concepts in the research, would that be enough effort? What if I told you I used AI to help proofread my final draft? Would that cross a line? What if I use Grammarly instead of Claude? Would that make a difference in how you feel about someone’s AI-assisted writing?
I would love to hear what type of AI disclosure you would like to see from authors, both in academia and in non-fiction work like the one you are reading right now. Maybe most importantly, if you write often and use AI in the process, what would you feel comfortable and proud to disclose?
This made me wonder where we draw the boundary around influence.
If we're poor judges of our own thinking, vulnerable to anchors, and often unaware of how our ideas are shaped, why would that concern apply only to AI?
It seems like the same argument could be made for schools, universities, professional training, media, and countless other systems that influence how we think.
Maybe AI isn't revealing a new problem so much as making an old one harder to ignore.
Sofia, your writing has captured a couple hours of my attention today and in a feeding-the-hungry mind kind of way! The introspection-illusion problem you raise is the hardest one, and I won’t wave it away. The research you bring is serious, and the discomfort it points to is real: we don’t always see clearly how much of our own process is ours.
But I’d reframe what the question even is. “Is the writing mine or the model’s” assumes authorship is a fixed quantity to be divided. I don’t think it is. Authorship is a matter of how much agency and precise thought sovereignty the human has integrated through their own iterative building. The question isn’t who supplied which words. It’s whether the human directed the thinking, the structure, and the evolution of the piece, or surrendered them.
That’s also why I don’t use the term thought partner, and I’m deliberate about it. Partner, co-author, collaborator, assistant: each one hands the mechanism more agency than clear governance warrants, and anthropomorphizes it. What I work in instead is what I call a Hybrid Ecology: an interactive field where idea development happens, where the human directs the words, the thought development, and the structure, and the model reflects, asks meaningful questions, pushes back, and offers iterative response to what the human is building, guardrailed against sycophancy in a way that tempers over-agreeability without assassinating creativity.
The human stays the author because the human stays the source and the director. The model is reflective structure, not a second mind. If we can’t reliably introspect after the fact, the answer isn’t to declare authorship unknowable. It’s to make the process auditable while it happens.
When you fence your own source text so you can see what the model added versus what you brought, when you label what’s warranted versus inferred as you go, when you revise one sentence at a time against your own meaning, you leave a trace. You don’t have to trust your memory of how much was yours. You can look at the record. The tutor-mode research you cite points the same way: the gain comes from forbidding the model the final answer. That’s a governance move. Treat the constraint, not the tool, as the thing you design, and it generalizes from tutoring into ordinary writing.
So I’d gently challenge the polarization itself, human-authored versus AI-authored. It’s the wrong axis. The real axis is whether mature, literate writers and builders bring discernment and integrity to the interaction, and whether we codify what they do into shareable methods and pedagogy. That’s the work I think is in front of us: not adjudicating how much effort counts, but building transferable governance practice until responsible human-AI interaction becomes a norm rather than a private struggle each writer wages alone.
I work through this in my Relational Autotheory white paper and in a recent piece called Warnings Are Not Solutions, if you’d like the practical version. Your essay is the right kind of unsettled.