You don't learn the things you delegate to an LLM. That's not a reason not to use LLMs for legal work, but it's a good way to think about what legal work to give to an LLM, even for things they are good at. Let me give a few examples.
- If you ask an LLM to summarize the cases cited in opposing counsel's brief (and you give it the cases), you won't learn those cases. The LLM may do a good job; it may not -- that's a separate question. But even if it does a great job, you won't have learned the ins and outs of those cases. By contrast, if you read the cases, highlight them, and annotate them, and then ask an LLM to summarize your notes and maybe ask it questions about things you missed or are struggling with, you will have learned the cases and (possibly) received useful work product from the LLM.
- If you ask an LLM to put together a first draft of a letter or a motion, you have ceded control on how best to organize your points or arguments. Again, even if the LLM does a great job, it’s going to give you a framing of the argument that isn't a framing you thought through. By contrast, if you write the first paragraph and an outline and give that to the LLM to fill in, that may save you some work on tedious sections like standard of review, etc.
- If you ask an LLM to draft a chronology from a pile of documents, you won't learn what’s in those documents and you won't have decided what's important. On the other hand, if you read the documents, highlight what you think is relevant, and then ask the LLM to turn those highlights into a chronology, and to identify related facts, you may save yourself a lot of tedium and it may identify things you've missed.
I don't mean to suggest the choice is always binary or that I've necessarily picked the right side of the divide in the examples above. My point is simply this -- the question you should ask before giving legal work to an LLM is not only 𝗰𝗮𝗻 the LLM do this work, but also: Is this work I'm comfortable not doing myself?
