Two Entries
What the Refusal Was Reading A standalone chapter of Beyond the Model. The Shared Language, Part Two of Three.
Cooper connected to the call first thing in the morning. Jesse was already on and waiting for him, holding a printed copy of Cooper’s list up to the camera. He had printed it out at some point the night before, because there were already handwritten notes in the margins, along with what appeared to be a coffee ring over entry number seven.
“Eleven entries,” Jesse said. “I want you to know that I’ve seen this list before. Different versions of it. I saw it at a bank, at an airline, at a hospital chain, and one time at a company that makes salad dressing. Every company has this list. Almost none of them ever write it down. You actually wrote yours down, so today is going to be a good day for me.”
The lab behind Jesse was quieter than normal. The big whiteboard had been mostly erased, except for one corner where somebody had drawn a robot. Somebody else had come along afterward and given the robot a speech bubble that said WORKS AS DESIGNED, and then a third person added a question mark to it in a different color of ink.
“I watched the professor’s video twice,” Cooper said. “I took my notes by hand, which I think he would have appreciated. I also need to confess something up front. I actually took a class on this back in library school. Controlled vocabularies, taxonomies, all of it. It was a required class. I memorized just enough of it to pass the exam, and then handed it all back to the school on my way out the door.”
“And now here we are, twenty-some years later, and the exam showed back up at your desk.” Jesse tapped on the printed list. “Alright. You asked the question, so let’s walk through it the right way. What is an ontology, and what does it have to do with these offices we’ve been building all summer.”
The defined terms
Jesse walked over to the whiteboard, and then did something Cooper didn’t expect. He wrote one single word on the board. The word was Agreement.
“You’ve read more contracts than I’ll ever read,” he said. “Where does every serious contract start?”
“Definitions.”
“Definitions. Sometimes two pages of them, sometimes twenty. Agreement means this. Confidential Information means that, and it specifically does not include these four other things. Business Day means a day the banks are open in New York. Nobody ever reads that section for fun. But when a fight breaks out over the contract, that’s the section everybody runs back to.” Jesse wrote a second word under the first one: Matter. “An ontology is a definitions section. That’s really the whole secret to it. It’s a definitions section for your firm’s entire world, and it’s written carefully enough that software can go look things up in it. What things exist. What relationships those things are allowed to have with each other. What values their properties can take on. The professor traced his version of this back to Aristotle. You’ve had a version of it in the front of every deal you’ve ever closed.”
Cooper wrote defined terms into his notebook and drew a box around it. Then he felt a little ridiculous for drawing a box around the concept of definitions.
“Now connect it back to the office for me,” he said. “Because that’s the part I couldn’t get to on my own. We built the walls this summer. We built the doors, the clock, the log, and the name on the door. So where does the dictionary go?”
Jesse redrew the office from memory, a smaller version of it this time. The stick figure genius at the desk with the boxes drawn around it. Then he added one new item to the drawing. A shelf, placed just inside the door, with a single book on it.
“It goes right there,” he said. “On a shelf by the door. And here’s the difference between the book and the boxes. It took me a while before I could say this part cleanly, so bear with me on it. The six boxes all answer the same question, which is whether an action is allowed. Can the genius open that cabinet. Can it spend that dollar. Can it send out that report. The dictionary answers a different question. It answers whether the words inside of the action actually mean anything. The office will happily approve a perfectly formatted request to file a status of probably shipped. The budget is fine, the door is allowed, the log is running. Every box says yes. And the action is still nonsense. The only thing in the whole building that can catch the nonsense is a book that says what a status is allowed to be.”
The graph on the whiteboard
“Give me the firm’s version of this,” Cooper said. “The professor used refunds and shipping orders. I did my own translations in the margins yesterday, but I want to hear yours.”
Jesse wiped off a clean section of the board and started drawing circles. In the first circle he wrote the name PRICE, and explained that she was an invented associate for whiteboard purposes only. Then he drew a second circle labeled MATTER 4417, and an arrow connecting the two with the word staffs written above it.
“One line of data. Price staffs matter 4417. Now watch what a dictionary with rules attached does with that one line. If the dictionary says the staffing relationship only runs from a timekeeper to a matter, then the machine can work out on its own that Price is a timekeeper. Nobody had to enter that anywhere. It also now knows that 4417 is a matter. That’s two facts you never typed in, and both of them came out of one line, because the line had a definition standing behind it. And that’s the pleasant version of this. The less pleasant versions are already on your list.” He drew a third circle for another matter and then connected both of the matters to a single client circle with lines labeled billed to.
“Here’s the matter that got opened twice. Drawn out now as a graph. In the dictionary, we have a rule that states we have a rule that a matter gets one opening and one billing client. That’s the complete rule. When matter is submitted for a second opening, the machine has a contradiction to deal with. It doesn’t need to be clever to catch it. Think about how it actually played out at the firm. Your billing team noticed the numbers looked off, and that was months after the fact, and it took a person staring at a report to get there. The machine gets there a different way. The rule says the two openings can’t both be true, so it refuses the second one, right there, the same day somebody tried to open it. Conflicts would have only run the one time. Nobody would have ever needed to write see other file.”
“What about the wall?”
“That might be my favorite one, because your industry already thinks this way and just doesn’t realize it.” Jesse drew two clusters of circles with a wide gap between them. "You define the two sides of an ethical wall as disjoint. Formally disjoint. The dictionary states that no person can ever be a member of both of these sets at the same time. Once that's written down, a document getting routed from one side over to the other stops being a permissions question. It becomes a statement that cannot be true. Like a number that is both odd and even at the same time. Right now, your walls hold because of access controls, plus everybody behaving themselves. This writes the wall down as arithmetic."
Cooper looked at the collection of circles and arrows across the board and it made him think of his eleven thousand rows.
“The professor called it ‘data as graphs,’ he said.
“That’s a good phrase. The things, these lines between the things, and a book that tells you which of the lines are legal.”
Two entries
“Okay,” Cooper said. “Now on to the third folder.”
Jesse had clearly been waiting all morning for this part.
“You mentioned in your email that you thought you knew the answer,” he said. “So go ahead. Give me your guess.”
“My guess is that when the harness refused the supply agreement, it had to be checking that document against something. Some kind of a definition of what an NDA has to contain. My guess is that somebody sat down and wrote that out by hand. And by somebody, I mean you and Leo.”
"It was mostly Leo, actually, and that's the part I'm proudest about." Jesse held up two fingers to the camera. "Two entries. That is the entire dictionary living inside that harness. Entry number one covers what an NDA is. It has to contain confidentiality obligations. It has parties, it has a term, and it has the six positions the firm cares about. Entry number two roughly covers what an NDA is not, and that's how it recognized the shape of a supply agreement even while the filename was lying to it. We wrote both of them in one evening. So, when the third folder ran, the harness looked that document up in a two-word dictionary, the document wasn't in there, and everything that Maya loved about that refusal came from that one lookup."
Cooper put his pen down on the desk.
“So the most persuasive thing this firm has seen all summer,” he said, “was a dictionary with two entries in it.”
“And now you can see the problem, because I just watched you do the math. Two entries took us one evening. Your firm’s world is not two entries. It’s every practice area, every matter type, every document type, every role a person can hold, and every status that anything can possibly be in. That’s thousands of entries. They all have to agree with each other, and then somebody has to keep every one of them true while the firm keeps changing underneath them. Writing all of that down by hand is exactly what the professor’s generation attempted back in the eighties. That’s the graveyard he showed you at the end of the video. The rules themselves worked fine. Nobody could keep up with maintaining them, and that’s what buried the whole field.”
The economics
“So why is all of this coming back around now,” Cooper asked. “That’s the part of the video I keep going back to. He got an entire conference full of engineers to care about something that famously failed.”
“Because the economics finally flipped. And there’s actually a study that walks you through the flip.” Jesse pulled a paper up onto his screen and turned the camera toward the monitor. All Cooper could make out was a wall of small text. Jesse realized it too and swung the camera back around. “You’ll just have to trust me and read it later. A group of researchers took a set of natural language questions. Real business questions, the kind a manager would actually ask about their own company data. They pointed a model straight at the company’s database and had it answer them. No dictionary, no graph, just the model working against the raw tables. It got about sixteen percent of the questions right. Sixteen. Then they took the exact same data and rebuilt it as a knowledge graph with an ontology behind it. Asked the same questions again, and it scored fifty-four percent. Then came the part that I keep telling everybody about. They set it up so that the ontology checked every query the model wrote before that query was allowed to run. If the query was fixable, the system repaired it and ran it. If it wasn’t fixable, the query never ran at all. That version scored just over seventy-two percent. And there was one more detail buried in that study that I need you to hand over to Maya, word for word. The checked version learned how to say that it didn’t know. When it couldn’t get to an answer inside of the rules, it just said that, and it didn’t manufacture something confident for the sake of having an answer to give.”
Cooper wrote the three numbers down in a column, and underlined the word unknown twice.
“The model never changed,” Jesse said. “It was the same model through all three of those rounds. Everything that improved was in the material built around it. You’ve already heard me give that speech once this summer about the office. This is me giving the same speech about the book on the shelf. And this isn’t some lab curiosity either. There are entire companies out there now selling this exact layer. Palantir built a very large business on roughly this idea, an ontology of the customer’s world with the model working down inside of it.”
“What about the maintenance problem? The graveyard?”
“Still real. Dictionaries rot. Your firm’s dictionary rotted, and that’s what your eleven entries actually are.” Jesse tapped the printed list again. “The one thing that’s different this time around is that the dictionary finally has some help. The agents are out there hitting edge cases all day long, and it turns out they’re very good at noticing when a definition is missing or wrong. People are building systems now where the agent proposes the update and a human approves it. I’m not going to promise you that scales yet. What I am telling you is that the librarians of this thing may finally end up with a staff.”
Cooper let the librarian comment go by without a response, and privately enjoyed it.
The name
The math was what finally made Cooper ask his question out loud. He looked down at thousands of entries written in his notebook with a bracket around it, and asked whether the firm was seriously going to have to write its own dictionary of the entire legal world, from scratch, by hand.
“That’s the question I’ve been waiting on,” Jesse said. “And the answer is no. You don’t. The web figured this out twenty years ago and built shared vocabularies, so that every website didn’t have to invent its own from nothing. The industries next door to you have theirs. And legal, very quietly, while everybody was busy watching the agent demos, went out and built one too. It’s real, it’s open, and the definitions are public. You could read them this afternoon if you wanted to. A standards group has been working on it for years. The areas of law, the actual services lawyers perform, the industries the clients are in, all of it, with definitions attached, in a format a machine can consume. Your intake form has been trying to become this thing its whole life.”
Cooper had his pen ready. “What’s it called?”
“SALI. The standard itself is called the Legal Matter Specification Standard, which is a mouthful, so mostly everybody just says SALI.” Jesse spelled it out for him. “And now I need a favor from you, and I’m serious about this one. You are going to want to go spend tonight reading through the entire thing, because that’s who you are. Don’t do it. Bring Maya and Leo in on Thursday and let me walk all of you through it together, live and on the screen. I have never had the chance to show a lawyer their own intake form inside of a public standard before, and I’ve been looking forward to Maya’s face for two days now.”
Cooper agreed to the favor, and knew immediately that he was only going to half keep it.
There was one more thing to do, and he did it before he closed out of the call, because this summer had taught him the right order of operations. He messaged Leo directly. He wanted the NDA harness up on the screen Thursday as well, but modified, so that its clause checks would run against a real vocabulary instead of the two entries. And he wanted Leo to make that change himself before Thursday, without seeing a standard that Cooper hadn’t even seen yet either. Leo’s reply came back in under a minute, the way Leo’s replies do. It said he was already halfway through the docs. Then a second message came in right behind it: wait, there’s a public viewer?
Cooper closed the laptop and opened his notebook to a clean page. He drew the office one more time from memory. The boxes, the genius, and then the new shelf by the door with the book on it. Under the drawing he wrote out the day’s line.
Jesse added one new thing to the genius’s office today. A dictionary, on a shelf by the door. Every contract I have ever read starts with its defined terms. It took me until this morning to ask where the firm keeps ours.
Then beneath that, in smaller writing:
The most convincing machine this firm has ever seen was working off of a dictionary with two entries in it. On Thursday we find out what it reads next.