Hammond is the chief scientist at Narrative Science, a Chicago startup dedicated to “automated narrative generation”—a platform that automatically data into easy-to-read narratives.
DL: First of all, tell me about Narrative Science.
Hammond: Our core functionality is that we have a system that can generate stories, and this may sound odd at first, but they are stories about the world, about your organization, about yourself, stories that are not about the data but are about the world. Now, in order to do that, the system actually happens to use data, structured data, in order to figure out what’s going on, because this data is hooked up to the world. So, if I am telling a story about the way a company is going, I look to the data about its earnings and the change and how it compares to other companies and its sector and all those things. But I’m not telling a story about the data. I’m telling a story about the world. I just happen to be using the data as a source. And, in the world of data, that is genuinely unique because if I give you a spreadsheet, I am giving you the data. If I give you a dashboard or a visualization or a graph, I’m actually still giving you the data in another form, and it might be easier to look at and you can figure out the story, but I’m not telling you about the world. I’m telling you about the data.
DL: Right. That is very different.
Hammond: It’s been a challenge for us to get people to understand that difference. We have no interest in telling you more about your data. We have an interest in telling you about your world.
DL: Right. That is very different.
Hammond: In fact, many people will ask about charts and graphs and stuff. It’s like yeah, if it’s part of the story, that’s fine. And often, people will say, “Oh, I like the numbers!” For me, that always cycles through a series of interactions having to do with, “What does it really mean for you to like the numbers?” and, “Once you’ve figured something out, how do you communicate it?”
DL: So you’re talking about the processing of the information into a narrative, obviously.
Hammond: In fact, I gave a talk relatively recently and there was a guy in the audience, who was like, “Hey, I’m an analyst, and I always love to see the spreadsheets and see the numbers and figure things out.” And I said, “That’s great. And by the way, when you say that, do you mean you or an intern”? And there was laughter, and he said, “Yes, that’s exactly right. OK, sometimes it’s an intern.” Of course it is.
DL: He wasn’t doing that.
Hammond: He wasn’t doing that. That’s using the bottom of your skill set. Looking at two numbers and deciding which one is bigger is not the uber skill that we were all looking for when we went to business school. That’s the core of what we do. We tell stories about the world.
Now, what is that story, and where can we tell those stories? The reality is we can tell the stories anywhere where data got gathered to monitor what is happening. And that’s what everyone has been doing. From businesses and sales force, logistics, education, and now all the Internet of Things, which is really all about capturing all of those things. And now there’s Fitbit, you listen to the CEO of Fitbit talk about how everyone loves the data. It’s fine to think this, but they don’t really want the data, all they really want to know is what’s going on inside them …
DL: Right. They want to understand it.
Hammond: … and none of them have the skills. But I think, in the human realm, not the corporate realm, but in the human realm, there are people who not only don’t have the skills to actually look at the data and understand what’s going on in their world, what’s happening to them, but they don’t have access to anyone else who has those skills. And if I’m a company, I can hire a data scientist who is really expensive, to explain what’s going on in my data, but if I’m a person I can’t do that, and more and more of that data is coming to us. And so, for me, our mission is … I always feel like I’m at the end of “The Grapes of Wrath.”
DL: Better than at the beginning …
Hammond: Whenever you find a spreadsheet that’s unintelligible, a dashboard that’s unintelligible, the data that is out there that is part of our world that people can’t interpret, Quill [Narrative Science’s platform] will be there.
It genuinely is the case that I intend to see an era in which we look at spreadsheets and databases in exactly the same way we currently look at computer punch cards. Computer punch cards were a mechanism for inputting data, and your output would be a stack of cards. So, nowadays we input data in tabular form, and we get something in tabular form back. Given the nature of this technology and where we can go, people will look at those spreadsheets with a sort of nostalgia.
People today are like, oh, yeah, I remember when my first program was using punch cards. And I want it to be people saying, “Oh yeah, I had to learn to how to use that spreadsheet.” I want that to go away.
DL: When I read about Narrative Science, it’s often about the idea that you’re replacing journalists. And that’s not what you’re talking about—although, ultimately, at some point, that may be the result.
Hammond: There are skills that journalists have that we are encroaching upon. There’s no way around it.
DL: That’s right.
Hammond: And there are a couple of special things that we will be able to do that simply can’t be done by human beings. My favorite example of this. … We always say we can provide content promoting even more. People are like, “What’s that?” I always take them back to Obama [while campaigning in 2008], when he came out, I think it had to do with the pipeline, and he said, “If everyone would just inflate their tires correctly, we could save 7 percent of fuel utilization in this country.” A lot of people made fun of him because they had their own agendas. And, unfortunately, most people would look at that and say, “I absolutely don’t know what that means to me. I don’t know what that means.”
He came back and said it another way: He said it would save the country $3 billion a year in fuel costs savings. And people, even though it was at least a big number that they knew about, they still said, “What does that mean to me?” And for me, I look at that and think, imagine if we had that story. And I also knew what car you drive, which is easy to know, how much you drive, where you live, gas mileage on that car, how much gas costs in your neighborhood. And that story could be: For you, that would mean a savings of $24 a month, or 24x12, or $288 in savings a year. And you read that, and you go, “Oh, I get that. I understand what that means for me.”
DL: What about privacy? What about the pushback on the fact that the stories you will tell about me are only as compelling as the depth of information you have about me? But that means I have to give all that up?
Hammond: For one thing, you’ve already given all of that up.
DL: But people don’t know that.
Hammond: Do you remember when the Web popped and the browsers, and people found out about cookies? And the tracking? And very quickly, people were saying, “Oh, when I go to a site, it actually knows me and remembers my preferences and it makes my life easier”? And they all said, “Oh, sure. Oh, that’s great.”
If you can get to a clear, clean, coherent, on-point and meaningful communication about what’s going on in the world, one that is embedded in your life … if I can take the data that is out there and make things better for you, you’ll be OK with that. The service I offer you will have to be worth any sense of loss of privacy. It has to be. It won’t work if it doesn’t. And that is what it comes down to: Can we provide you with a good enough service so that it works? I actually think that Narrative Science isn’t going to be at the forefront of gathering human data. But once the data is there, we’ll use them.
DL: That’s a different function.
Hammond: Yes, but we will be one of the drivers of making that data valuable to people.
DL: You just said you’re going to be providing clear, consistent information embedded in your life. That’s the value proposition of Narrative Science. That’s the value proposition of journalism.
Hammond: I know. Although we don’t work in pure media for the most part, we still do a chunk of work there. The ethos, and part of the mission of this company, a huge part of it, comes from or is driven by a sort of journalistic mission. Part of our mission now is that we want to inform people, we will make people smarter.
DL: Do you tell people that? Is it written down?
Hammond: That’s what our mission is. I say it all the time. Literally, we just finished the scope. It is literally part of our mission statement. We will make people smarter through the application of technology to generate stories about the world, based on data.
For me, there are components to making people smarter. Giving someone an answer doesn’t make them smarter. Giving someone the answer and the rationale behind that answer does. So if someone says, “How is the new Captain America movie going to do?” I say, “I think it’s going to make $120 million opening weekend.” The first thing out of anyone’s mouth if I say that will be, “Why do you think that?”
DL: And how do you respond?
Hammond: I’d say, “That’s because it’s part of the Marvel franchise, and here’s the history of that. Here’s the director’s history. Here’s what it’s competing against as it comes out. Here’s the time of year it’s coming out.” You know, I can go through that, and I can say that is my rationale. And someone walks away from that going, “Wow, not only do I know why I think that movie is going to do well, but I actually know now how to think about movies doing well. You taught me that.”
DL: Is that transparent in what you’re doing, or is that on the back side?
Hammond: It’s not necessarily explicit, but we do work for one of the financial ratings agencies, which is the explanation of one number. It’s an eight-page document that explains one number. And that’s its only the goal. I can now choose what I want to think about, but now I understand how to think about these problems.
DL: The things you’re talking about, there’s no end to it because complexity in the world never stops.
Hammond: I would say, I think the world is—and I don’t want you to take this the wrong way—the world is simpler than most people think. I think much simpler.
DL: How so?
Hammond: We have a push going on right now. It’s something we are referring to it as “narrative analytics.” It’s the notion that all the analysis we do will be in service of the narrative, and that we will have communication goals. Those communication goals will have information needs, and those information needs will be satisfied by a particular piece of analysis. That analysis will have data constraints.
In order to figure this out, I will need to know some things and have certain types of data. The simplest example would be, if I want to say something about the height difference between you and me, I will need to know what our height difference is. In order to know what the height differences are—and this is data, not just eyeballing you over—in order to know what the difference is, I need to run a calculation taking two numbers to see which is bigger and which is smaller. In order to do that calculation, I need to have a piece of data with you and with me that is our heights, our respective heights. Now, I have that, I can run that calculation and know those things and say what I need to say. If I don’t have any piece of that chain, I can’t. I just can’t. I can’t. And, in fact, for anything where I’m talking about the differences between two things, and this is the simplest case, the differences of two things, that chain is identical; it always looks exactly the same. Exactly the same.
Now, it could be that the metrics coming in, those two numbers coming in, can come from a whole bunch of different kinds of things, different kinds of methods. That’s fine. That is a different kind of calculation. That’s not narrative analytics, that’s a different calculation. The calculation for the narrative is fixed, and it has nothing to do with the machine. A human writer is not allowed, starting with data, to say something is bigger than another thing, unless they’ve gone through that process. Now, if I’m talking about how things change over time, suddenly I’m doing time-series analysis. If I’m talking about where things sit in population, I have to do cohort comparison.
DL: What you’re saying is there is a fixed number of narrative types in terms of stories and format, and that makes the world simpler than people believe.
Hammond: Yes. I wouldn’t know how fixed they are, but they’re a set. I can probably hit 90 percent of what I need to say using under 20 of these things. That’s my current data. And once you’ve got those, you’ve kind of got them. And that’s the beauty of this.
And so for us, we’re taking all the lessons learned from building these stories over the past few years and mapping it back into the system, and now our content architects are making decisions and looking at the source of those decisions. We are modeling the people who are our current users and bringing that modeling into the system.
DL: So there are variables you’re considering, and you’re identifying constants across those variables within and across populations.
Hammond: Yes. It’s interesting: Our content directors are coming out of analytics and mostly journalism. And right now, the way I just articulated that, they would say, yes, that’s right. Two years ago? As I was talking, people were gathering pitchforks and torches because they were like, no, every single client is different and every single content type is different, and everything we do is different. And I was always adamant, no, it’s not.
DL: There’s no business model in that kind of differentiation.
Hammond: And do you think that because you are talking to Aetna vs. Aon, they have different kinds of communication? They don’t. They both want to look at how is this thing doing. What can I do to make it be better?
DL: What I hear you saying—and it is still such a parallel to how we used to teach, or maybe are still teaching, journalism—that there are 10 questions you can ask any individual, and by the time you’ve asked all 10 questions, you know everything you need to know about the person. There are not 1,000 questions, even though everyone is different. But when you say that all the analytics and analysis are in service to the story, that’s journalism.
Hammond: Again, when you say driven by, then yes, that’s our model.
DL: Has that always been your model?
Hammond: No. But for me, personally, in terms of my research, the model is based on some sort of narrative arc. I’m starting with the fact that I know how to tell the story, and I know what the experience should look like. What I do, at every knife edge of that arc, is ask the question: “What belongs here?” That has been a huge driver for us. As we moved into this realm, where we did a lot of narrative construction with found objects and search, we moved into analytics. But the outline is the same. The outline says, this is what we need to know. So you have to go look for this, and the looking is now very calculated. It’s a different kind of “looking for” than a pure search. But you still need to answer these questions for me.
DL: So it’s about defining the base common denominators or variables across communities and companies.
Hammond: It’s defining that, and then making the world comfortable with it. Especially in the early days, I was always amazed when we would go in to talk to a client, and they would say, “Well, we know you can write these fantastic baseball stories and fantasy football stories, but our business is totally different than any other business …
DL: … you’ve ever seen…
Hammond: … in the history of time. So you don’t use resources to produce things? Oh, yes, we do. So you don’t have products and lines and people that sell things? Yes, we do have people that sell things. So you have an HR department, right? You have logistical and supply lines? You have a factory, right? Yes, of course. And yes, there are 10,000 variables that we care about. So, what kind of story do you want? Well, this kind of story. So, you need to know about eight different things at any given moment.
DL: Common denominators.
Hammond: But people, at all levels of what we do, righteously defend their uniqueness.
DL: Of course.
Hammond: And then, there’s the uber defense, in terms of looking at us: “Surely it can’t do what I do.”
DL: I was going to say that. Where does human intellect come into all of this?
Hammond: It’s a funny thing, even within this group, people are asking if we are doing AI [artificial intelligence]. And of course we are doing AI. We don’t do learning. I’ve been in and out of machine learning my entire career. My dissertation was on a system that would learn through errors. Literally it would build plans, run them, simulate the world, watch them fail, figure out why they failed and use that to do the fine-tune correction, remember the circumstance under which that failure occurred and then remember the correction. All those things. It was a very nice, it was an—I hesitate to use the word—organic workflow version of what learning is about. I believe it is the core of learning. So I love learning.
But we don’t do learning here. So people say that it can’t be AI. And for me, what is AI? AI is the modeling, mimicking or simulation of human behavior. Well, what kind of behavior? Well, not eating. That wouldn’t be interesting. What would be the set of characteristics we would think about being human that differentiates us? Planning. Understanding the synthesis of information.
We’re creating a system that does the one thing that differentiates us from beasts. Communication. So what we do, we have a system that actually models communication. It models the one thing that for most people is the differentiator of what it means to be human, and you are asking yourself if it is AI. Of course it is. What happens is they get in the weeds. They say it’s just Python code. Of course. The whole world is Python code.
DL: They aren’t thinking about it the way you are when they say that.
Hammond: It’s an ongoing conversation. And it’s hysterical.
DL: So why does it matter?
Hammond: It doesn’t really. It comes up in external conversations. I actually want our team to understand that they are working on—and I honestly believe this—they are working on a mission, forgetting about what the thing does and what impact it has. They are working on a very special mission.
I believe there are three core sciences: physics, understanding the laws of the physical world; biology, understanding of the laws of life; and cognitive science, understanding the laws of cognition and how it works. AI is the experimental branch of cognitive science, and that’s what we are doing. We have tools that other cognitive scientists do not have available to them, tools that let us say, “This is the way we think it works, and we will show you how.” And having them understand, for example, narrative analytics. I could give you a list of under 20 pieces of analysis and the corresponding communication goals associated with them, but we are going to patent it first. I can say, “If you want to communicate these things, you need to do these things.” And I will be damned if you can tell me how you are going avoid it—unless it’s direct observation.
DL: But you can’t scale direct observation. Are you going to do that?
Hammond: We are. We are pretty aggressive. I want the team to feel like, there’s the company and the company’s mission. I think the company’s mission is glorious. There’s the technology and the technology’s mission, which is even beyond this. No, we are going to be part of, and a major part of, understanding cognition, part of the science of cognition. Even in the language around the generation of language.
We have a couple of competitors who say they are focused on the natural language generation. But we are not focused on natural language generation. We do that, but that essentially means grammar. We do narrative generation. [That] means I first need to understand what’s happening in the world, and what’s important about what’s happening in the world. Then I need to know what’s interesting about what’s happening in the world, and then I need to know what you—you as an individual, as a person—need out of all of that. And I will use that insight to craft the narrative. Not just a sentence or whatever. I will use that to explain these things to you in a way you understand, in a way that’s meaningful and impactful specifically to you. That’s a different science than language generation.
Everyone I know in AI, who has built anything, a component of what they built, they built in natural language generation because you want to see what the system is doing. The best way to see what the machine is doing is to have it explain what it’s doing. Everyone does it, and it never shows up in anyone’s dissertation. That’s commodity work.
DL: I used to hate it when my students would say, “I will just type this up,” and I would say, “No, you are writing!” There’s a big difference between natural language generation and narrative generation.
Hammond: I think that is a really good analogy. I went to a workshop a few years back and gave a talk; I told them that I had no interest in learning for its own sake. I always think of learning as supporting the functionality of a broader system. Why else would you do it? What would it mean to learn something that you aren’t going to use? What would it mean to have a system designed around learning something that you aren’t going to use? In the process of doing, we are learning about the details of the data, we are learning the rules, or situations, that we have to respond to.
There was a guy who was a fine researcher who saw me after the workshop and said, “I guess, actually, I really believe on working on learning for learning’s sake.” OK, so you have to understand what you are doing, and it’s a fine thing to do: You’re working on the operative details of a system that may or may not be useful for someone else later on. You don’t have the constraint of utility, and you don’t actually have the data you would get from using it. So, go ahead and do it.
DL: Whatever makes you happy.
Hammond: So, I have people here who really want to work on learning. To what end for us? What would you learn? Of the things we use, what would you learn? There are actually answers to that. Usually, there’s not an answer. Well, I want to do predictive analytics. That’s fine. You do predictive analytics and data mining all you want. And you generate predictions until the cows come home. By the way, predictions, the ability to predict, is different than the ability to communicate. So you can generate your prediction, and give me the prediction and chain of reasoning with it, and we’ll make a story.
DL: So how does the industry define or talk about AI?
Hammond: Right now it does mean machine learning. Google has this huge press in deep learning. The guy who was the driving force in this is an old-school AI dude. The deep learning techniques are useful for doing one thing: recognizing something. You can’t infer things to save your life. That’s become the driver for it. Interestingly enough, people don’t see the Google car as a real focus of AI, even though it is a pure AI system. I think it’s more interesting as an AI system than anything they are doing on the learning side.
DL: That’s interesting.
Hammond: The way machine learning is right now, it’s, for almost everyone in the world, it’s incomprehensible what these things are doing. They build these massive networks and have chains of numbers moving through these networks, and what you do is, you say here’s an object, you lock down the inputs of that object, here’s the category it belongs in, then lock down the output and then see the adjustments on the nodes and the calculations that link those two things together, and then you do that a hundred million times.
DL: Right.
Hammond: And you have to be able to maintain consistency in those calculations. And that’s absolutely incomprehensible to anybody. And that’s why it’s now the locus of how people think about AI—because it’s incomprehensible. It’s magic.
DL: It has to be magic. Intelligence is what makes us human.
Hammond: I spent part of the day Monday in Austin with the IBM Watson team. The whole first part of the morning was them talking about cognitive computing and how it’s different from regular computing. The room was filled with AI people; Ph.D.s and computer science people who have been doing AI most of their lives. It was an older group. At one point, I really felt like saying, “You know that, this is what you tell the rest of the world, but for all of us, we know that what you are saying sounds more like marketing than science.” You talk about rule-based systems, that it isn’t a rule-based system. Of course it is. You are using syntactical analysis and pattern matching. You sum up over a large set of evidence that is absolutely algorithmic. Why would you say it’s radically different? It’s not. They want it to be radically different so people think it’s magic.
DL: What was the response?
Hammond: Essentially, the first part of the day was the marketing team more than anybody else. It was a very disappointing meeting for me, because I love this technology, I think it’s a genuinely brilliant technology, and now I’m watching the product team and marketing team force its fingers into the engineering team.
DL: And you can see it because you are far enough away from it to see it.
Hammond: There is a huge opportunity for us at Narrative Science because we can explain things. The big thing for Watson is it comes up with an answer. Our clients are telling us they don’t want an answer. Well, of course not: They want an explanation, they want a description. But Watson can’t generate a description.
DL: Sounds like the stereotype of an IT guy.
Hammond: Watson has become that. And it wasn’t just me. The whole room, we all saw this. We went through an interface and we were all like, “Well, where’s the answer?” Well, this is the answer: It’s the description. There was actually one guy who said that you’ve taken the one differentiating characteristic of Watson and removed it. And another guy said, “I can’t tell the difference between this and Google.”
It was the difference between answering, “Does Bob ride a bike?” And coming back with, “Absolutely he rides a bike, he has a bike helmet on, he’s got bike shorts on, he’s in really great shape and on rainy days his pants are covered with mud.” And that would be a good answer: an answer and evidence. What Watson does is, you ask does Bob ride a bike, and it says his pants are muddy. And literally, that’s what it’s doing. That doesn’t help me!