Originally posted 12-Jun-25
David Weinberger is Fellow, Senior Researcher, and member of the Fellows Advisory Board at Harvard’s Berkman Klein Center for Internet & Society. He writes and speaks about how AI is changing how we think about ourselves and our world — from morality to reality itself.
David was the co-director of the Harvard Library Innovation Lab, focusing on the future of libraries. He is a former writer-in-residence at Google AI groups and the editor of the “Strong Ideas” book series for MIT Press. He writes the “Perspective on Knowledge” column for KMWorld Magazine. He has been a philosophy professor, journalist, strategic marketing consultant to high tech companies, Internet entrepreneur, advisor to several presidential campaigns, and a Franklin Fellow at the US State Department.
David is an author and speaker about the effect of technology on ideas. He explored the effect of the Internet and AI on knowledge, on how we organize our ideas, and on the core concepts by which we think about our world. His work focuses on how technology — particularly the internet and machine learning — is changing our ideas. He has written books about the effect of machine learning’s complex models on business strategy and sense of meaning; order and organization in the digital age; the networking of knowledge; the Net’s effect on core concepts of self and place, and the shifts in relationships between businesses and their markets.
Here are definitions for five of David’s specialties:
- Artificial intelligence (AI): the capacity of a computer to perform operations analogous to learning and decision making in humans, as by an expert system.
- Knowledge: Plato’s definition: Knowledge is the set of beliefs that are true and that we are justified in believing. Knowledge results from a complex process that is social, goal-driven, contextual, and culturally-bound. We get to knowledge — especially ‘actionable’ knowledge — by having desires and curiosity, through plotting and play, by being wrong more often than right, by talking with others and forming social bonds, by applying methods and then backing away from them, by calculation and serendipity, by rationality and intuition, by institutional processes and social roles. Every single thing we know is part of a larger system of knowledge. If you know the water is boiling in your tea kettle because you hear its whistle, then you also know that water is a liquid, flames heat things, things can transmit heat to other things, and so on, until your entire knowledge framework has been drawn in.
- Machine learning (ML): giving computers the ability to learn without being explicitly programmed; a method of data analysis that automates analytical model building; using algorithms that iteratively learn from data to find hidden insights without being explicitly programmed where to look.
- Networked Knowledge: The value of a web of ideas comes from the differences among the participants in that web. If everybody’s saying the same thing, there’s negative value in networking them. Knowledge contains difference, rather than knowledge being that from which all disagreement has been driven, that which has been settled once and for all. Knowledge exists in networks that contain disagreement and difference.
- Understanding: seeing things connected into larger contexts. Understanding has degrees: you can understand a little about something or understand a lot. whereas knowledge is generally binary: you know, or you don’t know. Knowing without understanding is useless. Understanding is the goal of knowing. Understanding is an enhancement of knowledge: knowledge + connection.
David created the following content. I have curated it to represent his contributions to the field. For more about David, see Profiles in Knowledge.
Books by David Weinberger
AI Has a Revolutionary Ability to Parse Details. What Does That Mean for Business? with Michele Zanini
AI as an idea is making the world visible to us in its ever-changing specificity and details: an overwhelming riot of particulars, each related to all else in a landscape of creative chaos and emergence, finding patterns beyond our comprehension. In short, it’s a world in which everything is an exception.
Our economic growth will spring from this new ability to engage with the particulars — the specific differences among people, things, and situations (despite its new challenges to values such as fairness, transparency, and autonomy). Leaders who are aware of this change are best positioned to harness the possibilities.
As a philosophically-trained writer about technology, and as a researcher and advisor on innovative management practices, we’re already seeing these patterns across the business world. Here are four quick examples.
Strategy
Much of the value of high-level strategies comes from their being constants in a changing world. A strategy looks ahead a year, five years, even 10 years, and formulates a top-down vision that can be so grand only because it is so vague.
A long-term strategy is thus primed to miss the small signals — the particulars — that foretell a change in the landscape of emerging risks and opportunities. This includes the flapping of butterfly wings, most of which are irrelevant, but some of which are instances of the Butterfly Effect, ready to set off a cascade that can result in a tornado aimed at your business.
That’s why if businesses are going to heed thinkers such as Rita McGrath who implores them to look for “transient advantage,” they need to be hyper-alert to the weak signals that portend strong changes. That’s right in AI’s wheelhouse as manifested by companies such as Zignal Labs and Dataminr that generate billions of data points from their daily scans of thousands of sources, listening for the tornadoes foreshadowed in gently beating wings.
That’s some of what AI can do as a tool. As an idea, it wakes us up to the idea of those wing flaps and sends us looking for them. We see this already in companies that empower the expertise distributed throughout their ranks. An always-on, real-time conversational space open to all (in particular those at the edges of the organization) can enable people to talk about what’s interesting to them, such as an incipient trend and how it might affect the industry. This platform might be the first place to note small signals, collaboratively make sense of them, and potentially usher in a strategic shift.
This is changing the sense of where the knowledge resides in the organization, from a cluster of designated experts, to something akin to a metaphorical “neural network” of transient conversations, many (though not all) initiated and put together by AI that’s connecting a wide and diverse set of people across every layer of the hierarchy. Some of the most valuable insights might well emerge when the system uncovers strong agreement — or constructive disagreement — between people in functions that often don’t see eye to eye, such as sales and R&D, or finance and HR.
Talent management
In traditional talent management, people are recruited based on their credentials, prior experience, and other visible tokens of their skills. Once hired, the development of these employees is typically guided by generic competency models that are similar across companies within the same industry. The outcome: a one-size-fits-all view of people’s capabilities and interests.
AI can change that. It can uncover important skills that might not fit into the established checkboxes, such as handling exceptions well or being open to criticism. From this it can connect people to work together on a project, form internal interest groups, or strengthen and expand social bonds.
For example, the Guider learning and development platform matches mentees and mentors using AI that considers over 100 different factors based on users’ behaviors, rather than asking them to check off a handful of skills they’re interested in. It also is able to give detailed — specific! — reports that can guide the mentor-mentee interactions and relationship.
In the light of the idea of AI, even bigger changes are in store. Rather than thinking about our careers in terms of jobs with formal descriptions that draw on traditional categories, we might begin to think of ourselves as AI sees us: unique bundles of capabilities and interests, ready to participate in opportunities we could not foresee.
Leadership roles
A January 2023 paper by faculty at the Center for Strategic Leadership at the U.S Army War College predicts that AI will “directly influence the organizational structure of militaries.” For example, AI could “identify the soldier with the best situational awareness, put him or her in charge of the unit, and assign the rest of the team to supporting roles.”
If this is how authority should move in direct response to the particularities of a life-and-death situation, why not in less-fraught circumstances?
Indeed, perhaps AI will put together emergent hierarchies — dynamic and situation-relevant — that enable power to fluidly move to the people who can add unique value in a specific situation, irrespective of their formal credentials or position in the chain of command. Perhaps this is a way in which AI — rightly criticized for its tendency toward bias — could be used to empower people who don’t look or act according to the traditional norms of leadership.
Supply chain management
Through the 1990s, supply chains — however complex — still had to be simple enough to be understood and governed by human brains. Now AI as a tool is already beginning to transform container terminal operations, logistics workflow, inventory replenishment, demand forecasting, routing, and just about every other element of supply chain management by taking in vast amounts of data and re-coordinating logistics in real time.
For example, Haier, one of the the world’s largest appliance makers, uses an internet platform, COSMOPlat, to connect its millions of customers and tens of thousands of vendors around the globe. COSMOPlat interprets hundreds of thousands of customer inputs, which are quickly translated into design specs. These are then bid out to its vast supplier network. COSMOPlat can make unexpected connections, such as tapping a refrigerator insulation specialist to create a vibration-reducing material for washing machines. COSMOPlat even integrates its member companies’ networks to manage distribution and logistics, based on their strengths in each territory.
AI as an idea is thus letting us see supply chains as what they have always been: spontaneously coordinated loose networks of suppliers and partners. That affects how the whole business should be run in ways that go far beyond just which bots are managing its supply chain. Or, as Zhang Ruimin, the founder of Haier, has put it, the company should become a self-adaptive “rainforest”: complex, emergent, wildly interdependent, and in the service of the particulars of every situation.
Small Pieces Loosely Joined: A Unified Theory of the Web — Chapter 1: A New World
The distancelessness of the Web is just the most obvious of the disconnects between it and the real world. You could even classify them by using some big concepts from the real world, such as space, time, self, and knowledge:
Space: eBay is a Web space that occupies no space. Its “near” and “far” are determined by what’s linked to what, and the links are based not on contiguity but on human interest. The geography of the Web is as ephemeral as human interest: eBay pulled together a listings page for me based on my interest in “handmade quilts,” while simultaneously building pages for thousands of others who had other, unpredictable interests. Each of us looked across the space that is eBay and saw a vastly different landscape: mine of quilts, yours of Star Wars memorabilia, someone else’s of battery chargers.
Time: Earlier that morning, while waiting for my wife in our town center, I ducked into a store called Ten Thousand Villages that sells world crafts at a price fair to the artisans. For ten minutes I enjoyed being a Yuppie among the Chilean rainsticks and the Djembe drums from Burkina Faso. Then I saw my wife through the window, left the store, and closed the door behind me. Real-world time is a series of ticks to which schedules are tied. My time with eBay was different. As I investigated different auctions, placed a bid, and checked back every few hours to see if I’d been outbid, I felt as if I were returning to a story that was in progress, waiting for me whenever I wanted. I could break off in the middle when, for example, my son came home, and go back whenever I wanted. The Web is woven of hundreds of millions of threads like this one. And, in every case, we get to determine when and how long we will participate, based solely on what suits us. Time like that can spoil you for the real world.
Self: Buyers and sellers on eBay adopt a name by which they will be known. The eBay name of the woman selling the quilt I was interested in was “firewife30.” Firewife30 is an identity, a self, that lives only within eBay. If she’s a selfish bastard elsewhere but always acts with honor in eBay transactions, the “elsewhere” is not a part of Firewife30 that I can know about or should particularly care about. The real-world person behind firewife30 may even have other eBay identities. Perhaps she’s also SexyUndies who had 132 “sexy items” for sale at eBay while firewife30 was auctioning her quilt. Unlike real-world selves, these selves are intermittent and, most important, they are written. For all we know, firewife30 started out as firewife1 and it’s taken her this many drafts to get to a self that feels right to her.
Knowledge: I began my eBay search ignorant about quilts. By browsing among the 248 quilts for sale, I got an education. Yes, I could easily use the Web as a research tool, and at times during my quest I ran down some information — “sashing” is a border around each quilt block [vi] and a good quilter will get 10–12 stiches per inch [vii] — but I learned more and learned faster by listening to the voices of the quilters on eBay. I got trained in the features to look for, what the quilters consider to be boast-worthy, and what the other bidders thought was worth plunking their money down for. This was unsystematic and uncertified knowledge. But because it came wrapped in a human voice, it was richer and, in some ways, more reliable: the lively plurality of voices sometimes can and should outweigh the stentorian voice of experts.”
If a simple auction at eBay is based on new assumptions about space, time, self and knowledge, the Web is more than a place for disturbed teen-agers to try out roles and more than a good place to buy cheap quilts.
Everything Is Miscellaneous: The Power of the New Digital Disorder — Chapter 1: The New Order of Order
The Three Orders of Order
Bill Gates bought the Bettmann Archive, the most prestigious collection of historic photos in the United States, so he could bury it. In 2001 he hired nineteen trucks to move it from the melting summers of Manhattan to a cool limestone cave 220 feet underground in the middle of Pennsylvania. There, dehumidifiers the size of closets hold down the moisture level, and security guards patrol brightly lit streets carved out of stone. The site, run by the records management company Iron Mountain, is modern in every way, but if you walk far enough you’ll come to a dead end where a hole in the wall reveals an underground lake illuminated only by the light from the opening through which you’re looking.
The photographs in the Bettmann Archive are stored in a long narrow cavern whose arched walls of rough rock have been painted white but otherwise left unfinished. In rows of filing cabinets that stretch to the vanishing point are 11 million priceless photographs and negatives. They are arranged by the originating collections the Bettmann purchased over time. Within the collections the photos and negatives are generally ordered chronologically. The room is being slowly lowered to -4° Fahrenheit because Henry Wilhelm, a leading authority on film preservation, believes that at that temperature it will take five hundred years for the collection to deteriorate as much as it did in a single year when it was kept in Manhattan. Wilhelm was inspired to make film preservation his life’s work when, as a member of the Peace Corps deep in the Bolivian rain forest, he saw treasured family photographs in thatched houses. “They were deteriorating badly, and there was nothing I could do about it,” he says, still sounding frustrated decades later. The Bettmann facility he designed is the polar opposite of a Bolivian rain forest.
As you stand in the long cavern, you are in the midst of a huge first-order organization. In the first order of order, we organize things themselves — we put silverware into drawers, books on shelves, photos into albums. But when you go through the air lock that Wilhelm designed to connect the back chamber and the front one, you confront a prototypical example of the second order of order: a card catalog containing information about each of the eleven million objects in the back cavern. The catalog separates information about the first-order objects from the objects themselves, listing entries alphabetically by subject so that you can find, say, all the photos of soldiers across all of the archive’s collections. A code on this second-order object, the catalog card, points to the physical place where the first-order photo is stored in the back room. But quite a few of the Bettmann’s photographs are not listed in the card catalog. Some of the older collections arrived with catalogs entered in hand-written ledger books, one line per photo, listed in the order in which the photo was received. Finding a photo in one of those collections requires looking through the ledgers’ yellowing pages line by line, hoping to come across a description of the image you’re seeking. The ledgers are also a form of second-order classifications, just a much less efficient method than the card catalog.
The Bettmann’s second-order organization works, but it’s expensive to maintain, and retrieval times are sometimes measured in days. And there are limits inherent in the second order. Not all the information about the objects is recorded; a photograph of a Massachusetts soldier in the Civil War eating in a field, his rifle by his side, might be listed under “Civil War” and “soldier,” but probably not also under “Massachusetts,” “rifle,” “weapons,” “uniforms,” “dinner,” and “outdoors.” That means if you were to ask the Bettmann’s curators if they had a photo of a Civil War soldier eating outdoors, they would have to send someone into the stacks and stacks of filing cabinets to do a search through the photos themselves. Even if all that data were recorded, it would swell the size of the card catalog to the point of unusability: Searching through eleven million cards at one per second would take over four months of round-the-clock riffling.
All that work — a long line of trucks to move the archive, a hole dug deep into the earth, an ambient temperature growing so cold that you have to don arctic gear to enter the vault — and we still get so little use of those valuable — and expensive — assets. Indeed, a first- and second-order archive the size of the Bettmann literally cannot know everything it has.
The problems with the first two orders of order go back to the fact that they arrange atoms. There are laws about how atoms work. Things made of atoms tend to be unstable over time — paper yellows and disintegrates, negatives turn to soup — so we have to take measures to sway nature from its course. Atoms take up room, so collections of photos can get so large that we have to build card catalogs to remind us of where each photo is. And things made of atoms can be in only one spot at a time, so we have to decide whether a photo of a soldier eating should go into the Civil War folder or the Outdoor Meals folder.
But now we have bits. Content is digitized into bits, and the information about that content consists of bits as well. This is the third order of order and it’s hitting us — to use a completely inappropriate metaphor — like a ton of bricks. The third order removes the limitations we’ve assumed were inevitable in how we organize information.
For example, the digital order ignores the paper order’s requirement that labels be smaller than the things they’re labeling. An online “catalog card” listing a book for sale can contain — or link to — as much information as the seller wants, including user ratings, the author’s biography, and the full text of reviews. You can even let users search for a book by typing in any phrase they remember from it — “What’s the title of that detective novel where someone was described as having a face like a fist?” — which is like using the entire contents of the book as a label. That makes no sense when all that information has to be stored as atoms in the physical world but perfect sense when it’s available as bits and bytes in the digital realm.
You can see the third order in action by flying across the country from the Bettmann Archive to Seattle, where Corbis, the Bettmann’s parent company, has its headquarters. Corbis has charmingly renovated an old bank, knocking down walls to let in light and air, and even retaining the old circular vault door, symbolically open and inviting. Corbis holds over four million digital images, a collection smaller than the Bettmann’s but subject to the same issues of organization and control. Because the images as well as the information about the images are all fully electronic, Corbis organizes its photos without regard to the physical constraints that limit the curators back at the Bettmann. At Corbis, you can find a digital image of a Civil War soldier eating dinner by typing “soldier,” “Civil War,” and “meal” into the search engine, or by browsing a list of categories and subcategories. You can find what you need in seconds. If you don’t, you can be pretty sure it just isn’t in the Corbis collection.
Of course, it took Corbis many hours of preparation to reduce these search times to seconds. A team of nine full-time catalogers categorize each image Corbis acquires, anticipating how users are going to search. When a new image comes into the collection, one of the catalogers uses special software to browse the 61,000 “preferred terms” in the Corbis thesaurus for those that best describe the content of the image, typically attaching 10 to 30 terms to each one. The system incorporates about 33,000 synonyms (searches on “beach” turn up images labeled as “seaside”), as well as more than 500,000 permutations of names of people, movies, artworks, places, and more. That broadens the side of the barn so wide that if you misspell Katharine Hepburn’s name as Katherine or Catherine, you’ll still find all the images of the high-cheekboned screen legend in the Corbis collection. And if you’re looking for Muammar Gadhafi, at least seventeen different ways of spelling his name will get you what you want.
The differences between the Bettmann’s second-order organization and Corbis’s third-order method affect every aspect of their businesses.
The Bettmann is an attic that’s never been fully explored. It doesn’t know all the photos it owns; a ledger entry may be buried among thousands of others, and it may not describe the photo in a way that enables people who want it to find it. At Corbis, every image has been carefully cataloged and can be found by using the company’s search engine.
The Bettmann has to be parsimonious with its information: Create too many catalog cards for each photo and your card catalog bloats to unriffability. Like Staples, it bumps against the limits of the physical world. Corbis’s approach to information is sprawling and extravagant: The immateriality of bits encourages Corbis to put its images in every place where people might look for them.
Because the Bettmann collection’s second-order information — known as metadata because it’s information about information — is incomplete and spread across catalogs and ledgers, it’s accessible to only a handful of trained experts. Corbis, because its collection is third-order and thus fully digital and cataloged, is designed to be searched by any customer.
Finding photos among the Bettmann’s assets can be a slow, manual, expensive process. Corbis, on the other hand, not only can derive benefit from every image but knows so much about them that it can offload much of the job of searching for photos to its users.
Corbis’s digital images do not deteriorate with age and require far less physical maintenance. Overall, Corbis spends less per image, can make more per image, and is able to turn more of its images into productive assets.
The differences in the order of order even drive differences in the lighting. The Bettmann’s archives are brightly lit because their images are made of atoms that are visible only when light reflects off them. Corbis’s catalogers work in semidarkness because digital images on monitors are their own source of light.
But here’s the kicker: Like the iTunes Store, Corbis isn’t even a particularly good example of third-order organization. It’s doing what’s right for its business at the moment, but it’s still doing the basic second-order task of having professionals stick things into folders. Granted, the things and the folders are electronic, so Corbis can get more value from its assets at a lower cost. But there are other organizations that are able to move further down the third-order path. Corbis gives us only a taste of the revolution that’s under way. Just take a look at Flickr to see one way this is unrolling. With over 225 million photos already uploaded by users and almost a million added every day, Flickr’s collection dwarfs that of Corbis and Bettmann. Flickr has no professional catalogers. It relies solely on the labels users make up for themselves, without control or guidance. Yet it is remarkably easy to find photos at Flickr on almost any topic and to pull together collections of photos on themes that mix and match those topics at will. Want to find photos of dogs wearing red clown noses? A search at Flickr finds nineteen of them. Researching car-crash art? Flickr finds thirty-two photos that may help your studies.
The digital revolution in organization sweeps beyond how we find odd photos and beyond how we organize our businesses’ information assets. In fact, the third-order practices that make a company’s existing assets more profitable, increase customer loyalty, and seriously reduce costs are the Trojan horse of the information age. As we all get used to them, third-order practices undermine some of our most deeply ingrained ways of thinking about the world and our knowledge of it.
