We chose to build a product, not a MVP, Demo or PoC, but a product where people could see their ideas in context. Not as an adjunct visualised view, but as a primary interface, across as many languages possible, with real user agency to see, select, manipulate, viscerally interact with as much of all as we could get. Understanding our context is a shared endeavour, because none of us know know the whole truth, and never will. Large groups of people together with machine intelligence could make sense of our world.
This is being done across a wide technical space, from crawling, cleaning, calculating, connecting, to UX & UI via iron. We were initially constrained by limited hardware and a small team. Therefore we took a range of open tools, put them together in inappropriate ways, built on top of them to leverage them in ways the original creators never imagined. The rest we had to build, and (unfortunately) that ended up being a lot. Tens of thousands of articles became millions, tens of millions, hundreds and now billions. Suddenly we could see the web at scale, we could see all of medicine, research, patents, etc. and importantly how they all connect.
We forced ourselves to rewrite/rearchitect everything, again, and again, because a white page learns faster. Each time we solved problems in different ways. On the *fifth* rewrite we realised that we had created a remarkably elegant, scalable architecture that orchestrates many modalities. Many modalities augmented with human intelligence and vice versa will always trump singular approaches. There is no point solution in a complex plane. There will be no algorithm ex-machina- it is the many, many little things together that create the step change. Our role is to enable many “minds” and to put AI in its proper place- a collaborative one.
On the way we realised that we invented visual/visceral search. Not tools to narrow search, but vistas to broaden minds. Search that reveals more than what we searched for, not less. Interfaces with algorithmic agency where people can understand and influence the algorithms that manipulate their world(view). For the first time we can see the web, not as a list of documents, or a best guess answer, but the many right answers that make our world rich. See the overlap that connect us in interfaces that deepen understanding not amplify anger.
In the end, we realised that we do not have information overload, we had bad interfaces.
It took us just under 50 man years in the end. This is what we have built (so far).