As humans, symbolism is the key to understanding the world around us, it’s how we interpret objects, ideas, and the relationships between and among them.
We are wholly dependent upon analogy, which is what makes our current computing technology extremely convoluted, complex, and at this point in time, archaic.
The growing popularity of artificial intelligence (AI) and the use cases we are already seeing with OpenAI's ChatGPT aren't necessarily the best applications that go beyond mere "hype" and stock inflation.
Under traditional computing, we don’t fully understand what these artificial neural networks (ANN) are doing or why they even work as well as they do. The utter lack of transparency also provides a major disadvantage in our understanding of how data is collected and analyzed to spit out the results we so desperately attach ourselves to that we come to label as “progress.”
Consider the following example of an ANN that is able to distinguish “circles” and “squares” from one another.
One way to achieve that distinction is the obvious – if one output layer indicates a circle, and the other indicates a square.
But what if you wanted the ANN to discern that particular shape’s “color” – is it “red” or “blue”?
Since “color” is an entirely separate data set, it requires additional output neurons to be able to account for that feature in the final output. In this case, there would need to be four output neurons – one each for the blue circle, blue square, red circle, and red square.
Now, what if we wanted a computation that also considered additional information, such as “size” or “position/location”?
More features mean more neurons that need to account for each possibility associated in defining that particular feature (or combination of features) with the “circle” and the “square”.
In other words, it becomes incredibly complex.
Bruno Olshausen, a neuroscientist at the University of California, Berkeley, recently spoke to this need for having a neuron for every possible combination of features.
“This can’t be how our brains perceive the natural world, with all its variations. You have to propose…a neuron for all combinations,” he said, further explaining that we in essence, would need “a purple Volkswagen detector” or something so obscure to account for every possible combination of information we are hoping to consider in any given experiment.
Enter ‘hyperdimensional computing’.
What Is ‘Hyperdimensional Computing’?
The heart of hyperdimensional computing is the algorithm’s ability to decipher specific pieces of information from complex images (think of metadata) and then represent that collective information as a single entity, known as a “hyperdimensional vector.”
Unlike traditional computing, hyperdimensional computing allows us to solve problems symbolically and in a sense, be able to efficiently and accurately “predict” the outcome of a particular problem based on the data contained in the hyperdimensional vector.
What Olshausen argues, among other colleagues, is that information in the brain is represented by the activity of a ton of neurons, making the perception of our fictitious “purple Volkswagen” impossible to be contained by a single neuron’s actions, but instead, through thousands of neurons that, collectively, come to comprise a purple Volkswagen.
With the same set of neurons acting differently, we could see an entirely different concept or result, such as a pink Cadillac.
The key, according to a recent discussion in WIRED, is that each piece of information, such as the idea of a car or its make, model, color, or all of them combined, is represented as a single entity – a hyperdimensional vector or hypervector.
A “vector” is just an ordered array of numbers – 1, 2, 3, etc – where a 3D vector consists of three numbers – the x, y, and z coordinates of an exact point in 3D space.
A “hypervector”, on the other hand, could be an array of thousands or hundreds of thousands of numbers that represent a point in that amount of dimensional space. For example, a hypervector that represents an array of 10,000 numbers represents a point in 10,000-dimensional space.
This level of abstraction affords us the flexibility and ability to evolve modern computing and harmonize it with emerging technologies, such as artificial intelligence (AI).
“This is the thing that I’ve been most excited about, practically in my entire career,” Olshausen said. To him and many others, hyperdimensional computing promises a new world in which computing is efficient and robust and machine-made decisions are entirely transparent.
Transforming ‘Metadata’ Into Hyperdimensional Algorithms to Generate Complex Results
The underlying algebra tells us why the system chose that particular answer, which cannot be said for traditional neural networks.
In developing hybrid systems in which these neural networks can map things out IRL to hypervectors, and then allow for hyperdimensional algebra to take over is the crux of how AI should be used to actually empower us to better understand the world around us.
“This is what we should expect of any AI system,” says Olshausen. “We should be able to understand it just like we understand an airplane or a television set.”
Going back to the example with “circles” and “squares” and applying it to high-dimension spaces, we need vectors to represent the variables of “shape” and “color” – but also, we need vectors to represent the values that can be assigned to the variables – “CIRCLE”, “SQUARE”, “BLUE”, and “RED.”
Most importantly, these vectors must be distinct enough to actually quantify these variables.
Now, let’s turn attention to Eric Weiss, a student of Olshausen, who in 2015, demonstrated one aspect of hyperdimensional computing’s unique abilities in how to best represent a complex image as a single hyperdimensional vector that contains information about ALL the objects in the image – colors, positions, sizes.
In other words, an extremely advanced representation of an image’s metadata.
“I practically fell out of my chair,” Olshausen said. “All of a sudden, the light bulb went on.”
At that moment, more teams began focusing their efforts on developing “hyperdimensional algorithms” to replicate the “simple” tasks that deep neural networks had already been engaged in two decades prior – such as classifying images.
Creating a ‘Hypervector’ For Each Image
For example, if you were to take an annotated data set that consists of images of handwritten digits, this hyperdimensional algorithm would analyze the specific features of each image, creating a “hypervector” for each image.
Creating a “Class” of Hypervectors for Each Digit
From there, the algorithm would add the hypervectors for all images of “zero” to create a hypervector for the “idea of zero,” and repeats that for all the digits, generating 10 “class” hypervectors – one for each digit.
Those stored classes of hypervectors are now measured and analyzed against the hypervector created for a new, unlabeled image for the purpose of the algorithm determining which digit most closely matches the new image (based on the predetermined class of hypervectors for each digit).
IBM Research Dives In
In March, Abbas Rahimi and two colleagues at IBM Research in Zurich used hyperdimensional computing with neural networks to solve a classic problem in abstract visual reasoning – something that has presented a significant challenge for typical ANNs, and even some humans.
The team first created a “dictionary” of hypervectors to represent the objects in each image, where each hypervector in the dictionary represented a specific object and some combination of its attributes.
From there, the team trained a neural network to examine an image to generate a bipolar hypervector – where a particular attribute or element can be a +1 or -1.
“You guide the neural network to a meaningful conceptual space,” Rahimi said.
The value here is that once the network has generated hypervectors for each of the context images, and for each candidate for the blank slot, another algorithm is used to analyze the hypervectors to create “probability distributions” for a number of objects in the image.
In other words, algebra is able to be used to predict the most likely candidate image to fill the vacant slot. And the team’s approach yielded a near 88 percent accuracy on one set of problems, where neural network-only solutions were less than 61 percent accurate.
We’re Still In Infancy
Despite its many advantages, hyperdimensional computing is still very much in its infancy and requires testing against real-world problems and at much bigger scales than what we’ve seen so far – for example, the need to efficiently search over 1 billion items or results and find a specific result.
Ultimately, this will come with time, but it does present the questions of where and how we apply and integrate the use of artificial intelligence.
Read about how a 40-minute church service, powered by AI, drew in over 300 attendees in Germany as a first-of-its-kind experiment.
Click here to view full gallery at Hypemoon
