HTM Overview
What is HTM?
Hierarchical Temporal Memory (HTM) is a technology that models the neocortex. The neocortex occupies about 75% of the volume of a human brain and is responsible for all high level perception, including vision, hearing, language, and touch. Since HTM was derived from biology, it is suitable for tasks that are easy for people but hard for computers - for example, recognizing objects, making predictions, understanding speech, and discovering patterns in complex data. We believe the principles by which HTM technology operates will form the basis of machine intelligence.
HTM is a memory system that learns about its world by being exposed to sensory data over time and abstracting high-level concepts from the data. Abstraction allows HTM networks to generalize, providing flexibility and efficiency relative to the rigid rules processed by conventional computer programs. For example, patterns can be learned and recognized in the presence of incomplete or ambiguous data. By combining memories of pattern sequences with current input, an HTM network can predict what is likely to happen next. This prediction, in turn, can be used for a range of cognitive activity from imagining potential outcomes to detecting fraud.
The biological basis for HTM
The neocortex has a hierarchical structure. Each level, or region, in the hierarchy consists of several layers of densely packed cells. As sensory information arrives at the neocortex it ascends through the levels of the hierarchy. The cells in these regions become active to increasingly abstract and invariant aspects of the input. Whereas cells at the lowest level of the hierarchy respond optimally for simple features of the input, cells near the top of the hierarchy respond to high-level concepts such as faces, words, and actions.
Surprisingly, the regions in the neocortex have nearly identical structure regardless where in the hierarchy they reside and what type of sensory information they process. Visual regions look like auditory regions which look like language regions. This led to the discovery thirty years ago that the neocortex uses common mechanisms throughout. By understand these mechanisms we can emulate them and apply them to diverse problems of learning, inference, and prediction.
Hierarchical Temporal Memory models the hierarchy, the regions, and the cellular layers of the neocortex. HTM is not just a simulation, it is based on a theoretical understanding of how hierarchies work, and on exactly what neurons are learning when arranged in layers. Although the theory underlying HTM does not capture the full complexity of what the neocortex does, it is sufficiently powerful to solve difficult and commercially valuable problems.
Using HTM
An HTM network is currently implemented in software that runs on standard personal computers. The designer of an HTM network specifies the size and structure of the hierarchy and then trains the hierarchy by exposing it to sensory data. Sensory data might be similar to a human sense, such as visual data from a camera, but more often sensory data will not have a direct human counterpart. For example it might come from an array of special purpose sensors in a building or machine, or it could be historical data from a bank or business. What is important is to have lots of data to train on and the data has an element of time in it. An HTM network, and the brain, need to see patterns flowing over time in order to learn effectively.
At the heart of an HTM network are the learning algorithms that operate at each level of the hierarchy. They correspond to a layer of cells in the real neocortex, hence they are referred to as the HTM cortical learning algorithms. The vast majority of what is learned by these algorithms is how patterns follow each other in time. This knowledge is used to group together patterns and to make predictions of what will likely happen next. The HTM cortical learning algorithms combined in a hierarchy create a learning system that is both efficient and powerful. It can discover the underlying causes in a data stream, infer from novel noisy data, and make predictions.
Collecting and preparing data for training is a large part of creating an effective HTM network. First you have to determine if your problem is suitable for an HTM solution. The most important part is whether the data contains temporal correlations. Next you have to decide how to represent the data to the HTM network. This is roughly equivalent to what a sensory organ such as the retina or cochlea does. And finally you have to create training and testing regimes. Once you have accomplished these tasks the HTM network can be deployed and used with novel data.
