| 2. |
- Karlholm, Jörgen
(author)
-
Associative Memories with Short--Range Higher Order Couplings
- 1993
-
In: Neural Networks. - 0893-6080 .- 1879-2782. ; 6:3, s. 409-421
-
Journal article (peer-reviewed)abstract
- A study of recurrent associative memories with exclusively short-range connections is presented. To increase the capacity, higher order couplings are used. We study capacity and pattern completion ability of networks consisting of units with binary (±1) output. Results show that perfect learning of random patterns is difficult for very short coupling ranges, and that the average expected capacities (allowing small errors) in these cases are much smaller than the theoretical maximum, 2 bits per coupling. However, it is also shown that by choosing ranges longer than certain limit sizes, depending on network size and order, we can come close to the theoretical capacity limit. We indicate that these limit sizes increase very slowly with net size. Thus, couplings to at least 28 and 36 neighbors suffice for second order networks with 400 and 90,000 units, respectively. From simulations it is found that even networks with coupling ranges below the limit size are able to complete input patterns with more than 10% errors. Especially remarkable is the ability to correct inputs with large local errors (part of the pattern is masked). We present a local learning algorithm for heteroassociation in recurrent networks without hidden units. The algorithm is used in a multinet system to improve pattern completion ability on correlated patterns.
|
|
