| 1. |
|
|
| 2. |
- Hintze, Arend, Professor, et al.
(författare)
-
Detecting Information Relays in Deep Neural Networks
- 2023
-
Ingår i: Entropy. - : MDPI AG. - 1099-4300. ; 25:3
-
Tidskriftsartikel (refereegranskat)abstract
- Deep learning of artificial neural networks (ANNs) is creating highly functional processes that are, unfortunately, nearly as hard to interpret as their biological counterparts. Identification of functional modules in natural brains plays an important role in cognitive and neuroscience alike, and can be carried out using a wide range of technologies such as fMRI, EEG/ERP, MEG, or calcium imaging. However, we do not have such robust methods at our disposal when it comes to understanding functional modules in artificial neural networks. Ideally, understanding which parts of an artificial neural network perform what function might help us to address a number of vexing problems in ANN research, such as catastrophic forgetting and overfitting. Furthermore, revealing a network's modularity could improve our trust in them by making these black boxes more transparent. Here, we introduce a new information-theoretic concept that proves useful in understanding and analyzing a network's functional modularity: the relay information IR. The relay information measures how much information groups of neurons that participate in a particular function (modules) relay from inputs to outputs. Combined with a greedy search algorithm, relay information can be used to identify computational modules in neural networks. We also show that the functionality of modules correlates with the amount of relay information they carry.
|
|
| 3. |
- Hintze, Arend, Professor, et al.
(författare)
-
Neuroevolution gives rise to more focused information transfer compared to backpropagation in recurrent neural networks
- 2022
-
Ingår i: Neural Computing & Applications. - : Springer Science and Business Media LLC. - 0941-0643 .- 1433-3058.
-
Tidskriftsartikel (refereegranskat)abstract
- Artificial neural networks (ANNs) are one of the most promising tools in the quest to develop general artificial intelligence. Their design was inspired by how neurons in natural brains connect and process, the only other substrate to harbor intelligence. Compared to biological brains that are sparsely connected and that form sparsely distributed representations, ANNs instead process information by connecting all nodes of one layer to all nodes of the next. In addition, modern ANNs are trained with backpropagation, while their natural counterparts have been optimized by natural evolution over eons. We study whether the training method influences how information propagates through the brain by measuring the transfer entropy, that is, the information that is transferred from one group of neurons to another. We find that while the distribution of connection weights in optimized networks is largely unaffected by the training method, neuroevolution leads to networks in which information transfer is significantly more focused on small groups of neurons (compared to those trained by backpropagation) while also being more robust to perturbations of the weights. We conclude that the specific attributes of a training method (local vs. global) can significantly affect how information is processed and relayed through the brain, even when the overall performance is similar.
|
|
| 4. |
- Lehman, Joel, et al.
(författare)
-
The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities
- 2020
-
Ingår i: Artificial Life. - : MIT Press - Journals. - 1530-9185 .- 1064-5462. ; 26:2, s. 274-306
-
Tidskriftsartikel (refereegranskat)abstract
- Evolution provides a creative fount of complex and subtle adaptations that often surprise the scientists who discover them. However, the creativity of evolution is not limited to the natural world: Artificial organisms evolving in computational environments have also elicited surprise and wonder from the researchers studying them. The process of evolution is an algorithmic process that transcends the substrate in which it occurs. Indeed, many researchers in the field of digital evolution can provide examples of how their evolving algorithms and organisms have creatively subverted their expectations or intentions, exposed unrecognized bugs in their code, produced unexpectedly adaptations, or engaged in behaviors and outcomes, uncannily convergent with ones found in nature. Such stories routinely reveal surprise and creativity by evolution in these digital worlds, but they rarely fit into the standard scientific narrative. Instead they are often treated as mere obstacles to be overcome, rather than results that warrant study in their own right. Bugs are fixed, experiments are refocused, and one-off surprises are collapsed into a single data point. The stories themselves are traded among researchers through oral tradition, but that mode of information transmission is inefficient and prone to error and outright loss. Moreover, the fact that these stories tend to be shared only among practitioners means that many natural scientists do not realize how interesting and lifelike digital organisms are and how natural their evolution can be. To our knowledge, no collection of such anecdotes has been published before. This article is the crowd-sourced product of researchers in the fields of artificial life and evolutionary computation who have provided first-hand accounts of such cases. It thus serves as a written, fact-checked collection of scientifically important and even entertaining stories. In doing so we also present here substantial evidence that the existence and importance of evolutionary surprises extends beyond the natural world, and may indeed be a universal property of all complex evolving systems.
|
|
| 5. |
|
|
