Hasil untuk "Information theory"

A. Graesser, Murray Singer, T. Trabasso

Paul Meara, D. Sperber, Deirdre Wilson

C. Pollard, I. Sag

K. Chadan, P. Sabatier

D. Teece

R. Nelson, E. Phelps

S. Chaiken, Y. Trope

B. Nalebuff, J. Stiglitz

H. Simon

M. Lv ] ~ ~ A, Fred P Sloan, M. Lynne et al.

F. Attneave

X. Vives

T. Wilmshurst, Geoffrey Frost

A. Achille, Stefano Soatto

Using established principles from Information Theory and Statistics, we show that in a deep neural network invariance to nuisance factors is equivalent to information minimality of the learned representation, and that stacking layers and injecting noise during training naturally bias the network towards learning invariant representations. We then show that, in order to avoid memorization, we need to limit the quantity of information stored in the weights, which leads to a novel usage of the Information Bottleneck Lagrangian on the weights as a learning criterion. This also has an alternative interpretation as minimizing a PAC-Bayesian bound on the test error. Finally, we exploit a duality between weights and activations induced by the architecture, to show that the information in the weights bounds the minimality and Total Correlation of the layers, therefore showing that regularizing the weights explicitly or implicitly, using SGD, not only helps avoid overfitting, but also fosters invariance and disentangling of the learned representation. The theory also enables predicting sharp phase transitions between underfitting and overfitting random labels at precise information values, and sheds light on the relation between the geometry of the loss function, in particular so-called “flat minima,” and generalization.

P. Kirschner

Huigang Liang, Yajiong Xue

John Ndung'u GACHENGA, Dennis Kamau MUTHONI, Wilson Kipkemboi METTO

The study examined the moderating effect of firm size on the relationship between credit management practices and the financial sustainability of DT-SACCOs in Kenya. The study was grounded in information asymmetry theory, utilising a positivist paradigm and an exploratory research design. The target population consisted of 176 finance managers from 176 DT-SACCOs, providing a robust framework for analysis. The sample size was obtained using Yamane's formula, which resulted in 122 respondents, with a high response rate of 98 per cent for the structured questionnaires administered. Data was analysed using descriptive and inferential statistics. The inferential statistics revealed a strong positive association between credit management practices and financial sustainability, with p-values of 0.013. Notably, the Nagelkerke R-squared change demonstrated that firm size moderates the connection between credit management practices and financial sustainability. The study recommends enhancing financial sustainability through credit information sharing and establishing a deposit guarantee fund to protect members' funds in the event of license revocation or closure.

James P. Crutchfield, Alexandra Jurgens

We develop information theory for the temporal behavior of memoryful agents moving through complex -- structured, stochastic -- environments. We introduce and explore information processes -- stochastic processes produced by cognitive agents in real-time as they interact with and interpret incoming stimuli. We provide basic results on the ergodicity and semantics of the resulting time series of Shannon information measures that monitor an agent's adapting view of uncertainty and structural correlation in its environment.

Zhixiong Chen, Jiawei Yang, Zhenyu Zhou

In response to the requirements for large-scale device access and ultra-reliable and low-latency communication in the power internet of things, unmanned aerial vehicle-assisted multi-access edge computing can be used to realize flexible access to power services and update large amounts of information in a timely manner. By considering factors such as machine communication traffic, MAC competition access, and information freshness, this paper develops a cross-layer computing framework in which the peak Age of Information (AoI) provides a statistical delay boundary in the finite blocklength regime. We also propose a deep machine learning-based multi-access edge computing offloading algorithm. First, a traffic arrival model is established in which the time interval follows the Beta distribution, and then a business service model is proposed based on the carrier sense multiple access with collision avoidance algorithm. The peak AoI boundary performance of multiple access is evaluated according to stochastic network calculus theory. Finally, an unmanned aerial vehicle-assisted multi-level offloading model with cache is designed, in which the peak AoI violation probability and energy consumption provide the optimization goals. The optimal offloading strategy is obtained using deep reinforcement learning. Compared with baseline schemes based on non-cooperative game theory with stochastic learning automata and random edge unloading, the proposed algorithm improves the overall performance by approximately 3.52 % and 20.73 %, respectively, and provides superior deterministic offloading performance by using the peak AoI boundary.

Ane Rahbek Vierø, Michael Szell

Spatial data science is an emerging field building on geographic information science, geography, and data science. Here we first discuss the definition and history of the field, arguing that it indeed warrants a new label. Then, we present the design of our course Geospatial Data Science at IT University of Copenhagen and discuss the importance of teaching not just spatial data science tools but also spatial and critical thinking. We conclude with a perspective on the potential future for spatial data science, arguing that qualitative theory and methods will continue to play an important role despite new GeoAI-related advances.