Hasil untuk "Information theory"

S. Vrieze

M. Bloch, J. Barros

Andrew M. Saxe, Yamini Bansal, Joel Dapello et al.

The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior. In this work, we study the information bottleneck (IB) theory of deep learning, which makes three specific claims: first, that deep networks undergo two distinct phases consisting of an initial fitting phase and a subsequent compression phase; second, that the compression phase is causally related to the excellent generalization performance of deep networks; and third, that the compression phase occurs due to the diffusion-like behavior of stochastic gradient descent. Here we show that none of these claims hold true in the general case, and instead reflect assumptions made to compute a finite mutual information metric in deterministic networks. When computed using simple binning, we demonstrate through a combination of analytical results and simulation that the information plane trajectory observed in prior work is predominantly a function of the neural nonlinearity employed: double-sided saturating nonlinearities like yield a compression phase as neural activations enter the saturation regime, but linear activation functions and single-sided saturating nonlinearities like the widely used ReLU in fact do not. Moreover, we find that there is no evident causal connection between compression and generalization: networks that do not compress are still capable of generalization, and vice versa. Next, we show that the compression phase, when it exists, does not arise from stochasticity in training by demonstrating that we can replicate the IB findings using full batch gradient descent rather than stochastic gradient descent. Finally, we show that when an input domain consists of a subset of task-relevant and task-irrelevant information, hidden representations do compress the task-irrelevant information, although the overall information about the input may monotonically increase with training time, and that this compression happens concurrently with the fitting process rather than during a subsequent compression period.

L. Zadeh

R. Ahlswede, I. Csiszár

H. Theil

P. Smolensky

M. Lynne, Markus And, D. Robey

D. Leidner, Timothy R. Kayworth

M. Wilde

Finally, here is a modern, self-contained text on quantum information theory suitable for graduate-level courses. Developing the subject 'from the ground up' it covers classical results as well as major advances of the past decade. Beginning with an extensive overview of classical information theory suitable for the non-expert, the author then turns his attention to quantum mechanics for quantum information theory, and the important protocols of teleportation, super-dense coding and entanglement distribution. He develops all of the tools necessary for understanding important results in quantum information theory, including capacity theorems for classical, entanglement-assisted, private and quantum communication. The book also covers important recent developments such as superadditivity of private, coherent and Holevo information, and the superactivation of quantum capacity. This book will be warmly welcomed by the upcoming generation of quantum information theorists and the already established community of classical information theorists.

R. Feenstra, Zhiyuan Li, Miaojie Yu

Pengfei Zhang, Tianrui Li, Guoqiang Wang et al.

Abstract Multi-Source Information Fusion (MSIF) is a comprehensive and interdisciplinary subject, and is referred to as, multi-sensor information fusion which was originated in the 1970s. Nowadays, the types and updates of data are becoming more multifarious and frequent, which bring new challenges for information fusion to deal with the multi-source data. Consequently, the construction of MSIF models suitable for different scenarios and the application of different fusion technologies are the core problems that need to be solved urgently. Rough set theory (RST) provides a computing paradigm for uncertain data modeling and reasoning, especially for classification issues with noisy, inaccurate or incomplete data. Furthermore, due to the rapid development of MSIF in recent years, the methodologies of learning under RST are becoming increasingly mature and systematic, unveiling a framework which has not been mentioned in the literature. In order to better clarify the approaches and application of MSIF in RST research community, this paper reviews the existing models and technologies from the perspectives of MSIF model (i.e., homogeneous and heterogeneous MSIF model), multi-view rough sets information fusion model (i.e., multi-granulation, multi-scale and multi-view decisions information fusion models), parallel computing information fusion model, incremental learning fusion technology and cluster ensembles fusion technology. Finally, RST based MSIF related research directions and challenges are also covered and discussed. By providing state-of-the-art understanding in specialized literature, this survey will directly help researchers understand the research developments of MSIF under RST.

Yilun Xu, Shengjia Zhao, Jiaming Song et al.

We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of the observer. The resulting \emph{predictive $\mathcal{V}$-information} encompasses mutual information and other notions of informativeness such as the coefficient of determination. Unlike Shannon's mutual information and in violation of the data processing inequality, $\mathcal{V}$-information can be created through computation. This is consistent with deep neural networks extracting hierarchies of progressively more informative features in representation learning. Additionally, we show that by incorporating computational constraints, $\mathcal{V}$-information can be reliably estimated from data even in high dimensions with PAC-style guarantees. Empirically, we demonstrate predictive $\mathcal{V}$-information is more effective than mutual information for structure learning and fair representation learning.

Abdul Galib, Nurwahyuni Syahrir, Hasnidar Hasnidar

Main Purpose -  This study aims to reveal the role of awareness of sustainable practices, culture, and perceived behavioral control in improving tax compliance as an effort to maintain sustainable business practices for MSMEs in West Sulawesi by internalizing the Pappasang Kalindaqdaq Mandar. Method -  The research method used in this study is a mixed method with a concurrent model to analyze quantitatively and qualitatively simultaneously. Main Findings -  The results confirmed the theory of planned behavior, whereby awareness of sustainable practices and culture has a significant influence on increasing MSMEs' intention to behave in a compliant manner towards taxation, but perceived behavioral control did not have a significant influence. These findings indicate that aspects of awareness of sustainable practices and internalization of Pappasang Kalindaqdaq Mandar culture have a strong dominance in explaining MSME tax compliance in West Sulawesi. Theory and Practical Implications - The strong dominance of tax awareness and culture, but not accompanied by a significant influence on perceived behavioral control, requires further investigation. Further in-depth interviews are needed to obtain more in-depth information from tax authorities and MSMEs to uncover the actual role of control.  Novelty -  This research explores non-economic aspects from various perspectives such as awareness of sustainable practices (internal), culture (external), and perceived behavioral control (control belief) in improving tax compliance (external).

shafieh Ghodrati, Fatima Yawari, Hossein ghodrati

<p class="MsoNormal" style="margin-bottom: .0001pt; text-align: justify; line-height: normal; background: #F8F9FA;"><span style="font-family: 'Times New Roman','serif'; mso-ascii-theme-font: major-bidi; mso-fareast-font-family: 'Times New Roman'; mso-hansi-theme-font: major-bidi; mso-bidi-theme-font: major-bidi; color: #202124;">Introduction: The present research examines Afghan women in Dealing with the Transformation Arising from the Collapse of the System and the Re-establishment of the Taliban and its consequences.</span></p> <p class="MsoNormal" style="margin-bottom: .0001pt; text-align: justify; line-height: normal; background: #F8F9FA;"><span style="font-family: 'Times New Roman','serif'; mso-ascii-theme-font: major-bidi; mso-fareast-font-family: 'Times New Roman'; mso-hansi-theme-font: major-bidi; mso-bidi-theme-font: major-bidi; color: #202124;">Method: For this purpose, a qualitative approach has been used. The data was collected through grounded theory and targeted sampling and interviewed with 20 Afghan women. The tool for collecting information is a semi-structured interview and has been analyzed by coding (open, central, selective).</span></p> <p class="MsoNormal" style="margin-bottom: .0001pt; text-align: justify; line-height: normal; background: #F8F9FA;"><span style="font-family: 'Times New Roman','serif'; mso-ascii-theme-font: major-bidi; mso-fareast-font-family: 'Times New Roman'; mso-hansi-theme-font: major-bidi; mso-bidi-theme-font: major-bidi; color: #202124;">Findings: The text of the interview was in the form of 150 concepts and 15 categories was summarized and categorized based on strategies and consequences and the essence of them under the title of "Compulsory adaptation and opportunism in the field of re-vulnerability horror" was extracted. The results show that Afghan women considered the fall of the system and the second rule of the Taliban as sudden and unwanted and considered it the cause of rollback. These women state that the changes have created an abnormal situation that has affected their lives and has made them face challenges and limitations. These women took strategies that were specific to that conditions in order to maintain security and reduce possible damages in terms of cost-benefit calculation, available options, and rationality. According to those conditions, Afghan women took strange strategies such as separation of loved ones and Anxiety and new challenges, etc., are the consequences of the strategies that they used to face the changes.</span></p> <p class="MsoNormal" style="margin-bottom: .0001pt; text-align: justify; line-height: normal; background: #F8F9FA;"><span style="font-family: 'Times New Roman','serif'; mso-ascii-theme-font: major-bidi; mso-fareast-font-family: 'Times New Roman'; mso-hansi-theme-font: major-bidi; mso-bidi-theme-font: major-bidi; color: #202124;">&nbsp;</span></p>

Diah Sri Rejeki, Haria Saputry Wahyuni, Nor Zaina Zaharah Mohd Ariff

This study investigates the cross-cultural perceptions of Indonesian and Malaysian students regarding the virtual literacy tourism services offered by the National Library of the Republic of Indonesia. As part of ongoing digital transformation efforts in the literacy sector, these virtual tour services provide an alternative means of accessing information and promoting cultural heritage. The purpose of this research is to analyze perceptual differences between the two student groups using user experience theory, which encompasses cognitive, affective, and cultural dimensions. A quantitative comparative design was employed, involving a purposive sample of 100 participants, 50 students from Indonesia and 50 students from Malaysia. Data were collected through an online questionnaire using a 5-point Likert scale and analyzed through descriptive statistics and independent sample t-tests. The findings reveal that, overall, there are no significant differences between the two groups across most indicators. However, a notable distinction appears in the affective dimension, specifically the sense of presence, which is higher among Malaysian respondents. These results indicate that user experience is shaped by cultural context and highlight the potential of virtual library tours to foster cross-cultural engagement and understanding.

ABDULRASAQ MUSTAPHA

Kenya's exposure to climate risks and fiscal volatility has raised concerns about the pricing of its sovereign Eurobonds in global markets. This study investigates the impact of green finance announcements, ESG risk scores, and inflation on Kenya’s sovereign Eurobond yield spreads over U.S. Treasuries from 2015 to 2024. Employing a quantitative explanatory research design, the study analyzed secondary monthly data on yield spreads, macroeconomic indicators, and ESG metrics using multiple linear regression. Descriptive statistics and diagnostic tests confirmed data suitability, while correlation analysis revealed expected directional relationships. Findings show that green finance announcements significantly reduce Kenya’s sovereign risk premium, aligning with signaling theory that credible sustainability communication enhances investor confidence. ESG risk scores were also found to have a statistically significant negative effect on yield spreads, underscoring the importance of non-financial performance in sovereign debt pricing. Conversely, inflation had a significant positive effect, reflecting heightened risk aversion toward macroeconomic instability. The study concludes that climate and ESG signals now influence investor pricing behavior in African debt markets. It recommends that the Kenyan government institutionalize green finance disclosures, improve ESG reporting systems, and enforce effective inflation-targeting policies to reduce borrowing costs and enhance debt sustainability. The findings offer vital information for policymakers and investors in understanding the evolving dynamics of climate-adjusted sovereign risk.

Wanjala Gidraf Joseph, Irene Wanjiru Muriithi, Mundu Mustafa Muhamad et al.

The research explored the impact of advancing education opportunities in private universities on diversity and social inclusion in public universities in western Uganda. Institutional logic theory underpinned social inclusion and diversity in public universities, whereas marketization, neoliberalism, and institutional theories anchored opportunity expansion. The study adopted a cross-sectional design and quantitative approach. Seven hundred and forty-eight (748) lecturers constituted the target population, from which a sample of 302 responders was obtained. Information was collected through the Questionnaire. Data was analysed using Pearson r correlation coefficient at an alpha level of .05. Expanding opportunities and promoting social inclusion and diversity were found to be positively correlated (r (278) =.54, p =.030). The study concluded that social inclusion and diversity in public universities are positively correlated with the expansion of educational opportunities in private universities. The study recommended that, government implements policies that promote diversity and inclusion and prioritise funding for public universities.

Satyajit Dhadumia, M. Ashok Kumar

In information geometry, statistical models are considered as differentiable manifolds, where each probability distribution represents a unique point on the manifold. A Riemannian metric can be systematically obtained from a divergence function using Eguchi's theory (1992); the well-known Fisher-Rao metric is obtained from the Kullback-Leibler (KL) divergence. The geometric derivation of the classical Cramér-Rao Lower Bound (CRLB) by Amari and Nagaoka (2000) is based on this metric. In this paper, we study a Riemannian metric obtained by applying Eguchi's theory to the Basu-Harris-Hjort-Jones (BHHJ) divergence (1998) and derive a generalized Cramér-Rao bound using Amari-Nagaoka's approach. There are potential applications for this bound in robust estimation.

Т. А. Ivashchenko, Е. S. Dvoynikovaon.ru

In modern society knowledge of Creative Pedagogy ensures high potential of educational activities and achievement of a high level of education quality. The education system is faced with the task of qualitatively preparing a modern graduate for further education and work. Solving this problem requires the search and implementation of new teaching methods, which are a meaningful component of Creative Pedagogy. Scientific works do not fully present a comprehensive analysis of Creative Pedagogy. They primarily reveal only some aspects related to the creativity of a teacher and the principles of organizing a creative environment. The weak theoretical development of the issue under consideration allowed us to formulate the following research problem: what are the essential characteristics of Creative Pedagogy? The purpose of the article is to substantiate theoretical approaches to the definition of “Creative Pedagogy”. Solving the stated goal required the use of theoretical research methods (analysis of scientific literature, synthesis of existing knowledge on the problem under study, generalization). The empirical basis of the study is represented by analytical materials and information resources on the Internet.The results of the research include 1) the concept of “Creative Pedagogy” is considered as an innovative direction of the new education paradigm; 2) the potential of Creative Pedagogy in educational activities has been revealed; 3) the pedagogical conditions for the creative orientation of pedagogical activity have been determined; 4) the basic principles of creating a creative educational environment have been outlined.Key findings: The study extends pedagogical theory toward understanding the content of Creative Pedagogy; the presented basic principles can be used by teachers when organizing a creative educational environment, and the proposed creative teaching methods can be used to stimulate the cognitive activity of students.