Hasil untuk "General works"

Lin Wang, Kuk-Jin Yoon

Deep neural models, in recent years, have been successful in almost every field, even solving the most complex problem statements. However, these models are huge in size with millions (and even billions) of parameters, demanding heavy computation power and failing to be deployed on edge devices. Besides, the performance boost is highly dependent on redundant labeled data. To achieve faster speeds and to handle the problems caused by the lack of labeled data, knowledge distillation (KD) has been proposed to transfer information learned from one model to another. KD is often characterized by the so-called ‘Student-Teacher’ (S-T) learning framework and has been broadly applied in model compression and knowledge transfer. This paper is about KD and S-T learning, which are being actively studied in recent years. First, we aim to provide explanations of what KD is and how/why it works. Then, we provide a comprehensive survey on the recent progress of KD methods together with S-T frameworks typically used for vision tasks. In general, we investigate some fundamental questions that have been driving this research area and thoroughly generalize the research progress and technical details. Additionally, we systematically analyze the research status of KD in vision applications. Finally, we discuss the potentials and open challenges of existing methods and prospect the future directions of KD and S-T learning.

Quansheng Wu, ShengNan Zhang, Haifeng Song et al.

Abstract We present an open-source software package WannierTools , a tool for investigation of novel topological materials. This code works in the tight-binding framework, which can be generated by another software package Wannier90 Mostofi et al. (2008). It can help to classify the topological phase of a given materials by calculating the Wilson loop, and can get the surface state spectrum which is detected by angle resolved photoemission (ARPES) and in scanning tunneling microscopy (STM) experiments . It also identifies positions of Weyl/Dirac points and nodal line structures, calculates the Berry phase around a closed momentum loop and Berry curvature in a part of the Brillouin zone(BZ). Program summary Program title: WannierTools Program Files doi: http://dx.doi.org/10.17632/ygsmh4hyh6.1 Licensing provisions: GNU General Public Licence 3.0 Programming language: Fortran 90 External routines/libraries used: • BLAS ( http://www/netlib.org/blas ) • LAPACK ( http://www.netlib.org/lapack ) Nature of problem: Identifying the topological classifications of crystalline systems including insulators, semimetals, metals, and studying the electronic properties of the related slabs and ribbons systems. Solution method: Tight-binding method is a good approximation for solid system. Based on that, Wilson loop is used for topological phase classification. The iterative Green’s function is used for obtaining the surface state spectrum.

F. Danhier

T. Jungwirth, X. Marti, P. Wadley et al.

Antiferromagnetic materials are internally magnetic, but the direction of their ordered microscopic moments alternates between individual atomic sites. The resulting zero net magnetic moment makes magnetism in antiferromagnets externally invisible. This implies that information stored in antiferromagnetic moments would be invisible to common magnetic probes, insensitive to disturbing magnetic fields, and the antiferromagnetic element would not magnetically affect its neighbours, regardless of how densely the elements are arranged in the device. The intrinsic high frequencies of antiferromagnetic dynamics represent another property that makes antiferromagnets distinct from ferromagnets. Among the outstanding questions is how to manipulate and detect the magnetic state of an antiferromagnet efficiently. In this Review we focus on recent works that have addressed this question. The field of antiferromagnetic spintronics can also be viewed from the general perspectives of spin transport, magnetic textures and dynamics, and materials research. We briefly mention this broader context, together with an outlook of future research and applications of antiferromagnetic spintronics.

Liu Yang, Haiyan Yan, J. Lam

Chen Gao, Yu Zheng, Nian Li et al.

Recommender system is one of the most important information services on today’s Internet. Recently, graph neural networks have become the new state-of-the-art approach to recommender systems. In this survey, we conduct a comprehensive review of the literature on graph neural network-based recommender systems. We first introduce the background and the history of the development of both recommender systems and graph neural networks. For recommender systems, in general, there are four aspects for categorizing existing works: stage, scenario, objective, and application. For graph neural networks, the existing methods consist of two categories: spectral models and spatial ones. We then discuss the motivation of applying graph neural networks into recommender systems, mainly consisting of the high-order connectivity, the structural property of data and the enhanced supervision signal. We then systematically analyze the challenges in graph construction, embedding propagation/aggregation, model optimization, and computation efficiency. Afterward and primarily, we provide a comprehensive overview of a multitude of existing works of graph neural network-based recommender systems, following the taxonomy above. Finally, we raise discussions on the open problems and promising future directions in this area. We summarize the representative papers along with their code repositories in https://github.com/tsinghua-fib-lab/GNN-Recommender-Systems.

Qingsong Wang, Ping Ping, Xuejuan Zhao et al.

E. Everett, Achaleshwar Sahai, A. Sabharwal

Recent research results have demonstrated the feasibility of full-duplex wireless communication for short-range links. Although the focus of the previous works has been active cancellation of the self-interference signal, a majority of the overall self-interference suppression is often due to passive suppression, i.e., isolation of the transmit and receive antennas. We present a measurement-based study of the capabilities and limitations of three key mechanisms for passive self-interference suppression: directional isolation, absorptive shielding, and cross-polarization. The study demonstrates that more than 70 dB of passive suppression can be achieved in certain environments, but also establishes two results on the limitations of passive suppression: (1) environmental reflections limit the amount of passive suppression that can be achieved, and (2) passive suppression, in general, increases the frequency selectivity of the residual self-interference signal. These results suggest two design implications: (1) deployments of full-duplex infrastructure nodes should minimize near-antenna reflectors, and (2) active cancellation in concatenation with passive suppression should employ higher-order filters or per-subcarrier cancellation.

Miles Cranmer, Alvaro Sanchez-Gonzalez, P. Battaglia et al.

We develop a general approach to distill symbolic representations of a learned deep model by introducing strong inductive biases. We focus on Graph Neural Networks (GNNs). The technique works as follows: we first encourage sparse latent representations when we train a GNN in a supervised setting, then we apply symbolic regression to components of the learned model to extract explicit physical relations. We find the correct known equations, including force laws and Hamiltonians, can be extracted from the neural network. We then apply our method to a non-trivial cosmology example-a detailed dark matter simulation-and discover a new analytic formula which can predict the concentration of dark matter from the mass distribution of nearby cosmic structures. The symbolic expressions extracted from the GNN using our technique also generalized to out-of-distribution data better than the GNN itself. Our approach offers alternative directions for interpreting neural networks and discovering novel physical principles from the representations they learn.

D. Roodman, M. Nielsen, J. MacKinnon et al.

The wild bootstrap was originally developed for regression models with heteroskedasticity of unknown form. Over the past 30 years, it has been extended to models estimated by instrumental variables and maximum likelihood and to ones where the error terms are (perhaps multiway) clustered. Like bootstrap methods in general, the wild bootstrap is especially useful when conventional inference methods are unreliable because large-sample assumptions do not hold. For example, there may be few clusters, few treated clusters, or weak instruments. The package boottest can perform a wide variety of wild bootstrap tests, often at remarkable speed. It can also invert these tests to construct confidence sets. As a postestimation command, boottest works after linear estimation commands, including regress, cnsreg, ivregress, ivreg2, areg, and reghdfe, as well as many estimation commands based on maximum likelihood. Although it is designed to perform the wild cluster bootstrap, boottest can also perform the ordinary (nonclustered) version. Wrappers offer classical Wald, score/Lagrange multiplier, and Anderson–Rubin tests, optionally with (multiway) clustering. We review the main ideas of the wild cluster bootstrap, offer tips for use, explain why it is particularly amenable to computational optimization, state the syntax of boottest, artest, scoretest, and waldtest, and present several empirical examples.

Samith Abeywickrama, Rui Zhang, C. Yuen

Intelligent reflecting surface (IRS) that enables the control of the wireless propagation environment has been looked upon as a promising technology for boosting the spectrum and energy efficiency in future wireless communication systems. Prior works on IRS are mainly based on the ideal phase shift model assuming the full signal reflection by each of the elements regardless of its phase shift, which, however, is practically difficult to realize. In contrast, we propose in this paper a practical phase shift model that captures the phase-dependent amplitude variation in the element-wise reflection coefficient. Applying this new model to an IRS-aided wireless system, we formulate a problem to maximize its achievable rate by jointly optimizing the transmit beamforming and the IRS reflect beamforming. The formulated problem is non-convex and difficult to be optimally solved in general, for which we propose a low-complexity suboptimal solution based on the alternating optimization (AO) technique. Simulation results unveil a substantial performance gain achieved by the joint beamforming optimization based on the proposed phase shift model as compared to the conventional ideal model.

Xu Jin

In this note, we propose a novel adaptive fixed-time control scheme for output tracking problems of a class of multi-input multi-output (MIMO) nonlinear systems with asymmetric output constraint requirements, using a new universal barrier function. It is universal in the sense that the proposed scheme is a general one that also works for systems with symmetric constraints or without constraint requirements, without changing the control structure. Novel adaptive estimations and analysis are introduced to address system uncertainties in the fixed-time convergence settings. We show that under the proposed novel control scheme, each element in the system output tracking error vector of the MIMO nonlinear system can converge into small sets near zero with fixed-time convergence rate, while the asymmetric output constraint requirements on each element of the output tracking error are satisfied at all time. The proposed scheme can effectively deal with unmatched system uncertainties and uncertain gain functions. In the end, a simulation example on a two-degree-of-freedom robot manipulator is presented to demonstrate the efficacy of the proposed scheme.

James Martens

Natural gradient descent is an optimization method traditionally motivated from the perspective of information geometry, and works well for many applications as an alternative to stochastic gradient descent. In this paper we critically analyze this method and its properties, and show how it can be viewed as a type of approximate 2nd-order optimization method, where the Fisher information matrix can be viewed as an approximation of the Hessian. This perspective turns out to have significant implications for how to design a practical and robust version of the method. Additionally, we make the following contributions to the understanding of natural gradient and 2nd-order methods: a thorough analysis of the convergence speed of stochastic natural gradient descent (and more general stochastic 2nd-order methods) as applied to convex quadratics, a critical examination of the oft-used "empirical" approximation of the Fisher matrix, and an analysis of the (approximate) parameterization invariance property possessed by natural gradient methods, which we show still holds for certain choices of the curvature matrix other than the Fisher, but notably not the Hessian.

M. Barnett, R. Duvall

W. Wells, Paul A. Viola, H. Atsumi et al.

J. Yedidia, W. Freeman, Yair Weiss

G. Blobel, Joseph G. Gall, L. Glaser et al.

C. Bender, D. Brody, H. Jones

Requiring that a Hamiltonian be Hermitian is overly restrictive. A consistent physical theory of quantum mechanics can be built on a complex Hamiltonian that is not Hermitian but satisfies the less restrictive and more physical condition of space-time reflection symmetry (PT symmetry). One might expect a non-Hermitian Hamiltonian to lead to a violation of unitarity. However, if PT symmetry is not spontaneously broken, it is possible to construct a previously unnoticed symmetry C of the Hamiltonian. Using C, an inner product whose associated norm is positive definite can be constructed. The procedure is general and works for any PT-symmetric Hamiltonian. Observables exhibit CPT symmetry, and the dynamics is governed by unitary time evolution. This work is not in conflict with conventional quantum mechanics but is rather a complex generalization of it.

D. Altman, P. Royston

D. Whittaker