Scaling Laws for Neural Language Models
J. Kaplan, Sam McCandlish, T. Henighan
et al.
We study empirical scaling laws for language model performance on the cross-entropy loss. The loss scales as a power-law with model size, dataset size, and the amount of compute used for training, with some trends spanning more than seven orders of magnitude. Other architectural details such as network width or depth have minimal effects within a wide range. Simple equations govern the dependence of overfitting on model/dataset size and the dependence of training speed on model size. These relationships allow us to determine the optimal allocation of a fixed compute budget. Larger models are significantly more sample-efficient, such that optimally compute-efficient training involves training very large models on a relatively modest amount of data and stopping significantly before convergence.
7499 sitasi
en
Computer Science, Mathematics
Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
William L. Hamilton, J. Leskovec, Dan Jurafsky
Understanding how words change their meanings over time is key to models of language and cultural evolution, but historical data on meaning is scarce, making theories hard to develop and test. Word embeddings show promise as a diachronic tool, but have not been carefully evaluated. We develop a robust methodology for quantifying semantic change by evaluating word embeddings (PPMI, SVD, word2vec) against known historical changes. We then use this methodology to reveal statistical laws of semantic evolution. Using six historical corpora spanning four languages and two centuries, we propose two quantitative laws of semantic change: (i) the law of conformity---the rate of semantic change scales with an inverse power-law of word frequency; (ii) the law of innovation---independent of frequency, words that are more polysemous have higher rates of semantic change.
992 sitasi
en
Computer Science
Kinetics of thermal degradation of char-forming plastics from thermogravimetry. Application to a phenolic plastic
H. Friedman
3364 sitasi
en
Materials Science
A General Model for the Origin of Allometric Scaling Laws in Biology
Geoffrey B. West, James H. Brown, B. Enquist
Comparative Economic Organization: The Analysis of Discrete Structural Alternatives
O. Williamson
Whiteness as Property
C. Harris
Legal Determinants of External Finance
R. Porta, R. Porta, Florencio Lopez‐de‐Silanes
et al.
10105 sitasi
en
Economics, Business
Statistical mechanics of complex networks
R. Albert, A. Barabási
The emergence of order in natural systems is a constant source of inspiration for both physical and biological sciences. While the spatial order characterizing for example the crystals has been the basis of many advances in contemporary physics, most complex systems in nature do not offer such high degree of order. Many of these systems form complex networks whose nodes are the elements of the system and edges represent the interactions between them. Traditionally complex networks have been described by the random graph theory founded in 1959 by Paul Erdohs and Alfred Renyi. One of the defining features of random graphs is that they are statistically homogeneous, and their degree distribution (characterizing the spread in the number of edges starting from a node) is a Poisson distribution. In contrast, recent empirical studies, including the work of our group, indicate that the topology of real networks is much richer than that of random graphs. In particular, the degree distribution of real networks is a power-law, indicating a heterogeneous topology in which the majority of the nodes have a small degree, but there is a significant fraction of highly connected nodes that play an important role in the connectivity of the network. The scale-free topology of real networks has very important consequences on their functioning. For example, we have discovered that scale-free networks are extremely resilient to the random disruption of their nodes. On the other hand, the selective removal of the nodes with highest degree induces a rapid breakdown of the network to isolated subparts that cannot communicate with each other. The non-trivial scaling of the degree distribution of real networks is also an indication of their assembly and evolution. Indeed, our modeling studies have shown us that there are general principles governing the evolution of networks. Most networks start from a small seed and grow by the addition of new nodes which attach to the nodes already in the system. This process obeys preferential attachment: the new nodes are more likely to connect to nodes with already high degree. We have proposed a simple model based on these two principles wich was able to reproduce the power-law degree distribution of real networks. Perhaps even more importantly, this model paved the way to a new paradigm of network modeling, trying to capture the evolution of networks, not just their static topology.
19859 sitasi
en
Computer Science, Physics
How Much Should We Trust Differences-in-Differences Estimates?
Marianne Bertrand, E. Duflo, S. Mullainathan
10790 sitasi
en
Economics, Mathematics
Information Theory and Statistical Mechanics
E. Jaynes
12656 sitasi
en
Mathematics
Investor Protection and Corporate Governance
R. Porta, Florencio Lopez‐de‐Silanes, A. Shleifer
et al.
Recent research has documented large differences among countries in ownership concentration in publicly traded firms, in the breadth and depth of capital markets, in dividend policies, and in the access of firms to external finance. A common element to the explanations of these differences is how well investors, both shareholders and creditors, are protected by law from expropriation by the managers and controlling shareholders of firms. We describe the differences in laws and the effectiveness of their enforcement across countries, discuss the possible origins of these differences, summarize their consequences, and assess potential strategies of corporate governance reform. We argue that the legal approach is a more fruitful way to understand corporate governance and its reform than the conventional distinction between bank-centered and market-centered financial systems.
6693 sitasi
en
Business, Economics
Thermodynamics : An Engineering Approach
Y. Çengel, M. Boles
The Social Psychology of Procedural Justice
Allan Lind, J. Thibaut, L. Walker
et al.
6308 sitasi
en
Political Science
Theoretical Basis, Experimental Design, and Computerized Simulation of Synergism and Antagonism in Drug Combination Studies
T. Chou
4930 sitasi
en
Medicine, Computer Science
Enjoying the Quiet Life? Corporate Governance and Managerial Preferences
Marianne Bertrand, S. Mullainathan
Much of our understanding of corporations builds on the idea that managers, when they are not closely monitored, will pursue goals that are not in shareholders’ interests. But what goals would managers pursue? This paper uses variation in corporate governance generated by state adoption of antitakeover laws to empirically map out managerial preferences. We use plant‐level data and exploit a unique feature of corporate law that allows us to deal with possible biases associated with the timing of the laws. We find that when managers are insulated from takeovers, worker wages (especially those of white‐collar workers) rise. The destruction of old plants falls, but the creation of new plants also falls. Finally, overall productivity and profitability decline in response to these laws. Our results suggest that active empire building may not be the norm and that managers may instead prefer to enjoy the quiet life.
Accepted for Publication in the Astrophysical Journal Osse Observations of 3c 273
W. Johnson, C. Dermer, R. L. Kinzer
et al.
Homo Sacer: Sovereign Power and Bare Life
Kalliopi Nikolopoulou, Giorgio Agamben, D. Heller-Roazen
5338 sitasi
en
Philosophy
Composite medium with simultaneously negative permeability and permittivity
David R. Smith, Willie J Padilla, D. Vier
et al.
8319 sitasi
en
Medicine, Materials Science
Machine Learning Guided Polymorph Selection in Molecular Beam Epitaxy of In2Se3
Ryan Trice, Mintyu Yu, Eric Welp
et al.
Indium selenide (In2Se3), a layered chalcogenide with multiple polymorphs, is a promising material for optoelectronic and ferroelectric applications. However, achieving polymorph-pure thin films remains a major challenge due to the complex growth space. In this work, Bayesian Optimization (BO) is successfully leveraged to guide the molecular beam epitaxy (MBE) growth of In2Se3 on Al2O3 substrates. By training a predictive Gaussian Process Regressor with sequential learning, we efficiently explored substrate temperature, indium flux, selenium flux, and cracker temperature, reducing experimental trials required for successful synthesis. A γ-In2Se3 film with 91% phase purity was achieved after fewer than ten trials. Attempts to isolate α-In2Se3 were limited by amorphous film formation at low temperatures, indicating that single-step co-deposition is unsuitable for crystalline α-In2Se3 on Al2O3. Overall, this study validates Bayesian Optimization as a powerful approach for phase-selective growth in complex materials systems.
Вплив збройних конфліктів на довкілля: сучасні проблеми міжнародно-правової відповідальності
O. V. Hilevych, V. O. Shulga
У науковій статті досліджуються критичні питання впливу збройних конфліктів на навколишнє середовище крізь призму міжнародно-правової відповідальності. На тлі безпрецедентних руйнувань, спричинених сучасною війною в Україні, автори наголошують на тривожній тенденції: довкілля дедалі частіше перетворюється не лише на театр бойових дій, а й на цілеспрямовану мішень. Це призводить до масштабних і довготривалих екологічних катастроф, наслідки яких відчуватимуться десятиліттями. Дослідження обґрунтовує нагальну потребу в кардинальному оновленні міжнародного правового підходу до захисту навколишнього середовища в умовах війни, зокрема через запровадження та міжнародне визнання поняття «екоциду» як окремого злочину. Це дозволить притягати до відповідальності винних у найбільш тяжких екологічних злочинах.
Автори проводять ретельний аналіз чинних норм міжнародного гуманітарного права, включаючи Женевські конвенції та їхні додаткові протоколи. У роботі висвітлюється недостатня ефективність цих норм у сучасних умовах воєнного протистояння, де масштаби та характер екологічних руйнувань значно перевищують передбачені існуючим законодавством. Дослідження також зосереджується на таких ключових формах міжнародно-правової відповідальності, як репарації та сатисфакція. Вони розглядаються як дієві інструменти не лише для компенсації колосальних екологічних збитків, а й для досягнення всеосяжної справедливості для постраждалих громад та держав.
Особливу увагу приділено аналізу історичного прецеденту – діяльності Компенсаційної комісії ООН після війни в Перській затоці. Цей механізм, який забезпечив компенсацію збитків, завданих довкіллю, пропонується як життєздатна модель для створення аналогічного механізму в українському контексті. Запровадження такого механізму дозволило б не лише забезпечити необхідне фінансування для екологічного відновлення України, але й сформувати прозору та підзвітну систему відповідальності під пильним контролем міжнародних організацій.
Ця наукова стаття є важливим внеском у розвиток правової науки у сфері охорони довкілля та міжнародної відповідальності. Вона пропонує конкретні, обґрунтовані рішення щодо покращення міжнародної правової бази, адаптуючи її до нових, безпрецедентних викликів, які постають перед світовою спільнотою в умовах сучасних війн.