We present an interpretation of Inception modules in convolutional neural networks as being an intermediate step in-between regular convolution and the depthwise separable convolution operation (a depthwise convolution followed by a pointwise convolution). In this light, a depthwise separable convolution can be understood as an Inception module with a maximally large number of towers. This observation leads us to propose a novel deep convolutional neural network architecture inspired by Inception, where Inception modules have been replaced with depthwise separable convolutions. We show that this architecture, dubbed Xception, slightly outperforms Inception V3 on the ImageNet dataset (which Inception V3 was designed for), and significantly outperforms Inception V3 on a larger image classification dataset comprising 350 million images and 17,000 classes. Since the Xception architecture has the same number of parameters as Inception V3, the performance gains are not due to increased capacity but rather to a more efficient use of model parameters.
Deep residual networks were shown to be able to scale up to thousands of layers and still have improving performance. However, each fraction of a percent of improved accuracy costs nearly doubling the number of layers, and so training very deep residual networks has a problem of diminishing feature reuse, which makes these networks very slow to train. To tackle these problems, in this paper we conduct a detailed experimental study on the architecture of ResNet blocks, based on which we propose a novel architecture where we decrease depth and increase width of residual networks. We call the resulting network structures wide residual networks (WRNs) and show that these are far superior over their commonly used thin and very deep counterparts. For example, we demonstrate that even a simple 16-layer-deep wide residual network outperforms in accuracy and efficiency all previous deep residual networks, including thousand-layer-deep networks, achieving new state-of-the-art results on CIFAR, SVHN, COCO, and significant improvements on ImageNet. Our code and models are available at this https URL
Saining Xie, Ross B. Girshick, Piotr Dollár
et al.
We present a simple, highly modularized network architecture for image classification. Our network is constructed by repeating a building block that aggregates a set of transformations with the same topology. Our simple design results in a homogeneous, multi-branch architecture that has only a few hyper-parameters to set. This strategy exposes a new dimension, which we call cardinality (the size of the set of transformations), as an essential factor in addition to the dimensions of depth and width. On the ImageNet-1K dataset, we empirically show that even under the restricted condition of maintaining complexity, increasing cardinality is able to improve classification accuracy. Moreover, increasing cardinality is more effective than going deeper or wider when we increase the capacity. Our models, named ResNeXt, are the foundations of our entry to the ILSVRC 2016 classification task in which we secured 2nd place. We further investigate ResNeXt on an ImageNet-5K set and the COCO detection set, also showing better results than its ResNet counterpart. The code and models are publicly available online.
We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin.
Tsung-Yi Lin, Piotr Dollár, Ross B. Girshick
et al.
Feature pyramids are a basic component in recognition systems for detecting objects at different scales. But pyramid representations have been avoided in recent object detectors that are based on deep convolutional networks, partially because they are slow to compute and memory intensive. In this paper, we exploit the inherent multi-scale, pyramidal hierarchy of deep convolutional networks to construct feature pyramids with marginal extra cost. A top-down architecture with lateral connections is developed for building high-level semantic feature maps at all scales. This architecture, called a Feature Pyramid Network (FPN), shows significant improvement as a generic feature extractor in several applications. Using a basic Faster R-CNN system, our method achieves state-of-the-art single-model results on the COCO detection benchmark without bells and whistles, surpassing all existing single-model entries including those from the COCO 2016 challenge winners. In addition, our method can run at 5 FPS on a GPU and thus is a practical and accurate solution to multi-scale object detection. Code will be made publicly available.
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke
et al.
Very deep convolutional networks have been central to the largest advances in image recognition performance in recent years. One example is the Inception architecture that has been shown to achieve very good performance at relatively low computational cost. Recently, the introduction of residual connections in conjunction with a more traditional architecture has yielded state-of-the-art performance in the 2015 ILSVRC challenge; its performance was similar to the latest generation Inception-v3 network. This raises the question: Are there any benefits to combining Inception architectures with residual connections? Here we give clear empirical evidence that training with residual connections accelerates the training of Inception networks significantly. There is also some evidence of residual Inception networks outperforming similarly expensive Inception networks without residual connections by a thin margin. We also present several new streamlined architectures for both residual and non-residual Inception networks. These variations improve the single-frame recognition performance on the ILSVRC 2012 classification task significantly. We further demonstrate how proper activation scaling stabilizes the training of very wide residual Inception networks. With an ensemble of three residual and one Inception-v4 networks, we achieve 3.08% top-5 error on the test set of the ImageNet classification (CLS) challenge.
Top-performing deep architectures are trained on massive amounts of labeled data. In the absence of labeled data for a certain task, domain adaptation often provides an attractive option given that labeled data of similar nature but from a different domain (e.g. synthetic images) are available. Here, we propose a new approach to domain adaptation in deep architectures that can be trained on large amount of labeled data from the source domain and large amount of unlabeled data from the target domain (no labeled target-domain data is necessary). As the training progresses, the approach promotes the emergence of "deep" features that are (i) discriminative for the main learning task on the source domain and (ii) invariant with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a simple new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation. Overall, the approach can be implemented with little effort using any of the deep-learning packages. The method performs very well in a series of image classification experiments, achieving adaptation effect in the presence of big domain shifts and outperforming previous state-of-the-art on Office datasets.
In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.
We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.
We investigate architectures of discriminatively trained deep Convolutional Networks (ConvNets) for action recognition in video. The challenge is to capture the complementary information on appearance from still frames and motion between frames. We also aim to generalise the best performing hand-crafted features within a data-driven learning framework. Our contribution is three-fold. First, we propose a two-stream ConvNet architecture which incorporates spatial and temporal networks. Second, we demonstrate that a ConvNet trained on multi-frame dense optical flow is able to achieve very good performance in spite of limited training data. Finally, we show that multitask learning, applied to two different action classification datasets, can be used to increase the amount of training data and improve the performance on both. Our architecture is trained and evaluated on the standard video actions benchmarks of UCF-101 and HMDB-51, where it is competitive with the state of the art. It also exceeds by a large margin previous attempts to use deep nets for video classification.
We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.
To address the issue of low closed-loop feedback bandwidth caused by the long latency of Carrier-Envelope Phase (CEP) measurement systems for amplified femtosecond laser pulses, and to meet the requirements for real-time single-shot measurement in 10 kHz repetition rate systems, this paper proposes a microsecond-level low-latency CEP measurement technique based on a Field-Programmable Gate Array (FPGA). To tackle the problem of non-uniform spectral sampling resulting from nonlinear wavelength-frequency mapping, the system implements a real-time linear interpolation algorithm for the interference spectrum. This approach effectively suppresses computational spurious peaks introduced by non-uniform sampling and significantly reduces measurement errors. Adopting a fully pipelined parallel processing architecture, the system achieves a CEP processing latency of approximately 89 μs, representing an improvement of 2–3 orders of magnitude compared to traditional Central Processing Unit (CPU)-based solutions. Hardware-in-the-loop testing, conducted by injecting a known sinusoidal phase modulation into the interference spectrum of a 10 kHz laser amplification system, demonstrates that the computational error of the proposed algorithm is less than 30 mrad. This work paves the way for achieving single-shot CEP feedback locking in high-repetition-rate laser amplification systems.
Deep coal resources with abundant reserves and considerable thermal potential are receiving increased attention in mining engineering, given the accelerating transformation of the global energy structure and the growing demand for clean energy. To address extraction challenges and environmental pressures while ensuring economic feasibility and sustainable development, efforts are made to enable carbon reduction and green transformation under high-efficiency utilization of deep coal resources. A systematic review of “deep coal resource fluidized mining”, “coal chemical mining”, and “coal-based power” informs the introduction of a detonation-generation mining approach and its technical framework. The approach places coal-powder detonation combustion technology at its core and integrates advanced detonation combustion-mechanical/magnetohydrodynamic power generation, forming a detonation-turbine/MHD hybrid power system that supports efficient conversion and clean utilization of coal resources. Four fundamental theories are presented, including the Coal-powder Detonation Energy Release mechanism, the Coupled Coal-powder Detonation-generation Power Scheme, a Full Life Cycle Detonation-power Generation Dynamic Management Mechanism, and the Blasting-electric Power Deep coal mining theory and method. Discussion centers on four key technologies: Stable coal/gas two-phase detonation, detonation model construction and dynamic process optimization, detonation-based power generation efficiency assessment, and comprehensive design for detonation-based coal mining, demonstrating their role in upgrading deep coal mining practices. On this foundation, a systematic engineering strategy is proposed to clarify the synergy between mining processes and the detonation-based power generation mode, highlight safety management and process optimization priorities at each critical stage, and refine the overall detonation-generation pathway for deep coal resource development. This pathway offers valuable insights for establishing a coal-based power system and promoting the clean and efficient utilization of deep coal resources in China.
This article presents a model for describing the architecture of software-intensive systems, based on the use of multiple, concurrent views. This use of multiple views allows to address separately the concerns of the various ‘stakeholders’ of the architecture: end-user, developers, systems engineers, project managers, etc., and to handle separately the functional and non functional requirements. Each of the five views is described, together with a notation to capture it. The views are designed using an architecture-centered, scenariodriven, iterative development process.
Laura Nistor, Cătălin Lisa, Tsuyoshi Michinobu
et al.
Background: 2-[4-(Dimethylamino)phenyl]-3-([4-(dimethylamino)phenyl]ethynyl)buta-1,3-diene-1,1,4,4-tetracarbonitrile (DDMEBT) is a thermally robust organic material of interest for applications requiring controlled volatility. Understanding its thermal stability, decomposition mechanism, and sublimation behavior is critical for optimizing deposition conditions in industrial processes. Methods: A comprehensive set of techniques was employed, including thermogravimetric analysis coupled with mass spectrometry and FTIR spectroscopy (TG/MS/FTIR), differential scanning calorimetry (DSC), ATR-FTIR spectroscopy, X-ray diffraction (XRD), scanning electron microscopy (SEM), energy-dispersive X-ray spectroscopy (EDX), dynamic vapor sorption (DVS) analysis, polarized light microscopy (POM), and molecular modeling. Sublimation kinetics were investigated under isothermal conditions (130–150 °C) using anthracene as reference. Significant findings: DDMEBT exhibits a sequential three-step degradation mechanism, independent of heating rate, with high thermal stability (final residue ∼77 %) attributed to its nonplanar architecture and intermolecular π–π/dipole–dipole interactions. Thermal analysis revealed melting at ∼190 °C, structural rearrangements (196–230 °C), and an amorphous-to-crystalline transition at 270 °C. Sublimation proceeds via zero-order kinetics with low volatility (0.178 μg/min at 130 °C) and an activation energy of 66.5 kJ/mol. The determined vapor pressure (1998–4000 Pa) and transport coefficients confirm a thermally activated, hydrodynamically stable process. These findings establish a reliable basis for sublimation modeling and provide guidelines for optimizing material processing in high-temperature, low-volatility applications.
As the world’s largest loess deposit region, the Loess Plateau’s vegetation dynamics are crucial for its regional water–heat balance and ecosystem functioning. Leaf Area Index (LAI) serves as a key indicator bridging canopy architecture and plant physiological activities. Existing studies have made significant advancements in simulating LAI, yet accurate LAI simulation remains challenging. To address this challenge and gain deeper insights into the environmental controls of LAI, this study aims to accurately simulate LAI in the Loess Plateau using deep learning models and to elucidate the spatiotemporal influence of soil moisture and temperature on LAI dynamics. For this purpose, we used three deep learning models, namely Artificial Neural Network (ANN), Long Short-Term Memory (LSTM), and Interpretable Multivariable (IMV)-LSTM, to simulate LAI in the Loess Plateau, only using soil moisture and temperature as inputs. Results indicated that our approach outperformed traditional models and effectively captured LAI variations across different vegetation types. The attention analysis revealed that soil moisture mainly influenced LAI in the arid northwest and temperature was the predominant effect in the humid southeast. Seasonally, soil moisture was crucial in spring and summer, notably in grasslands and croplands, whereas temperature dominated in autumn and winter. Notably, forests had the longest temperature-sensitive periods. As LAI increased, soil moisture became more influential, and at peak LAI, both factors exerted varying controls on different vegetation types. These findings demonstrated the strength of deep learning for simulating vegetation–climate interactions and provided insights into hydrothermal regulation mechanisms in semiarid regions.
Ludmila Cardoso Fagundes Mendes, Roberta Vieira Gonçalves de Souza, Danielly Marcianny Silva Eulário
É fundamental criar ambientes hospitalares acessíveis, acolhedores e confortáveis. Infere-se que arquitetos, designers e engenheiros exercem um papel crucial durante o ciclo de vida da edificação hospitalar. No entanto, entende-se que a participação dos demais usuários do ambiente hospitalar é importante também na idealização de mudanças no espaço físico. Esta pesquisa teve como objetivo, examinar a perspectiva de profissionais influentes no planejamento de edifícios hospitalares quanto à introdução de estímulos no ambiente físico. Indicadores de bem-estar foram selecionados a partir das Teorias do Design de Suporte e do Design Baseado em Evidências, tendo sido aplicado um questionário versando sobre Senso de Controle, Apoio Social, Distrações Positivas e Iluminação Natural. Participaram 96 arquitetos, designers, engenheiros, gestores hospitalares, médicos, enfermeiros e outros profissionais da área. As respostas indicaram que, para a maioria dos respondentes, o ambiente físico interfere muito no bem-estar de todos os tipos de usuário de Estabelecimentos Assistenciais de Saúde (EAS) sendo a presença de iluminação natural o componente considerado mais relevante para seu bem-estar. Segundo os profissionais, os indicadores de bem-estar têm relevância superior para pacientes internados em enfermarias e para funcionários da assistência em regime de plantão igual ou superior a 12h, e menor relevância para pacientes não internados. Preservar a privacidade dos pacientes e disponibilizar iluminação natural em quartos e em enfermarias foram os indicadores de bem-estar mais priorizados pelos participantes da pesquisa. De modo geral, os projetistas tendem a priorizar mais os indicadores em novas edificações de EAS do que em reformas ou ampliações.
Architecture, Urban groups. The city. Urban sociology