Towards Explainable Deep Learning for Ship Trajectory Prediction in Inland Waterways
Tom Legel, Dirk Söffker, Roland Schätzle
et al.
Accurate predictions of ship trajectories in crowded environments are essential to ensure safety in inland waterways traffic. Recent advances in deep learning promise increased accuracy even for complex scenarios. While the challenge of ship-to-ship awareness is being addressed with growing success, the explainability of these models is often overlooked, potentially obscuring an inaccurate logic and undermining the confidence in their reliability. This study examines an LSTM-based vessel trajectory prediction model by incorporating trained ship domain parameters that provide insight into the attention-based fusion of the interacting vessels' hidden states. This approach has previously been explored in the field of maritime shipping, yet the variety and complexity of encounters in inland waterways allow for a more profound analysis of the model's interpretability. The prediction performance of the proposed model variants are evaluated using standard displacement error statistics. Additionally, the plausibility of the generated ship domain values is analyzed. With an final displacement error of around 40 meters in a 5-minute prediction horizon, the model performs comparably to similar studies. Though the ship-to-ship attention architecture enhances prediction accuracy, the weights assigned to vessels in encounters using the learnt ship domain values deviate from the expectation. The observed accuracy improvements are thus not entirely driven by a causal relationship between a predicted trajectory and the trajectories of nearby ships. This finding underscores the model's explanatory capabilities through its intrinsically interpretable design. Future work will focus on utilizing the architecture for counterfactual analysis and on the incorporation of more sophisticated attention mechanisms.
Inland-LOAM: Voxel-Based Structural Semantic LiDAR Odometry and Mapping for Inland Waterway Navigation
Zhongbi Luo, Yunjia Wang, Jan Swevers
et al.
Accurate geospatial information is crucial for safe, autonomous Inland Waterway Transport (IWT), as existing charts (IENC) lack real-time detail and conventional LiDAR SLAM fails in waterway environments. These challenges lead to vertical drift and non-semantic maps, hindering autonomous navigation. This paper introduces Inland-LOAM, a LiDAR SLAM framework for waterways. It uses an improved feature extraction and a water surface planar constraint to mitigate vertical drift. A novel pipeline transforms 3D point clouds into structured 2D semantic maps using voxel-based geometric analysis, enabling real-time computation of navigational parameters like bridge clearances. An automated module extracts shorelines and exports them into a lightweight, IENC-compatible format. Evaluations on a real-world dataset show Inland-LOAM achieves superior localization accuracy over state-of-the-art methods. The generated semantic maps and shorelines align with real-world conditions, providing reliable data for enhanced situational awareness. The code and dataset will be publicly available
Safe Robust Predictive Control-based Motion Planning of Automated Surface Vessels in Inland Waterways
Sajad Ahmadi, Hossein Nejatbakhsh Esfahani, Javad Mohammadpour Velni
Deploying self-navigating surface vessels in inland waterways offers a sustainable alternative to reduce road traffic congestion and emissions. However, navigating confined waterways presents unique challenges, including narrow channels, higher traffic density, and hydrodynamic disturbances. Existing methods for autonomous vessel navigation often lack the robustness or precision required for such environments. This paper presents a new motion planning approach for Automated Surface Vessels (ASVs) using Robust Model Predictive Control (RMPC) combined with Control Barrier Functions (CBFs). By incorporating channel borders and obstacles as safety constraints within the control design framework, the proposed method ensures both collision avoidance and robust navigation on complex waterways. Simulation results demonstrate the efficacy of the proposed method in safely guiding ASVs under realistic conditions, highlighting its improved safety and adaptability compared to the state-of-the-art.
Inland Waterway Object Detection in Multi-environment: Dataset and Approach
Shanshan Wang, Haixiang Xu, Hui Feng
et al.
The success of deep learning in intelligent ship visual perception relies heavily on rich image data. However, dedicated datasets for inland waterway vessels remain scarce, limiting the adaptability of visual perception systems in complex environments. Inland waterways, characterized by narrow channels, variable weather, and urban interference, pose significant challenges to object detection systems based on existing datasets. To address these issues, this paper introduces the Multi-environment Inland Waterway Vessel Dataset (MEIWVD), comprising 32,478 high-quality images from diverse scenarios, including sunny, rainy, foggy, and artificial lighting conditions. MEIWVD covers common vessel types in the Yangtze River Basin, emphasizing diversity, sample independence, environmental complexity, and multi-scale characteristics, making it a robust benchmark for vessel detection. Leveraging MEIWVD, this paper proposes a scene-guided image enhancement module to improve water surface images based on environmental conditions adaptively. Additionally, a parameter-limited dilated convolution enhances the representation of vessel features, while a multi-scale dilated residual fusion method integrates multi-scale features for better detection. Experiments show that MEIWVD provides a more rigorous benchmark for object detection algorithms, and the proposed methods significantly improve detector performance, especially in complex multi-environment scenarios.
Visual Trajectory Prediction of Vessels for Inland Navigation
Alexander Puzicha, Konstantin Wüstefeld, Kathrin Wilms
et al.
The future of inland navigation increasingly relies on autonomous systems and remote operations, emphasizing the need for accurate vessel trajectory prediction. This study addresses the challenges of video-based vessel tracking and prediction by integrating advanced object detection methods, Kalman filters, and spline-based interpolation. However, existing detection systems often misclassify objects in inland waterways due to complex surroundings. A comparative evaluation of tracking algorithms, including BoT-SORT, Deep OC-SORT, and ByeTrack, highlights the robustness of the Kalman filter in providing smoothed trajectories. Experimental results from diverse scenarios demonstrate improved accuracy in predicting vessel movements, which is essential for collision avoidance and situational awareness. The findings underline the necessity of customized datasets and models for inland navigation. Future work will expand the datasets and incorporate vessel classification to refine predictions, supporting both autonomous systems and human operators in complex environments.
Optimizing Periodic Operations for Efficient Inland Waterway Lock Management
Julian Golak, Alexander Grigoriev, Freija van Lent
et al.
In inland waterways, the efficient management of water lock operations impacts the level of congestion and the resulting uncertainty in inland waterway transportation. To achieve reliable and efficient traffic, schedules should be easy to understand and implement, reducing the likelihood of errors. The simplest schedules follow periodic patterns, reducing complexity and facilitating predictable management. Since vessels do not arrive in perfectly regular intervals, periodic schedules may lead to more wait time. The aim of this research is to estimate this cost by evaluating how effective these periodic schedules manage vessel traffic at water locks. The first objective is to estimate a periodic arrival pattern that closely matches a dataset of irregular vessel arrivals at a specific lock. We develop an algorithm that, given a fixed number of vessel streams, solves the problem in polynomial time. The solution then serves as input for the subsequent part, where we consider algorithms that compute operational schedules by formulating an optimisation problem with periodic arrival patterns as input, and the goal is to determine a periodic schedule that minimises the long-run average waiting time of vessels. We present a polynomial-time algorithm for the two-stream case and a pseudo-polynomial-time algorithm for the general case, along with incremental polynomial-time approximation schemes. In our numerical experiments, use AIS data to construct a periodic arrival pattern closely matching the observed data. Our experiments demonstrate that when evaluated against actual data, intuitive and straightforward policies often outperform optimal policies specifically trained on the periodic arrival pattern.
USVTrack: USV-Based 4D Radar-Camera Tracking Dataset for Autonomous Driving in Inland Waterways
Shanliang Yao, Runwei Guan, Yi Ni
et al.
Object tracking in inland waterways plays a crucial role in safe and cost-effective applications, including waterborne transportation, sightseeing tours, environmental monitoring and surface rescue. Our Unmanned Surface Vehicle (USV), equipped with a 4D radar, a monocular camera, a GPS, and an IMU, delivers robust tracking capabilities in complex waterborne environments. By leveraging these sensors, our USV collected comprehensive object tracking data, which we present as USVTrack, the first 4D radar-camera tracking dataset tailored for autonomous driving in new generation waterborne transportation systems. Our USVTrack dataset presents rich scenarios, featuring diverse various waterways, varying times of day, and multiple weather and lighting conditions. Moreover, we present a simple but effective radar-camera matching method, termed RCM, which can be plugged into popular two-stage association trackers. Experimental results utilizing RCM demonstrate the effectiveness of the radar-camera matching in improving object tracking accuracy and reliability for autonomous driving in waterborne environments. The USVTrack dataset is public on https://usvtrack.github.io.
Enhancing Maritime Domain Awareness on Inland Waterways: A YOLO-Based Fusion of Satellite and AIS for Vessel Characterization
Geoffery Agorku, Sarah Hernandez, Hayley Hames
et al.
Maritime Domain Awareness (MDA) for inland waterways remains challenged by cooperative system vulnerabilities. This paper presents a novel framework that fuses high-resolution satellite imagery with vessel trajectory data from the Automatic Identification System (AIS). This work addresses the limitations of AIS-based monitoring by leveraging non-cooperative satellite imagery and implementing a fusion approach that links visual detections with AIS data to identify dark vessels, validate cooperative traffic, and support advanced MDA. The You Only Look Once (YOLO) v11 object detection model is used to detect and characterize vessels and barges by vessel type, barge cover, operational status, barge count, and direction of travel. An annotated data set of 4,550 instances was developed from $5{,}973~\mathrm{mi}^2$ of Lower Mississippi River imagery. Evaluation on a held-out test set demonstrated vessel classification (tugboat, crane barge, bulk carrier, cargo ship, and hopper barge) with an F1 score of 95.8\%; barge cover (covered or uncovered) detection yielded an F1 score of 91.6\%; operational status (staged or in motion) classification reached an F1 score of 99.4\%. Directionality (upstream, downstream) yielded 93.8\% accuracy. The barge count estimation resulted in a mean absolute error (MAE) of 2.4 barges. Spatial transferability analysis across geographically disjoint river segments showed accuracy was maintained as high as 98\%. These results underscore the viability of integrating non-cooperative satellite sensing with AIS fusion. This approach enables near-real-time fleet inventories, supports anomaly detection, and generates high-quality data for inland waterway surveillance. Future work will expand annotated datasets, incorporate temporal tracking, and explore multi-modal deep learning to further enhance operational scalability.
Performance Analysis of MADOCA-Enhanced Tightly Coupled PPP/IMU
Cheng-Wei Wang, Shau-Shiun Jan
Precise point positioning (PPP), which is characterized by reliable positioning accuracy and flexibility, has been regarded as a highly promising technique. Precise ephemeris is essential for PPP; however, the conventionally used standard product 3 components have an almost biweekly latency. The multi-global navigation satellite system (GNSS) advanced demonstration tool for orbit and clock analysis (MADOCA), a novel next-generation service, aims to provide real-time correction messages for rapid-convergence PPP in regional areas. Additionally, to ensure seamless navigation during signal-interrupted conditions, an inertial measurement unit (IMU) can be tightly integrated with the motion constraint models. This paper presents a comprehensive analysis of standalone MADOCA-PPP and MADOCA-enhanced tightly coupled PPP/IMU. The approaches were evaluated under multiple scenarios. In suburban regions, the horizontal root mean square error (RMSE) was 0.4 m, with a 95th percentile horizontal error of 0.6 m. In GNSS-challenging environments, the horizontal RMSE was 0.92 m, with a 95th percentile horizontal error of 1.6 m.
Canals and inland navigation. Waterways, Naval Science
A Horizontal Accuracy Metric for Magnetic Navigation
Prasenjit Sengupta
This paper develops a navigation accuracy metric for magnetic navigation when adapted to civil aviation. Metrics that are currently used may not directly apply to magnetic navigation because the assumptions behind these metrics are based on the statistics of more conventional navigation modalities where the uncertainty distribution in the horizontal plane is typically radially symmetric. Magnetic navigation challenges these assumptions. New standard deviation limits based on the probability of exceedance of radial error are derived. Use is made of quadratic forms over random variables, specifically the Hoyt distribution and its associated probability density function and cumulative density function. Although the density functions lack closed-form solutions, new approximations for this distribution are utilized in order to make rapid computation possible, enabling their use for navigation purposes. When used in conjunction with robust numerical methods, the new approach accurately calculates the bounds on error distributions that would meet the requirements associated with Performance Based Navigation.
Canals and inland navigation. Waterways, Naval Science
Doppler Positioning Using Multi-Constellation LEO Satellite Broadband Signals as Signals of Opportunity
Amir Allahvirdi-Zadeh, Ahmed El-Mowafy, Kan Wang
This paper investigates the potential of signals of opportunity for positioning using broadband low Earth orbit constellations. We developed analytical absolute and differential models based on Doppler-shift observations from multi-constellation satellite bursts across various frequency ranges. Owing to the unavailability of multi-constellation broadband receivers, simulations were conducted with the application of two primary restrictions common for these satellites: a 30° elevation mask angle and a 15-s intermittency for observations. Signal attenuation factors were modeled, indicating that free space loss was the dominant factor whereas cloud and fog losses were minimal. The accuracy of absolute static positioning, considering the aforementioned broadband restrictions, reached 4.32 m. The kinematic receiver showed similar trends, with a degraded accuracy of 4.83 m. Tests in urban areas revealed significant accuracy degradation to approximately 10 m. However, the differential model significantly improved kinematic positioning accuracy, achieving promising sub-meter levels even with a limited number of satellites.
Canals and inland navigation. Waterways, Naval Science
SS-RAIM-Based Integrity Architecture for CDGNSSs Against Satellite Measurement Faults
Dongchan Min, Noah Minchan Kim, Gihun Nam
et al.
Carrier-phase differential global navigation satellite systems (CDGNSSs) present an attractive option for autonomous vehicles that require accurate and safe navigation. The key to high precision in a CDGNSS is resolving integer ambiguities. However, the discrete nature of ambiguities complicates the analysis of position errors in relation to satellite measurement faults, which poses challenges in protection level (PL) calculation. This paper presents an integrity architecture based on solution separation receiver autonomous integrity monitoring. The test statistic for this monitor is defined in the position domain, directly capturing position errors due to faults. This approach facilitates easier and less conservative evaluations of PLs. This paper provides detailed derivations of PLs and monitor thresholds starting from a common definition of integrity and continuity risk. Additionally, this work presents a method for ensuring that PLs reliably bound actual position errors using a measurement overbounding technique. Simulation results show that the monitor detects most faults and that the PLs bound the position errors from undetected faults.
Canals and inland navigation. Waterways, Naval Science
Classification of Authentic and Spoofed GNSS Signals Using a Calibrated Antenna Array
Michael C. Esswein, Mark L. Psiaki
New optimization-based methods have been developed to use measured direction-of-arrival (DoA) information in order to classify received global navigation satellite system signals into authenticated and spoofed sets and to augment that information with pseudorange information when DoA information alone is insufficient to achieve the needed classification. These methods are designed for a system that is being developed to mitigate spoofing and jamming by using signals from a controlled radiation pattern antenna. These new spoofing classification methods operate on DoA outputs from trackers of various signals. This paper presents a multi-hypothesis test that considers all possible hypotheses regarding the authenticated and spoofed sets of tracked signals. A combinatorial analysis is performed in which all possible authenticated-set/spoofed-set classifications are generated for a given set of tracked signals and the correct authenticated set is determined among the different combinations. Results from Monte Carlo simulations show that using a combined DoA and pseudorange method is suitable for determining the correct combinations.
Canals and inland navigation. Waterways, Naval Science
Structural health monitoring of inland navigation structures and ports: a review on developments and challenges
P. Negi, R. Kromanis, A. Dorée
et al.
Inland navigation structures (INS) facilitate transportation of goods in rivers and canals. Transportation of goods over waterways is more energy efficient than on roads and railways. INS, similar to other civil structures, are aging and require frequent condition assessment and maintenance. Countries, in which INS are important to their economies, such as the Netherlands and the United States, allocate significant budgets for maintenance and renovation of exiting INS, as well as for building new structures. Timely maintenance and early detection of a change to material or geometric properties (i.e., damage) can be supported with the structural health monitoring (SHM), in which monitored data, such as load, structural response, environmental actions, are analyzed. Huge scientific efforts are realized in bridge SHM, but when it comes to SHM of INS, the efforts are significantly lower. Therefore, the SHM community has opportunities to develop new solutions for SHM of INS and convince asset owners of their benefits. This review article, first, articulates the need to keep INS safe to use and fit for purpose, and the challenges associated with it. Second, it defines and reviews sensors, sensing technologies, and approaches for SHM of INS. Then, INS and their components, including structures in ports, are identified, described, and illustrated, and their monitoring efforts are reviewed. Finally, the review article emphasizes the added value of SHM systems for INS, concludes on the current achievements, and proposes future trajectories for SHM of INS and ports.
Analysis of vessel traffic flow characteristics in inland restricted waterways using multi-source data
Wenzhang Yang, Peng Liao, Shangkun Jiang
et al.
To effectively manage vessel traffic and alleviate congestion on busy inland waterways, a comprehensive understanding of vessel traffic flow characteristics is crucial. However, limited data availability has resulted in minimal research on the traffic flow characteristics of inland waterway vessels. This study addresses this gap by conducting vessel-following experiments and fixed-point video monitoring in inland waterways, collecting multi-source data to analyze vessel traffic flow characteristics. First, the analysis of vessel speed distribution identifies the economic speed for vessels operating in these environments. Next, the relationship between microscopic vessel speed and gap distance is examined, with the logarithmic model emerging as the most accurate among various tested models. Additionally, the study explores the relationships among macroscopic speed, density, and flow rate, proposing a novel piecewise fundamental diagram model to describe these relationships. Lastly, the inland vessel traffic states are categorized using K-means clustering algorithm and applied to vessel navigation services. These findings provide valuable insights for enhancing inland waterway transportation and advancing the development of an integrated waterway transportation system.
Distributed MPC for autonomous ships on inland waterways with collaborative collision avoidance
Hoang Anh Tran, Tor Arne Johansen, Rudy R. Negenborn
This paper presents a distributed solution for the problem of collaborative collision avoidance for autonomous inland waterway ships. A two-layer collision avoidance framework that considers inland waterway traffic regulations is proposed to increase navigational safety for autonomous ships. Our approach allows for modifying traffic rules without changing the collision avoidance algorithm, and is based on a novel formulation of model predictive control (MPC) for collision avoidance of ships. This MPC formulation is designed for inland waterway traffic and can handle complex scenarios. The alternating direction method of multipliers is used as a scheme for exchanging and negotiating intentions among ships. Simulation results show that the proposed algorithm can comply with traffic rules. Furthermore, the proposed algorithm can safely deviate from traffic rules when necessary to increase efficiency in complex scenarios.
A Lightweight Target-Driven Network of Stereo Matching for Inland Waterways
Jing Su, Yiqing Zhou, Yu Zhang
et al.
Stereo matching for inland waterways is one of the key technologies for the autonomous navigation of Unmanned Surface Vehicles (USVs), which involves dividing the stereo images into reference images and target images for pixel-level matching. However, due to the challenges of the inland waterway environment, such as blurred textures, large spatial scales, and computational resource constraints of the USVs platform, the participation of geometric features from the target image is required for efficient target-driven matching. Based on this target-driven concept, we propose a lightweight target-driven stereo matching neural network, named LTNet. Specifically, a lightweight and efficient 4D cost volume, named the Geometry Target Volume (GTV), is designed to fully utilize the geometric information of target features by employing the shifted target features as the filtered feature volume. Subsequently, to address the substantial texture interference and object occlusions present in the waterway environment, a Left-Right Consistency Refinement (LRR) module is proposed. The \text{LRR} utilizes the pixel-level differences in left and right disparities to introduce soft constraints, thereby enhancing the accuracy of predictions during the intermediate stages of the network. Moreover, knowledge distillation is utilized to enhance the generalization capability of lightweight models on the USVInland dataset. Furthermore, a new large-scale benchmark, named Spring, is utilized to validate the applicability of LTNet across various scenarios. In experiments on the aforementioned two datasets, LTNet achieves competitive results, with only 3.7M parameters. The code is available at https://github.com/Open-YiQingZhou/LTNet .
Predicting Barge Presence and Quantity on Inland Waterways using Vessel Tracking Data: A Machine Learning Approach
Geoffery Agorku, Sarah Hernandez, Maria Falquez
et al.
This study presents a machine learning approach to predict the number of barges transported by vessels on inland waterways using tracking data from the Automatic Identification System (AIS). While AIS tracks the location of tug and tow vessels, it does not monitor the presence or number of barges transported by those vessels. Understanding the number and types of barges conveyed along river segments, between ports, and at ports is crucial for estimating the quantities of freight transported on the nation's waterways. This insight is also valuable for waterway management and infrastructure operations impacting areas such as targeted dredging operations, and data-driven resource allocation. Labeled sample data was generated using observations from traffic cameras located along key river segments and matched to AIS data records. A sample of 164 vessels representing up to 42 barge convoys per vessel was used for model development. The methodology involved first predicting barge presence and then predicting barge quantity. Features derived from the AIS data included speed measures, vessel characteristics, turning measures, and interaction terms. For predicting barge presence, the AdaBoost model achieved an F1 score of 0.932. For predicting barge quantity, the Random Forest combined with an AdaBoost ensemble model achieved an F1 score of 0.886. Bayesian optimization was used for hyperparameter tuning. By advancing predictive modeling for inland waterways, this study offers valuable insights for transportation planners and organizations, which require detailed knowledge of traffic volumes, including the flow of commodities, their destinations, and the tonnage moving in and out of ports.
Traffic Cameras to detect inland waterway barge traffic: An Application of machine learning
Geoffery Agorku, Sarah Hernandez PhD, Maria Falquez
et al.
Inland waterways are critical for freight movement, but limited means exist for monitoring their performance and usage by freight-carrying vessels, e.g., barges. While methods to track vessels, e.g., tug and tow boats, are publicly available through Automatic Identification Systems (AIS), ways to track freight tonnages and commodity flows carried on barges along these critical marine highways are non-existent, especially in real-time settings. This paper develops a method to detect barge traffic on inland waterways using existing traffic cameras with opportune viewing angles. Deep learning models, specifically, You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), and EfficientDet are employed. The model detects the presence of vessels and/or barges from video and performs a classification (no vessel or barge, vessel without barge, vessel with barge, and barge). A dataset of 331 annotated images was collected from five existing traffic cameras along the Mississippi and Ohio Rivers for model development. YOLOv8 achieves an F1-score of 96%, outperforming YOLOv5, SSD, and EfficientDet models with 86%, 79%, and 77% respectively. Sensitivity analysis was carried out regarding weather conditions (fog and rain) and location (Mississippi and Ohio rivers). A background subtraction technique was used to normalize video images across the various locations for the location sensitivity analysis. This model can be used to detect the presence of barges along river segments, which can be used for anonymous bulk commodity tracking and monitoring. Such data is valuable for long-range transportation planning efforts carried out by public transportation agencies, in addition to operational and maintenance planning conducted by federal agencies such as the US Army Corp of Engineers.
Incorporating Navigation Context into Inland Vessel Trajectory Prediction: A Gaussian Mixture Model and Transformer Approach
Kathrin Donandt, Dirk Söffker
Using data sources beyond the Automatic Identification System to represent the context a vessel is navigating in and consequently improve situation awareness is still rare in machine learning approaches to vessel trajectory prediction (VTP). In inland shipping, where vessel movement is constrained within fairways, navigational context information is indispensable. In this contribution targeting inland VTP, Gaussian Mixture Models (GMMs) are applied, on a fused dataset of AIS and discharge measurements, to generate multi-modal distribution curves, capturing typical lateral vessel positioning in the fairway and dislocation speeds along the waterway. By sampling the probability density curves of the GMMs, feature vectors are derived which are used, together with spatio-temporal vessel features and fairway geometries, as input to a VTP transformer model. The incorporation of these distribution features of both the current and forthcoming navigation context improves prediction accuracy. The superiority of the model over a previously proposed transformer model for inland VTP is shown. The novelty lies in the provision of preprocessed, statistics-based features representing the conditioned spatial context, rather than relying on the model to extract relevant features for the VTP task from contextual data. Oversimplification of the complexity of inland navigation patterns by assuming a single typical route or selecting specific clusters prior to model application is avoided by giving the model access to the entire distribution information. The methodology's generalizability is demonstrated through the usage of data of 3 distinct river sections. It can be integrated into an interaction-aware prediction framework, where insights into the positioning of the actual vessel behavior in the overall distribution at the current location and discharge can enhance trajectory prediction accuracy.