Data Description.

We train the predictive model using vessel monitoring data from GFW (10). For 16,261 unique longliner, trawler, and squid jigger vessels, we calculate a number of features on an annual basis from 2012 to 2018 (SI Appendix, Table S1 and SI Appendix). We call the unit of observation a vessel-year. These features represent aggregate annual observable vessel behavior features. We also include vessel characteristic features such as vessel flag and engine power. This training dataset includes 66,336 vessel-years of observation. We limit the analysis to these three gear types because they are the only gears for which we have known or highly suspected cases of forced labor for vessels that carry AIS, could be matched to GFW data, and broadcast sufficient and reliable AIS positions. We exclude vessels that broadcast less than 100 AIS messages per year, appear to be offsetting their vessel position latitude/longitude coordinates, broadcast multiple names per hour, or broadcast more than 95% of their messages from within the Taiwanese, Japanese, Chinese, Republic of Korea, or Democratic People’s Republic of Korea exclusive economic zones (EEZs). The high level of vessel congestion within these EEZs greatly reduces AIS coverage and limits our ability to calculate features for these locations (10). The full training dataset is provided in Dataset S1 (SI Appendix).

To identify which vessel-years should be labeled as positive for using forced labor, we developed a comprehensive database of 193 reported cases of forced labor that occurred on specific fishing or refrigerated fish cargo (“reefer”) vessels. We define a reported case of forced labor as a situation that displays one or more of the ILO forced labor indicators (SI Appendix, Fig. S1 and SI Appendix). In addition, these cases must also have one or more of the following case features: 1) eyewitness account, 2) nonofficial investigation (e.g., by an NGO through investigative journalism), 3) official investigation (e.g., by a government enforcement body), 4) arrests made, 5) charges filed, 6) conviction made, or 7) penalties sanctioned. Recognizing that the presence of ILO forced labor indicators does not guarantee involuntary or forced labor, and also recognizing that cases often simultaneously exhibit a number of different forced labor indicators, we endeavor only to detect forced labor broadly as specified by any of the 11 ILO indicators of forced labor, and do not distinguish between whether vessels may be using bonded labor or slave labor or which of the 11 indicators a particular vessel may be exhibiting. We found reported cases through extensive gray and scientific literature review, and also through discussions with NGOs including Liberty Shared, Environmental Justice Foundation, and Greenpeace Asia. For each case, we collect information on vessel identity including vessel name, Maritime Mobile Service Identity (MMSI) number, IMO number, call sign, information on when the vessel was thought to be conducting that behavior, and the source for the information (Dataset S2 and SI Appendix).

The three most commonly observed ILO indicators relate to servitude labor: 1) abusive working and living conditions, 2) restriction of movement, and 3) isolation (SI Appendix, Fig. S1 and SI Appendix). Debt bondage was only reported in one vessel case. While the information provided in case reports is likely not comprehensive of the conditions aboard these vessels, these indicators suggest that the model is primarily trained on forced labor cases indicative of servitude rather than bonded labor. Additionally, only four cases reported that convictions had been made and penalties sanctioned. This may reflect insufficient reporting of case information in publicly available reports, but may also reflect insufficient policy and legal institutions for prosecuting cases of forced labor in fisheries. Either of these insufficiencies would indicate a lack of appropriate deterrence for preventing vessels from using forced labor in fisheries. According to AIS data, the MMSI numbers associated with several of these vessels continued to operate in years following when they were labeled as positive (SI Appendix, Fig. S1 and SI Appendix). In 2018, 13 of 23 (57%) vessels with reported forced labor were still operating, with 11 of those 13 vessels (85%) being classified as high-risk. While this indicates these vessels were not taken out of commission following forced labor reports, it remains unclear if this is because sanctions were not imposed or if the vessel simply changed ownership in order to continue operations.

We label vessel-years as positive if the vessel is contained in our database of reported forced labor cases and if the year is the single year prior to when the case was reported, yielding n = 21 unique vessel cases. Since most case reports do not specify the time period during which abuses took place, we assume the abuses took place in the year prior to the report. While we will refer to this as our baseline model variation, this is an assumption we also test through a robustness check, where we vary our assumption to be that vessels should be labeled as positive in the 2 y prior to the report, in the 3 y prior, etc. We detail this robustness check below. For vessels included in the forced labor vessel database, any year not labeled as positive is excluded from the training dataset since we hypothesize these vessel-years have a higher chance of existing than other vessels, although we are uncertain due to the dynamic nature of forced labor on board fishing vessels (SI Appendix, Fig. S1 and SI Appendix). We later use the model to make risk predictions for these excluded vessel-years. Vessels for which we have no information and which are not included in the forced labor database are unlabeled, meaning they could be free of forced labor or could have forced labor that has not yet been detected (SI Appendix, Fig. S2 and SI Appendix). These labels allow us to perform PU learning using the training dataset (described in the following section). To apply positive labels, we matched our database of forced labor vessel cases to the GFW training dataset using MMSI number, IMO number, call sign, and/or vessel name, and for the year prior to which the case was reported. In cases where the only vessel identification information that could be matched was vessel name, we conservatively disregard matches that use common vessel names, including Viking, Lucky Star, and Greenstar. SI Appendix, Fig. S2, summarizes the number of labeled and unlabeled vessel-years by fishing gear and year and positive training data set label year assumption. A total of 35 reefer vessels were also matched and were used to generate a model feature that describes the number of suspected transshipment events that a particular vessel had with other vessels in the forced labor database.

Importantly, since these forced labor cases exhibit varying levels of evidence and occurred within various areas of legal jurisdiction, we are not implicating these vessels with any specific crimes or actions. Rather, we are labeling these vessels as high-risk vessels that warrant further scrutiny according to the ILO forced labor indicators. Case information was often sparse and did not usually indicate whether labor was involuntary or not, so we endeavored to consistently capture which ILO forced labor indicators were present. Vessel identification information sometimes included MMSI number, IMO number, or call sign, but often just included vessel name. In these cases, we searched for names in online databases such as MarineTraffic.com and endeavored to find matching vessels from the same flag as was reported in the case. Since this may not always provide a perfect match to the vessel in question, we again are not implicating these vessels with any specific crimes or actions. We also acknowledge that there is an inherent time lag between when a vessel may be using forced labor, when that vessel gets caught or when witnesses emerge, and when the case is reported to the public. This means that we may be observing fewer cases in recent years, which could lead the model to underpredict risk in these recent years. Given the sparsity of case information and the difficulty in matching vessels, the authors call for increased transparency for publishing detailed forced labor case reports and for increased use of AIS devices, MMSI numbers, and IMO numbers.

The data for each model feature come from one of three sources: 1) directly observed using AIS data; 2) inferred using the GFW fishing classification algorithm, which classifies individual AIS messages by gear type and labels them as either fishing or not fishing; or 3) vessel characteristics that come from either a known vessel registry (where available) or from the GFW vessel characterization algorithm (where vessel registry information is not available). The GFW fishing classification algorithm determines which gear type a fishing vessel uses and whether or not that vessel is fishing. For longliners and trawlers, the algorithm uses a convolutional neural network that has classification F1 scores, which combine precision and recall, of 0.93 for drifting longlines and 0.96 for trawlers (10). For squid jiggers, the algorithm uses a heuristic that labels vessels as fishing for squid if the vessel is more than 10 nautical miles from shore and moving at less than 1.5 kn for more than 4 h at night. This heuristic is reliable since squid jiggers have a distinctive behavior in which they fish only at night and only while moving very slowly (37). The GFW vessel characterization algorithm is a separate convolutional neural network that is trained using data from known vessel registries and predicts vessel length (R² = 0.9 across all gear types), engine power (R² = 0.83), gross tonnage (R² = 0.77), and crew size (R² = 0.73) (10). For the training dataset used in this analysis, 59% of vessel-years have known vessel length from registries, 58% have known gross tonnage, 48% have known engine power, and 16% have known crew size; the remainder of the parameters not contained within registries are obtained from the GFW vessel characterization algorithm. We include vessel characteristics as model features in our baseline model variation, although we test the sensitivity of our results to the inclusion of these features in a robustness check detailed below.

Model Development and Testing.

We use machine learning to develop a predictive model that discriminates between fishing vessels that are at high risk of using forced labor from those vessels that are not. We consider a number of predictive features that measure vessel behavior (e.g., average duration of voyages, average number of hours spent fishing per day, and number of suspected transshipment events with other vessels) as well as static vessel characteristics (e.g., vessel flag, gear type, and engine power; SI Appendix, Table S1, and SI Appendix). The ideal training dataset to build this model would include a list of vessels that are known to have used forced labor and a list of vessels that are known to have not used forced labor, with both lists being randomly sampled from these two classes of vessels. However, although we compiled a list of vessels that were reported to use forced labor (i.e., “positive” cases), we do not have a list of vessels that we know did not use forced labor (i.e., “negative” cases). Upon discussion with a number of human rights experts, there is not currently a certification or transparency scheme that can reliably guarantee that a specific vessel is not using forced labor. Even with existing schemes, vessels often self-select into these schemes and are therefore not necessarily representative of the larger fleet of vessels. Therefore, any vessel that was not reported to use forced labor is either truly clean of forced labor or simply was not caught (i.e., unlabeled cases). Our analysis therefore is an example of PU classification learning (12). If we had both positive and negative vessels in the training dataset, we would not need to use a PU learning approach and could instead use traditional supervised learning methods. More traditional supervised learning methods may more accurately be able to discriminate between positive and negative vessels because these methods could leverage behavior from known negative vessels, rather than relying on behavior from vessels that are unlabeled and thus could either be negative or positive (e.g., ref. 28). In future research, the model could potentially be improved by incorporating confirmed negative cases that could be obtained from improved social responsibility certification schemes, or from randomly sampled in-person vessel inspections. These types of inspections could also increase our number of positive training cases. However, these types of inspections would be expensive and logistically challenging to conduct, and could provide risks to both those conducting the inspections as well as to the crew aboard inspected vessels.

PU learning has a relatively nascent literature, but there are a number of methods for dealing with this problem that include 1) training a traditional classification model using positive cases and assuming unlabeled cases are negatives; 2) biased learning, where unlabeled cases are treated as negative cases with an unknown amount of class-label noise; and 3) training a traditional classification model using positive cases and assuming unlabeled cases are negatives, and then adjusting the predictions by a constant factor using the known true conditional probability of being positive (38). Because we do not know the true fraction of positive vessels, we focus on the first two of these PU approaches.

The first approach, building a traditional naïve classifier, is the most straightforward, but also leads to biased predictions. Although the probability of being positive should theoretically be biased by a constant factor across all cases and the relative probabilities across cases should be ranked correctly, this only applies if the observed positive cases were chosen at random from all positive cases (38). We know that this assumption does not hold in our case.

The second approach, biased learning, aims to address the class label noise due to unlabeled positive cases. One biased learned technique leverages bagging (bootstrap resampling with replacement), whereby many models are trained using all positive cases and a down-sampled number of unlabeled cases. Predictions are then calculated as the average across model runs (i.e., bags) (39). This reduces the relative importance of unlabeled cases that should actually be labeled positive, and has been shown to perform better than training a traditional classifier (40). Each iteration of the mode leverages a base classifier that can be any supervised learning method (12). Much of the PU literature uses support vector machines (SVMs) as the base classifier, although random forests have been shown to perform as well as SVMs or better in some cases (41).

In order to determine the best method for our application, we test four model variations: 1) traditional naïve classifier using an SVM, 2) classifier trained using biased learning with SVM base classifier and the bagging approach, 3) traditional naïve classifier using a random forest, and 4) classifier trained using biased learning with random forest base classifier and the bagging approach.

For each random forest variation, we use the ranger R package and specify 1,000 trees and use the default values for the remaining hyperparameters (42). For each SVM model variation, we use the kernlab R package with a radial basis function and the default hyperparameter values (43). For variations that use the bagging technique, we build up to 100 different classifiers that are trained using up to 100 bags of the data that each have all positive cases and a random down-sampled subset of unlabeled cases resampled with replacement. The number of unlabeled cases included in each bag is a hyperparameter that is tuned (referred to as the down-sampling ratio) and varies from the number of positive cases up to five times the number of positive cases. We calculate the average model score across all bag classifiers and use this as the final model score.

Given our extremely limited training dataset of known positive cases, we do not have sufficient data to reserve a completely separate testing portion of the dataset for model validation. Instead, we use a 10-fold cross-validation (CV) analysis to tune hyperparameters for these four different model variations and to evaluate the performance of these variations to choose the best model. Importantly, to avoid data leakage, we do not split forced labor vessels or media sources across analysis and assessment folds. We suspect that all forced labor vessels described by a particular media source may behave similarly, so we do not want to train and test the model on these similarly behaving vessels. Similarly, individual vessels may behave similarly across separate years, so we do not want to train and test the model using the same vessel across multiple years.

During CV, we optimize three hyperparameters simultaneously: 1) the threshold for determining the cutoff used to classify a vessel as a high risk; 2) for model variations with bagging, we tune the number of unlabeled vessels that are used in each bag, which is a down-sampling ratio of the number of positive vessels; and, 3) for model variations with bagging, we tune the number of bags. Hyperparameters are tuned by maximizing the mean of a modified F1 score across folds while minimizing the SD and aiming for model simplicity. The modified F1 score has been shown to be an appropriate model performance metric in the PU setting (44). It incorporates recall (the fraction of positive vessel-years that are correctly identified as positive) and detection prevalence (the percentage of vessel-years that are labeled as positives by the model), and can be calculated using only observed known positives and is thus appropriate for the PU learning environment. It is defined as recall squared divided by detection prevalence, and is proportional to the square of the geometric mean of recall and precision (precision is the fraction of correctly identified positives out of all identified positives) (45). The score therefore equally weights the importance of minimizing both type 1 and type 2 errors. We calculate the averages and SDs for recall, detection prevalence, and modified F1 score across folds for all model variations. Since we assess these metrics using CV, this gives us a sense of how the model will perform when predicting out of sample. We do not look at other traditional model performance metrics such as precision, area under the precision–recall curve, or area under the receiver operating characteristic curve, since these are known to be biased in the PU setting (46).

Within the model building procedure, we include a data-preprocessing step that is done both within the CV procedure and within the final model building procedure. During CV, the preprocessing is always completed using only the analysis data, and not using the assessment data, in order to avoid data leakage. The preprocessing is defined as follows: 1) impute missing numeric values using K nearest neighbors based on Gower’s distance, with k = 5 neighbors, and imputing based on the gear, flag, and vessel length predictors (47, 48); 2) transform numeric predictors using Box–Cox transformation (49); 3) create dummy variables for all categorical predictors; 4) remove numeric predictors that have zero variance or are highly sparse (i.e., the number of unique values divided by the total number of samples is less than 10) and unbalanced (i.e., the frequency of the most prevalent value is higher than 19 times the frequency of the second most prevalent value); 5) remove numeric predictors that have a correlation with other predictors greater than 0.75; 6) center numeric predictors to have a mean of 0; and 7) scale numeric predictors to have an SD of 1. The preprocessed data for the full model training dataset are summarized in Fig. 1.

SI Appendix, Fig. S3, shows the recall, detection prevalence, and modified F1 score and recall of the various model variations during 10-fold CV and using the optimized threshold. We see that the base classifiers without bagging show large SDs for the modified F1 score, and so we eliminate these models as candidates. Next, we see that model performance stabilizes at 50 bags and above, although the mean and SD with 100 bags are the most stable across down-sampling ratios, with similar mean performance between both random forest and SVM. To maximize model simplicity, we therefore set the down-sampling ratio to 1. For 100 bags and a down-sampling ratio of 1, the random forest has the highest modified F1 score. We therefore select our optimized model variation to be random forest with 100 bags and a down-sampling ratio of 1. This specification has the following mean performance across the 10 folds: modified F1 score, 4.3; recall, 0.92; and detection prevalence, 0.2.

Using the optimized model building procedure determined using CV, we then train the final model using the entire training dataset and the optimized hyperparameters. SI Appendix, Fig. S4, summarizes the number of positive and negative classifications predicted by this model, broken apart by whether the original training dataset label was positive or unlabeled. Using the baseline model assumption, the model correctly classifies 20 of 21 positive vessel-years and also identifies 12,000 new high-risk vessel-years that were previously unlabeled. SI Appendix, Fig. S5, shows the feature importance, averaged across bags, for the final model. For each bag, the feature importance for that random forest classifier is the unbiased corrected Gini index, a measure of how well individual features do at reducing node impurity when they are used as the splitting feature in a decision tree. Importantly, feature importance is relative and does not provide information about the directionality of each feature’s ability to accurately identify risk.

Robustness Checks.

In order to test the sensitivity of our results, we perform two robustness checks: 1) given that vessel characteristics in the GFW database are often inferred rather than directly known from vessel registries, we run model variations that do not use any vessel characteristics as model features; 2) given that, for vessels with reported forced labor, it is usually unknown exactly which years forced labor may have been occurring prior to the report, we run model variations that make different assumptions about how many years prior to the report to label as positive. For each robustness check, we replicate the entire model training and prediction process as outlined above. We again use CV to tune the key hyperparameter of the cutoff threshold for determining which vessels should be labeled as high-risk. We focus on using the random forest model variation with 100 bags and an undersampling ratio of 1 in order to directly compare the results from these robustness checks with those from our baseline model assumptions.

For the first robustness check that tests the sensitivity of our results to the use of vessel characteristics as model features, we run variations of the model that omit the model features of engine power, tonnage, vessel length, crew size, and AIS device type. The model variation that includes vessel characteristics as model features is our baseline model variation. For the second robustness check, we vary the number of years in the training dataset that are labeled as positive for the vessels that are reported to have used forced labor. A value of 1 is the baseline model variation and means that only the year prior to a vessel being reported for forced labor is labeled as positive, a value of 2 means that the 2 y prior to being reported are labeled as positive, etc. Across the range of robustness checks we have 23 unique positive vessel cases, an increase from 21 unique cases in the baseline variation, since some vessels were not observed in the year prior to being reported (case information for all 23 cases vessels is provided in SI Appendix, Fig. S1, and SI Appendix). The CV model performance results are shown in SI Appendix, Fig. S6, which summarizes detection prevalence, modified F1 score, and recall across all model variations including the baseline variation. The predicted final model results are shown in SI Appendix, Fig. S7, which summarizes the sensitivity of our main results to different model variations: the fraction of correctly identified true positives, fraction of vessel-years identified as positives, fraction of vessels identified as positives, number of crew working on positive vessels, number of vessel-years identified as positives, and the number of vessels identified as positives across all model variations include the baseline variation. Model variations that leverage vessel characteristics, even though these characteristics are inferred for many vessels, consistently have higher modified F1 scores than variations that do not use these characteristics. There is not a clear pattern for which year assumption yields the highest F1 score, although detection prevalence declines as the year assumption increases. As the year assumption increases and more positive vessel-years are used in model training, this may impose a more restrictive constraint on what similar unlabeled vessel-years should look like, which may decrease detection prevalence. Therefore, for the results presented in the Results and also in Fig. 2, we present the minimum and maximum value ranges from SI Appendix, Fig. S7, using model variations that include vessel characteristics and all year assumption model variations. We also present results using only two significant digits.

Given the sensitivity of our results to the assumption of which vessel-years to label as positive, we encourage reports that detail forced labor cases to provide as much information as possible regarding when suspected violations were taking place. We also encourage the use of more detailed public vessel registries that can provide known vessel length, engine power, tonnage, and crew size in order to further refine model features.