Efficient collective swimming by harnessing vortices through deep reinforcement learning

Learning efficient swimming strategies: Differences between 2D and 3D flow fields. (A) The smart swimmer relies on a predefined set of variables to identify its “observed state” (such as range and bearing relative to the leader that are depicted). Additional observed-state parameters are described in Methods. (B) Comparison of vorticity field in the wake of 2D (Upper) and cross-section of the 3D (Lower) swimmers (red, positive; blue, negative). In 2D, the leader’s wake vortices are aligned with its centerline. In contrast, in 3D flows, the wake vortices are diverging, leaving a quiescent region behind the leader. In 2D, smart followers must align with the leader’s centerline. In 3D, they must orient themselves at an angle to harness the wake vortex rings (WRs). Every half a tail-beat period, the smart follower in 2D simulations (ISη) autonomously selects the most appropriate action encoded in policy πη learned during training simulations, which allows it to maximize long-term swimming efficiency (Movie S4). The smart follower is capable of adapting to deviations in the leader’s trajectory (Movie S5), as these situations are encountered when performing random actions during training. (C) Relative horizontal displacement of the smart followers with respect to the leader, over a duration of 50 tail-beat periods starting from rest (solid blue line, ISη; dash-dot red line, ISd). (D) Lateral displacement of the smart followers. (E) Histogram showing the probability density function (PDF; left vertical axis) of swimmer ISη’s preferred center-of-mass location during training. In the early stages of training (first 10,000 transitions; green bars), the swimmer does not show a strong preference for maintaining any particular separation distance. Toward the end of training (last 10,000 transitions; lilac bars), the swimmer displays a strong preference for maintaining a separation distance of either Δx=1.5L or 2.2L. The solid black line depicts the correlation coefficient, with peaks in the black curve signifying locations where the smart follower’s head movement would be synchronized with the flow velocity in an undisturbed wake (see SI Appendix for relevant details). (F) Comparison of body deformation for swimmers ISη (Upper) and ISd (Lower), from t=27 to t=29. Their respective trajectories are shown with the dash-dot lines, whereas the dashed gray line represents the trajectory of the leader. A quantitative comparison of body curvature for the two swimmers may be found in SI Appendix, Fig. S1.

Energetics data for a smart follower maximizing its swimming efficiency. Swimming efficiency (A) and CoT (B) for ISη (solid blue line) and SSη (dash-double-dot black line), normalized with respect to the CoT of a steady solitary swimmer. Four instances of maximum and minimum efficiency, which occur periodically throughout the simulation at times (nTp+0.12), (nTp+0.37), (nTp+0.62), (nTp+0.87), have been highlighted. Tp=1 denotes the constant tail-beat period of the swimmers, whereas n represents an integral multiple. The decline in η at point E (t≈27.7, η=0.86) results from an erroneous maneuver at t≈26.5 (Movie S7), which reveals the existence of a time delay between actions and their consequences.

Flow field and flow-induced forces for ISη, corresponding to maximum efficiency. (A) Vorticity field (red, positive; blue, negative) with velocity vectors shown as black arrows (Upper) and velocity magnitude shown in Lower (bright, high speed; dark, low speed). The snapshots correspond to t=26.12, i.e., point ηmax(A) in Fig. 3A. Demarcations are shown at every 0.2L along the body center line for reference. The wake vortices intercepted by the follower (W1U, W1L, W2U, W2L), the lifted vortices created by interaction of the body with the flow (L1, L2, and L3), and secondary vorticity S1 generated by L1 have been annotated. (B) Flow-induced force vectors (Upper) and body deformation velocity (Lower) at t=26.12. (C and D) Deformation power (C) and thrust power (D) (with negative values indicating drag power) acting on the upper surface of follower. The red line indicates the average over 10 different snapshots ranging from t=30.12 to t=39.12. The envelope signifies the SD among the 10 snapshots. (E and F) Deformation power (E) and thrust power (F) on the lower (left lateral) surface of the swimmer.