Monitoring Horses in Stalls: From Object to Event Detection

The gist: This study presents a prototype vision-based monitoring system that automates the detection and tracking of horses and people inside stables using object detection and multi-object tracking techniques.

Problem Definition

Monitoring the daily behavior of horses plays a critical role in their care and management, facilitating the early identification of health problems and abnormal behaviors (1.1). To address this issue, numerous startups have developed various horse monitoring systems (1.2). The primary objective of these indoor systems is the detecting and tracking horses within their stables (1.2).

Existing Work

Several research efforts have explored related areas, such as Delgado et al., who compiled and annotated a dataset of 10,000 images of stalled horses in different postures and trained a YOLOv3 model for object detection (1.2). Kholiavchenko et al. developed a dataset for wildlife behavior recognition using drone footage and implemented a pipeline combining YOLOv8 and the SORT tracking algorithm (1.2). Chan et al. introduced the YOLO-Behaviour framework for automatic detection of animal behavioral events (1.2). An alternative approach involved Kil et al., who employed the Loopy horse pose estimation model based on anatomical landmarks to analyze the behavior of stabled horses (1.2).

Proposed Solution

The proposed solution utilizes YOLO and SORT as core components (1.4). The key contributions include:

1. Employing CLIP and GroundingDINO to expedite dataset annotation (1.4).

2. Performing object tracking under complex conditions—distinguishing horses from people and handling blind spots (corners or edges of the stall that are not captured by the camera) (1.4).

3. Designing an event detection system that identifies when horses or people are inside or outside a stall by estimating event start and end times (1.4).

4. Conducting qualitative analysis of the system’s predictions to assess performance (1.4).

Materials and Methods

The overarching goal of the horse monitoring system is to detect and log various events occurring within a stall, such as abnormal horse behavior or interactions between horses and stablehands (2.1). The system is designed to detect four distinct events related to horses:

1. The horse is visible and inside the stall (2.1).

2. The horse is visible and outside the stall (i.e., in the adjacent hall) (2.1).

3. The horse is invisible but inside the stall (i.e., in a blind spot) (2.1).

4. The horse is invisible and outside the stall (i.e., has left the area) (2.1).

A fifth event type is defined when multiple horses are present in the stall simultaneously, and these five event types are applied analogously to people (2.1).

Dataset Collection involved two iterations for training:

1. First Iteration: Data volume was reduced through Stratified Sampling, Frame Selection (taking every 60th frame), and automatic annotation using the GroundingDINO model, followed by manual correction in CVAT (2.2).

2. Second Iteration: Automatic labeling was performed using a YOLOv11 model trained on the previously labeled dataset, with all labels refined manually in CVAT (2.2).

Multi-Object Detection and Tracking employed Ultralytics YOLOv11 for object detection, and the Ultralytics BoT-SORT algorithm for object tracking (2.3). During inference, two practical challenges were addressed using heuristics:

1. Class Prediction Instability: Class probabilities fluctuated across frames, resolved by assigning the class with the highest cumulative probability across all frames in a clip (2.3).

2. ID Inconsistency: The tracker occasionally assigned different IDs to the same object across frames, resolved by merging IDs for objects of the same class that never co-occurred in the same frame (2.3).

From Multi-Object Tracking to Event Detection, a five-step procedure was followed to convert object trajectories into stall events (2.4):

1. Localization: Each object is classified as either inside or outside the stall by checking the intersection between the object’s bounding box and the predefined floor polygon of the stall (2.4). Objects that never enter the stall during a clip are discarded (2.4).

2. Frame State Aggregation: The states of all detected objects of the same class are aggregated into a single frame-level state using rules in Table 3 (2.4).

3. Temporal Event Merging: Consecutive frames with the same frame state are merged into temporal events (2.4). For instance, if frames k through n have the state horse inside, a single event is created for that time range (2.4).

4. Classifying 'Non-Localized' Events: Events with non-localized objects are classified as either Inside (invisible) or Outside (invisible), using the last known frame of localization and evaluating the object’s distance from the stall entrance and whether the bounding box touches the frame’s edges (2.4).

5. Inter-Clip Correction: Classifications at clip boundaries are adjusted, such as reclassifying an event as inside (invisible) if it transitions from outside (invisible) to inside (invisible) across clips (2.4).

Results and Discussion

The system operated with a confidence threshold of 0.5, an IoU threshold of 0.5, and a frame stride of 20 over a 24-hour period using four cameras (k1, k2, k3, and k4) (3). The system generated non-empty event predictions for 360 clips out of 5,760 processed one-minute video clips (3). The analysis focused on the 10 clips containing both horse and person events as representative cases for qualitative analysis (3).

The analysis showed that the system accurately predicted horse-related events in all 10 clips (Table 4) (3). However, for person-related events, only 2 out of 10 clips had entirely correct predictions (3). Errors observed included False positives, Temporal shifts, and False negatives (3). The primary cause of these person-related errors was not sufficient YOLO's recall on the “person” class due to the limited number of annotated person instances in the training set (3). Figure 3 illustrates sequences where YOLO failed to detect a person (3). A potential alternative for distinguishing between “inside” and “outside” positions is drawing a separating line at the base of the wall, though this method is limited by camera angle and positioning (3).

Conclusion

The study developed a prototype system for monitoring horses and people in stables using CV techniques (4.1). The contributions included Dataset Development, Model Training, Event Detection System implementation, and Qualitative Evaluation (4.1). While YOLOv11 demonstrated strong performance on the horse class (Precision = 0.99, Recall = 0.958), its performance on the person class was less reliable (4.2). Future work will focus on expanding the training dataset for the underrepresented “person” class and developing a quantitative evaluation dataset to support comprehensive, event-level benchmarking using metrics such as t-IoU and mAP (4.2). The system lays the groundwork for real-time behavioral monitoring in equine facilities (4.2).

Author Contributions include Conceptualization, methodology, software development, formal analysis, investigation, visualization, and original draft preparation by Dmitrii Galimzianov (11). Viacheslav Vyshegorodtsev contributed Resources, data curation, and review and editing of the manuscript (11). Ivan Nezhivykh contributed Data curation and software development (11). The authors declare a financial interest in the product presented in this paper (5). During the preparation of this manuscript, the authors used OpenAI’s ChatGPT to improve readability and language (5). Ethical guidelines were followed as no experiments were conducted on human or non-human subjects that would require additional ethical approval (5). The references listed support the work on horse monitoring systems, object detection models, and tracking algorithms (12). The paper is a preprint accepted for publication in Doklady Mathematics (1). The final authenticated version will be available online at https://pleiades.online (1).

--- Page 1 ---

This is a preprint of the article accepted for publication in Doklady Mathematics (© 2026, Pleiades Publishing, Ltd., copyright holder as specified in the journal) Monitoring Horses in Stalls: From Object to Event Detection. The content of this preprint may differ from the published version (1).

--- Page 2 ---

Preprint — accepted in Doklady Mathematics (© 2026, Pleiades Publishing, Ltd.) (CV) techniques. Although the functionalities of the latter systems vary, the primary objective is the detecting and tracking horses within their stables [2] (1).

--- Page 3 ---

Preprint — accepted in Doklady Mathematics (© 2026, Pleiades Publishing, Ltd.) In real-time monitoring systems, both the classification and temporal localization of events are critical. The temporal Intersection over Union (t-IoU) metric [13] is a suitable measure for evaluating a system's ability to detect event start and end times (1).

--- Page 4 ---

Preprint — accepted in Doklady Mathematics (© 2026, Pleiades Publishing, Ltd.) First Iteration. We selected one-minute clips from six different camera feeds.

1. Improved data annotation via foundation models: Employing CLIP and GroundingDINO to expedite dataset annotation. This leverages these models to create a custom dataset, reducing reliance on labor-intensive manual labeling by automatically generating initial bounding boxes for horses and people.

2. Enhanced tracking robustness under occlusion: The system can better handle complex scenarios by addressing Class Prediction Instability by assigning the class with the highest cumulative probability across all frames in a clip, and resolving ID Inconsistency by merging IDs for objects that never co-occurred, preventing erroneous object disappearance during transient occlusions.

3. Refined event classification logic: The system can better distinguish between critical events by implementing the Inter-Clip Correction rule: if the last event in clip n–1 is 'inside (invisible)' and the first event in clip n is 'outside (invisible),' the latter is reclassified as 'inside (invisible),' assuming the object remains in the blind spot.

4. Improved person detection recall: To mitigate errors related to human presence, future work should focus on augmenting the dataset with more labeled samples of people in various lighting and positional conditions to improve YOLO's recall on the person class, as current performance is cited as a limitation due to limited number of annotated person instances.

5. Enhanced temporal event metrics: The system's evaluation can be made more rigorous by constructing a manually labeled evaluation set in future work to support more rigorous benchmarking using metrics such as t-IoU and mAP, moving beyond frame-based assessments which the paper notes fail to capture the temporal dimension crucial in monitoring systems.

Monitoring the behavior of stalled horses is essential for early detection of health and welfare issues but remains labor-intensive and time-consuming. In this study, we present a prototype vision-based monitoring system that automates the detection and tracking of horses and people inside stables using object detection and multi-object tracking techniques. The system leverages YOLOv11 and BoT-SORT for detection and tracking, while event states are inferred based on object trajectories and spatial relations within the stall. To support development, we constructed a custom dataset annotated with assistance from foundation models CLIP and GroundingDINO. The system distinguishes between five event types and accounts for the camera's blind spots. Qualitative evaluation demonstrated reliable performance for horse-related events, while highlighting limitations in detecting people due to data scarcity. This work provides a foundation for real-time behavioral monitoring in equine facilities, with implications for animal welfare and stable management.