Accurate monitoring of pig behavior is crucial for enhancing animal welfare and production efficiency in agriculture. Traditional observation limits the early detection of health issues and results in ineffective farm management. This study evaluates the performance of the You Only Look Once (YOLO) family, specifically YOLOv8n (nano), YOLOv8m (medium), YOLOv8x (extra-large), and a novel approach, YOLOv8ma (modified architecture), in classifying key behaviors—drinking, eating, sleeping, and standing—that correlate with health status for accurate behavioral analysis in group-housed pigs. This is based on overall performance, behavior detection accuracy, the impact of 2x and 3x augmentation, real-time processing, generalization capability, adaptability to environmental conditions, model size, and processing speed. The results indicate that the YOLOv8ma model’s performance increased from 0.947 without augmentation to 0.957 with 2x augmentation, while precision improved from 0.896 to 0.910. Compared to the YOLOv8 models, the YOLOv8ma also increased mAP@0.5 and precision by 6.32% and 4.59%, respectively. The model features shorter training times of 33.24 to 86.70 seconds per epoch and processing speeds of 40 frames per second, making it well-suited for fast-paced scenarios. PigSenseAI is a scalable, cost-effective web application that enables real-time behavior detection, classification, and automated alerts through an intuitive interface.