Rapid and precise Aspergillus species-level identification was needed to guide clinicians in effective treatment. Matrix-assisted laser desorption/Ionization time-of-flight mass spectrometer (MALDI-TOF MS) offers high identification accuracy with more complex processes and high-cost instruments, compared to conventional Aspergillus culture and microscopy. This research proposes an Aspergillus identification technique at the species level from colony images using machine learning (ML) based image processing methods for rapid diagnosis. Twenty-two strains of Aspergillus were confirmed as belonging to eight species using the MALDI-TOF MS and then cultured for 3 days on two types of medium. After fungal-colony image collection and enhancement, three main features, including Laws' texture energy measures (LTE), dominant a*- and b*- colors, and statistical feature matrix (SFM), were extracted from colony images for training four ML algorithms: K-nearest neighbors (KNN), Gaussian naive bayes (GNB), linear support vector machine (SVM), and random forest (RF). Training accuracy of all MLs using each feature reveals that dominant colors are the most significant feature; it can be further improved by combining colors with SFM, LTE, or both. The highest average accuracy is around 96% after the KNN and SVM are trained and predicted by dominant colors together with SFM and/or LTE for Aspergillus cultured on two different media.