May 5, 2022 - Reading time: ~1 minute
The Average Distance Classifier is a simplified version of the Nearest Neighbour classifier. It is really easy to implement compared to other classification algorithms. Because it discards a lot of the data given to it, its accuracy can be worse than other classifiers. The algorithm can be thought of as a nearest-neighbor where each label is a single neighbor.
Lets say you have a dataset with 3 features for each data point, and 15 labels in total. You have 20 examples for each label, giving you a total row count 300.
Instead of storing all your features like the classic nearest-neighbor, you only store the average of them for each label. This means while you would store 300 rows for nearest neighbor, you only need to store 15 rows for the average distance classifier.
This is a huge reduction of your data, which should make your final model easier to store and faster to execute. But at the same time, it will make your accuracy a lot worse. But due to the ease of implementation and the minuscule computational requirements, Average Distance Classifier might be an acceptable choice if your classification task is a simple one.