The more dimensions you need to learn and classify, the more that the samples of data you need in order to get enough valid information upon which to create generalizations. Without enough data, you will have very few examples for each combination of dimensions. This means you need lots and lots of data to make sure there are enough real examples to train the systems for predictable results. This is why Machine Learning is data hungry. “A typical rule of thumb is that there should be at least 5 training examples for each dimension in the representation”. As the number of dimensions on which a machine learning system is trained goes up, initially the performance of the model improves, but then later substantially decreases. One big goal of machine learning is to reduce the number of dimensions needed upon which a good machine learning model can be built that provides good generalizations.
Glossary of Artificial Intelligence (AI), Machine Learning (ML), and Big Data Terms
Curse of Dimensionality
