LESSON: Language variation

How much linguistic variation is too much? We developed a system with a lot of variation – at word, phrase, and sentence level. Still, we needed to make sure that all variants read well. The tool that we chose to assert that was regex (Regular Expressions). This meant that highly complex regexes had to be written to capture all variants, and every time small changes in the narratives were made, many of the highly complex regexes needed to be updated. It turned out that the vast amount of variation wasn’t even required by the client.