Sentence aggregation

This first section demonstrates one way to use lex rules for planning multiple sentences. We demonstrate how to highlight similarities and differences between the content of sentences. You will learn how to automatically aggregate sentences — that is, to join two or more similar sentences together.

Sometimes the sentences you generate from your JSON data can look very similar. For example:

This is too repetitive! Aggregating sentences helps to avoid this repetition. For example:

Let's go through an example. The JSON data we’ll use is:

{ 
    "employees": [
        {
            "name": "Peter Jones",
            "job": "Head of Innovation",
            "teamSize": 1,
            "location": "New York"
        },
        {
            "name": "George Scott",
            "job": "Head Janitor",
            "teamSize": 2,
            "location": "New York"
        },
        {
            "name": "Anna Smith",
            "job": "Head of Marketing",
            "teamSize": 12,
            "location": "New York"
        },
        {
            "name": "Sue Collins",
            "job": "Chief Software Developer",
            "teamSize": 15,
            "location": "London"
        }
    ]
}

And we’ll use the following lex rule:

<S id="officeId" classes="officeClass">
    <Subj word="{{msg.name}}">
        <Feature name="surface_aggregatable" value="true"/>
    </Subj>
    <VP word="work"/>        
    <Obj string="{{msg.location}}" preposition="in"/>
</S>

An equivalent parse tree for the lex rule is shown below:

In this lex rule, there is a new feature: <Feature name="surface_aggregatable" value="true"/>. Because the feature is added inside the subject (i.e. inside the Subj tags) it enables the subjects of sentences generated by this rule to be aggregated.

JSON data values are accessed in the lex rule with "{{msg.name}}" and "{{msg.location}}". This time, the keys (“name” and “location”) of the JSON values are used instead of "{{msg.data}}" that we saw in the last section, Subject-verb agreement.

To generate one sentence from the lex rule, use the realise function:

Remember that the realise function’s parameter is a tuple. This time the tuple contains the first element of the WholeJSON array "employees" (the element with Peter Jones’s data) and the value of "classes" specified in the opening tag of the lex rule, officeClass.

Now we’ve seen that the lex rule works with one person’s data. Let’s use it to generate sentences for the entire array. First, let’s do this one sentence at a time (without aggregation) using the forAll function:

The forAll function goes through the JSON employees array and applies the realise function to each element in the array. The result is four sentences, one for each person’s data.

Next, we’ll call realise with the entire "employees" array:

This time, we are producing sentences for all array elements in a single call to realise. The result now has only two sentences. The subjects of the first three sentences have been aggregated because, apart from their subjects, these sentences are identical. Notice that the verb "work" has been correctly inflected for a plural subject.

The effect of aggregation is to highlight the similarities between the first three people (they all work in New York) and the differences between them and the fourth person (who works in London).

The image below shows the result of aggregating the first three sentences as a parse tree.