5 — Providing support
As with any software system, when considering how to support NLG systems, it is best to record user issues and enhancement requests in a structured fashion. Keep in mind that for an NLG system, the user is the reader of the narratives produced. So the user could be an internal client, an external client, or the consumer of an external client's reports.
Having progressed through the testing phase of the development life cycle, you will have caught most user issues and addressed them already. But with NLG development, you can run the project on any number of data sets – generating thousands of narratives – and still there will be edge cases or data irregularities that you did not encounter in your testing.
Keeping track of user issues and enhancement requests
Your project will evolve over time. Sometimes this happens because a user encounters issues that did not come up in testing. Other times it's because requirements change as time passes.
Even if your NLG implementation is small, it is best to track issues that come up from the outset. Arria recommends that you log all customer questions and issues in a service-desk system (or just something as simple as a spreadsheet). This will provide you with a searchable resource so you can quickly find similar scenarios in past support tickets. You'll cut down the time spent on investigating issues and you'll be able to reuse responses to clients and solutions to problems. Such a tracking system can also help you track responses from Arria when an issue is due to a bug or missing functionality in NLG Studio that will help in future support requests from your client, as well as when you’re creating NLG Studio scripts in the future.
As requests come in, make sure to funnel them to the right person.
One method for implementing enhancements is to have the user edit the narrative output, then give that back to the NLG developer to make the required changes. For obvious reasons, this would work better for internal clients than for external clients.
Ideally, enhancement requests should include the following:
abstract description (e.g. "If nothing has changed, just say this in a simple way.")
a concrete example (e.g. "The stock price has not changed.")
Remember that if you would like to submit an enhancement request for NLG Studio, you can do so by contacting Arria Support.
Supporting your internal and external clients
Suppose you've built an NLG project that is used for portfolio reporting. A client alerts you that the system doesn't generate a good narrative when the portfolio includes recently opened accounts. The report they're getting is leaving out critical information for one of their accounts, which was newly opened. To address this issue, you add an account to the portfolio so that the narrative reports on what the user needs to see.
In NLG projects, the issues requiring support and maintenance fall into two categories:
changes to the input data
changes to the narrative
An example of an enhancement requiring change of data input would be the following: you need to work with a new source of stock data because a client's data is coming from a new supplier and it's in a different format. So you have to change the system to deal with this new input format.
An enhancement that requires changes to the narrative would be what to do if a stock splits. Testing may have missed this edge case, and so now you find you need to update your NLG Studio project with a conditional and with an entirely new sentence that is generated under a certain condition.
Managing client expectations
If your NLG implementation has a lot of clients, consider setting their support expectations by writing a service-level agreement (SLA) outlining response times and categories of changes requested (e.g. ranging from simple fixes like adding a new sentence, to larger pieces of work such as rewriting a report to accommodate multiple new data columns).
Also consider how you would like to be contacted with change requests. Should clients contact you by email, phone, text, or through a message logged directly into a service desk application?
