The new wave of automation: Unpicking the business impact

There’s a new wave of work automation now starting to impact businesses. Yes, partly this is enabled by Machine Learning technologies and techniques: but there’s more to it. The new wave of automation brings advances in interaction, insights and integration. This report shows how different technology applications play in this landscape.
Written by MWD Advisors’ lead analysts Angela Ashenden, Neil Ward-Dutton and Craig Wentworth, this report is the second of a two-part series.

The new wave of automation we outlined in three distinct, but related, layers in our first report – interaction, insight and integration – is most visible through the impact of specific applications that package technologies from one or more of our three layers together in order to address particular kinds of work scenario.

In this second report, we provide you with a simple framework for thinking about the potential for these new applications to impact organisations like yours (or your customers’), and then we work through five different example technology applications, analysing their nature and potential impact of each.

Backdrop: a spectrum of new automation use cases

The figure below shows you a useful way to think about the potential impact of applications that are part of the new wave of automation. The first thing (indeed, perhaps the most important thing) to note about this framework is that its starting point is an examination of how a technology will impact individual tasks.


Source: MWD Advisors

Your unit of analysis: tasks, not jobs

There’s been a lot of very widely-read and shared debate in the media[1][2] about how future automation will destroy jobs: this debate is very worthwhile, but the danger is that we focus our thinking too much on ‘jobs’ as if they were homogeneous things rather than the messy collections of responsibilities and roles that the vast majority of them really are.

Even brain surgeons have to fill in forms from time to time, and even back-office administrators have to manage exceptions. Almost all jobs involve the performance of many different kinds of tasks: some tasks are very prescribed, systematic and routine; others require the application of non-trivial judgement and discretion. When we look at the potential impact of automation, we need to explore and analyse at the level of the individual tasks that support business activities, services and processes.

The figure above examines the impact of the new wave of automation on particular tasks in two dimensions. On the vertical axis, we segment tasks according to the level of expertise required to successfully complete them; on the horizontal access, we segment tasks according to the frequency with which they will be performed.

As you can see, there are four broad categories of tasks we need to consider:

  • Truly expert tasks. The role of new automation technologies in the context of these tasks is to provide ‘expert assistants’ – to augment what experts can do, enhancing their expertise through the in-context application of sophisticated analytics technologies. A great example of how technology assists in these tasks is the current collaboration between IBM and the Memorial Sloan-Kettering Cancer Center in the USA: the two organisations are working together to deliver expert software-assisted cancer diagnoses. In these use cases, the likely value of new automation technology is in how it can process and analyse vast volumes of (perhaps very diverse/dispersed) data quickly, also referencing against an existing corpus – something that’s not really practical for an individual expert to do in a timely way.
  • Knowledge work tasks. These tasks don’t require truly advanced professional expertise but they do require people who have professional experience in a given subject matter area (for example, fraud management, customer service, sales, complaints management) to apply discretion and professional judgement to investigate problems, close cases, and so on. The role of new automation technologies in the context of these tasks is to provide ‘case advisors’ – tools that can provide context-specific insights and action recommendations to enable every member of a team or organisation performing a given knowledge-dependent role to be as high-performing as the best.
  • High-volume clerical tasks. These tasks do commonly require training, but it’s principally training on how to use the GUIs of (very often old and inflexible) business software applications to most efficiently perform data entry, query and reporting tasks. Many of these tasks today involve people performing highly repetitive actions that transfer data between systems in order to ‘close the loop’ in the back-office administration of business services. The role of new automation technologies in the context of these highly repetitive, programmatic tasks is to automate them as completely as possible – to drive business productivity and process quality and accuracy.
  • Personal administrative tasks. These tasks – like meeting scheduling, email management and so on – are common, ‘hygiene’ tasks that should in principle require little effort but can cause work-time disruption and create frustrating, inefficient communication loops between people and teams. Improving the productivity of an individual personal administrative task may not be in itself something that justifies a technology investment, but in aggregate, in some organisations, there may be a case for investing in personal productivity services or products that can fulfil some of the more transactional parts of the roles of human personal assistants.

In looking at these different task types and how they’re addressed in the new wave of automation, we can see that the tasks that appear in the lower half of the figure are most likely to be automated in their entirety, whereas the tasks in the upper half of the figur are most likely to be supported with automated elements. We call this state ‘augmented’ rather than ‘automated’ (after the established concept of ‘augmented reality’[3]).

Now we’re going to take a look at how this ‘new wave of automation’ is coming into play in different ways.

Application example: Chatbots

Development of, investment in and experimentation with chatbots is one of the most high-profile areas of activity we see as part of the new wave of automation. Chatbots typically automate conversations with humans via text-based messaging, and increasingly leverage machine learning services that perform natural language processing (NLP) that parse incoming text, identify objects and subjects in that text, and provide appropriate responses by referring to prebuilt (and continuously updating) knowledge-bases. Chatbots can be used to answer questions, assist with searches, make initial diagnoses of and propose solutions to problems.

Business interest in chatbots in particular isn’t only driven by the mainstream availability of highly-effective NLP toolkits; it’s also driven by broad familiarity with messaging apps (Facebook Messenger, WhatsApp, Slack and so on). Particularly in customer-facing scenarios, another major factor is the tantalising commercial potential of a new, very low-cost interaction channel that can work seamlessly across web, mobile and social platforms while also delivering acceptable quality of interaction.

or_new_automation_1116_pt2_fig2Source: MWD Advisors

The figure above shows that chatbots are primarily part of the interaction layer of the new wave of automation. It also provides an overview of some example use cases for chatbots in a business context:

  • Personal assistants, like Apple’s Siri, Microsoft’s Cortana and Amazon’s Alexa, are principally designed to help individuals find information and make simple requests through consumer devices (iPhones, Microsoft PC-based systems, Amazon Echo).
  • News bots, from providers like CNN in the consumer space, and Tangowork in the internal communications space, are designed to provide news services through chat interfaces, shifting the publication emphasis from push to pull, and taking another approach to the personalisation of news.
  • ‘Conversational commerce’ bots are starting to appear: H&M and Sephora, for example, provide chatbots as part of the popular Kik messaging platform (and also, of course, Amazon’s Echo/Alexa is designed to make it almost friction-free for customers to order). Here, bots are used not only to provide individuals with information, but also to enable them to buy products through conversation.
  • Virtual agents, like the Matilda pensions advisor, the DoNotPay legal advice bot (helping individuals argue against parking tickets) and RBS’ Luvo (based on IBM Watson technology), are designed to provide specialised advice and perform customer services functions. Luvo is particularly interesting because when it can’t help with inquiries itself, it invites human advisors into conversations, and then learns from their expert responses to refine its own knowledge base.

Application example: Recommendation engines

Recommendation engines have become a familiar part of the e-commerce landscape: whenever we shop on Amazon, Expedia or hundreds of specialised retail sites our experiences are personalised and tailored using these technologies. Every time Amazon shows you the “page you made” or your grocery retailer’s website asks if you want to add any of a set of promotional products to your basket, you’re seeing the effect of these engines.

The technology powering most recommendation engines in use in e-commerce and customer support scenarios today is principally based on established statistical modelling and analytics technologies; however, as we explained above, these technologies are increasingly being provided in the context of everyday online experiences (rather than being used offline by highly-trained specialists). Machine learning techniques have long played a role in specialised scenarios, but as the availability of large training datasets improves (because use of online services is expanding so quickly) and as the need to integrate models “live” into operational environments and have them run 24×7 has become greater, the incremental, machine-learning approach to model tuning (as opposed to the batch creation / update of models by specialist analysts) is becoming increasingly popular.

or_new_automation_1116_pt2_fig3Source: MWD Advisors

The figure above shows that recommendation engines are primarily part of the insights layer of the new wave of automation. It also provides an overview of some example use cases for recommendation engines in a business context:

  • Next Best Action services, increasingly used in customer service and support centres to help agents make optimal recommendations to customers or prospects, typically combine data from multiple sources and types, which can involve significant up-front cost and effort, but have very tangible benefits – great examples of organisations already doing this often come up in the telecoms sector, like Orange UK.
  • Personalised Learning is an emerging space. Personalised learning systems go beyond the typical Learning Management Systems (LMS) approach and use analytics to match learning content to individuals’ capabilities, roles, goals and interests using, in part, the experiences of peers to make learning more timely and relevant. IBM Kenexa is already doing some really interesting work in this area.
  • Expertise Location is a fairly established space. Expertise location services combine real-time data about individuals’ activities with models that analyse other individuals’ skills and interests, to make recommendations of experts who might be able to help with tasks. Vendors doing this in the collaboration space include Jive Software (with its Genius feature), and specialist ManyWorlds.

Application example: Content Classification services

As we’ve already highlighted, mainstream statistical analysis tools are now able to apply a measure of machine-learnt model tuning that can drive recommendations and judgements. In the content classification arena – which is of particular interest to fields of business that rely on and/or generate large amounts of unstructured content – it’s the ability of automated classification services to work at scale that makes them particularly interesting. Today’s content classification services can process very large volumes of content from a wide variety of documents, across a wide variety of sources, in ways that any human team would find challenging. The output of classification services, in the form of rich document metadata, can also surface connections between documents and topic-based document clusters that may otherwise remain hidden.

Although the most obvious application area for content classification services is in the legal arena, these algorithms and technology approaches are becoming relevant to much broader ranges of business applications – for example see OpenText’s acquisition of Recommind, now being applied to a variety of information analytics challenges at scale.

or_new_automation_1116_pt2_fig4Source: MWD Advisors

The figure above shows that content classification services are primarily part of the insights layer of the new wave of automation. It also provides an overview of some example use cases for content classification services in a business context:

  • E-Discovery – a process in litigation and legal investigations that combines natural-language processing (NLP) on the front end, to help ingest and understand structured and unstructured documents; with ML on the back end, to spot patterns and connections and to learn about what to surface from cases, and what to skip over. Without automation, e-discovery scenarios typically require significant numbers of fairly expensive resources. However, now that reading and analysis can be automated, e-discovery tools can whittle down useful documents.
  • Smart Digital Mailrooms, which automate the ingestion and analysis of multiple kinds of correspondence. Document classification at its most basic, which creates metadata to label document types, is fine; but real value comes from a granularity of classification that’s much finer, and that’s where sophisticated textual analytics comes in. The more rich the metadata that can be created on ingestion, the more sophisticated downstream processing that can be automated – driving content recommendations, search applications, and so on.

Application example: Cognitive IoT applications

This set of use cases is about monitoring and control of, and behavioural learning from physical assets – creating an “Internet of things that think”. Here the applicability of machine learning techniques comes from the complexities of scale – which is a function of the potentially vast number of connected devices you may want to work with, and the vast amount of data those devices may generate. Cognitive IoT applications can not only learn from the vast datasets presented to them (refining the models and training the systems that might predict device failures, for example); they can also apply what they learn to help connected devices adapt to dynamic operating environments.

In some use cases, large volumes of data, a need for real-time action, and the remoteness of devices (with often poor connectivity back to a hub or up to the cloud) have given rise to developments in ‘cognitive edge computing’ – where more of the data processing is done close to or on actual devices themselves, at least to serve the most immediate operational requirements.

or_new_automation_1116_pt2_fig5Source: MWD Advisors

The figure above shows that Cognitive IoT applications are primarily part of the insights layer of the new wave of automation. It also provides an overview of some example use cases for Cognitive IoT applications in a business context:

  • Predictive maintenance and facilities management are possible to improve without learning systems elements, but there are advantages to taking a cognitive approach. Using learning systems techniques, data processing systems can cope much better with new data sources and types as they become available; and in dynamic physical systems analytics can also ‘learn’ self-correction tactics to filter out noise. IBM, for example, infuses its asset management tools with Watson analytics to do just that.
  • In a facilities management context, a ‘smart building’ includes Building Management Systems (BMS) that are able to learn patterns of usage, and cross-reference these with other data (footfall, calendars, weather data and so on) to tune building performance. What’s more, analysis of data patterns over time can be used to improve building designs.
  • The last example here is of improving the passenger experience – in connected cars or mass transit. We’re starting to see experimentation within ‘smart cities’ programmes around intelligent transit systems, and early examples show that it’s important to create systems that not only use automation to autonomously drive transit vehicles, but also to have elements of those systems interact in ‘human’ ways with users. This particular use case is a little different in that it straddles the insights and interaction layers of the new wave of automation. A worthwhile example here is the use of IBM Watson technologies in a new experimental city transport service called Olli in Washington DC, USA.

Application example: Robotic Automation

Robotic Process Automation (RPA), as we introduced above – together with its cousin Robotic Desktop Automation (RDA) – is part of the integration layer in the new wave of automation. RPA and RDA technologies employ automated software ‘robots’ to act as synthetic application users, gathering and updating information in applications via their existing (Windows-based, web-based, or other) GUIs rather than via specialised integration interfaces or hooking into underlying databases or program structures.

At its heart is relatively mature technology, which is nevertheless largely valued today because of how it solves problems that organisations have typically created for themselves – where individuals have to get data in and out of constellations of legacy systems.

or_new_automation_1116_pt2_fig6Source: MWD Advisors

The figure above shows that there are two primary types of use case we see for Robotic Automation:

  • Automation of clerical data tasks in service centres. Here, Robotic Automation is capturing imaginations in large multinationals, business process outsourcing (BPO) providers and shared services organisations particularly, as a way to deliver the benefits of automated systems integration – particularly in support of high-volume, transactional service work – without long, complicated IT-driven projects or big investments in enterprise architecture initiatives.
  • Application migration. Here, the value of Robotic Automation spreads more widely: it’s not only attractive in environments where relatively straightforward, highly repeatable clerical work is concentrated. Organisations already familiar with Robotic Automation technologies are using them as tools to more quickly enable and complete application migration projects: using robots to extract legacy applications’ data through synthetic user queries, and then using robots to input that data into new target applications. One clear advantage here is that in using applications’ established user interfaces rather than directly linking into databases or APIs, integration work (and its testing) is significantly simplified – because migration logic can automatically benefit from user-facing data validation, and formatting logic that’s already been built into the applications in question.

Application example: Self-service Integration

The marketplace for Self-service Integration platforms has grown hugely over the past couple of years. Here, as is the case with Robotic Automation technologies, the key to the huge level of interest is not down to new technology particularly; it’s more to do with the ability of existing technology packaged in new ways to deliver integration results more quickly, and with lower commitment required from expensive, scarce IT resources. These platforms provide very user-friendly, low-code graphical tools that enable people to ‘hook together’ business software applications using graphical building blocks and simple scripting actions.

The proliferation of cloud-based business applications provided to suit the needs of sales and marketing, HR and other support functions has bolstered the Self-service Integration trend: adoption of these applications has very often been led from outside IT groups and those adopters are inclined to look for similar kinds of cloud-based, low (initial) cost, easy-to-use, self-service offerings that they can use to hook their applications up with other resources.

In some scenarios, Self-service Integration offerings are substitutes for Robotic Process Automation offerings. The technologies have similar value propositions: relatively low-cost, fast integration of applications with relatively low demand on IT resources. However whereas the sweet-spot for RPA is in automating application interactions where those (mostly old and ossified) applications are “difficult to reach”, the sweet-spot for Self-service Integration is in co-ordinating actions and data transfers between modern applications that are technically quite easy to reach.

or_new_automation_1116_pt2_fig7As the figure above shows, there are two primary types of use case we see for Self-service Integration, mirroring those we see for RPA:

  • Automation of application tasks. Here, Self-service Integration platforms take responsibility for ‘listening’ for events that occur in one application, perform a pipeline typically consisting of data transformation tasks, and then triggering one or more other applications (writing data or invoking other actions). Individual business departments within large organisations, as well as small businesses, are driving use of these tools.
  • Data synchronisation. Here, Self-service Integration platforms work in batch or burst mode, taking responsibility for bulk transformation and transfer of data between applications, data platforms and analytics tools.
Download PDF version