Как-то меня попросили сделать обзор применения технологии обработки естественного языка или Natural Language Processing на английском. Ни в коем случае не путать с нейро-лингвистическим программированием, о котором так любят говорит коучи и прочие self-help гуру. Переводить на русский не буду, поэтому вывешу сюда как есть, может быть кому-то пригодится=)

Natural language processing as a service

Natural language processing is a field at the intersection of computer science, artificial intelligence and linguistics.
The goal for computers is to process or “understand” natural language in order to perform tasks that are useful, for example performing tasks, like making appointments or buying things.
The alternate usage is question answering and human-to-machine communations (Siri, Google Assistant, Facebook M, Cortana).

Tech

Primarily, the architecture is more or less the same – the spoken(or written) input (top left) is analyzed, words are recognized, sentences are parsed and interpreted in context, application-specific actions take place (top right); a response is planned, realized as a syntactic structure, then to suitably inflected words, and finally to spoken output; different types of linguistic knowledge inform each stage of the process.

Таблица технологии

Trends (more on each later) –

  • Interactive Voice Response (IVR)
  • Optical Character Recognition (OCR)
  • Pattern and Image Recognition
  • Classification and Categorization
  • Text Analytics
  • Speech Analytics
  • Support and Maintenance Services

Since the sphere is deeply interconnected with machine and deep learning, any sufficient progress in these spheres will affect the language recognition dramatically.

Tech barriers

Natural language systems that have been deployed for real-world applications still cannot perform common-sense reasoning or draw on world knowledge in a general and robust manner.
Natural language processing is successful in meeting the challenges as far as syntax is concerned. But it still has to go a long way in the areas of semantics, e.g. finding the meaning of a word or a word sense, determining scopes of quantifiers, finding referents and context.

Barriers of non-tech origin
The government regulation and laws that prohibit solution to language barriers is a huge difficulty. Low adoption rates are also stopping the growth. Areas that have no need to enter a multilingual marketplace are failing by a global community.

Main customers and why do they need it?

  • Government agencies
  • NLP Original Equipment Manufacturers (OEMs)
  • NLP service providers
  • System integrators
  • Consulting firms
  • Research organizations
  • Resellers and distributors
  • Media

The demand for NLP solutions is fueled mostly by the increased need to process unstructured and semi-structured data in enterprise networks. The rapid expansion in the acceptance of social networking and social media platforms and the growing focus on social media monitoring and social media intelligence (SMI) are other compelling factors influencing demand for NLP technologies.

How are the companies using it?

  • NLP technologies are gaining traction in text analytics, designed to unlock business value buried within the vast data assets of a company in a resourceful and seamless manner. Examples – Addstructure, Angel.ai, Klevu
  • Speech analytics is another important and growing application area, where NLP tools are finding preference for capturing the voice of the customer taking into consideration phrasing, punctuation, and tone of voice and other parameters. Examples – api.ai, mindmeld
  • As a recognition platform, NLP is sensing opportunities in interactive voice recognition (IVR), image recognition, and optical character recognition (OCR) verticals.
  • Spam mail blocking is another growing use case for NLP technologies.
  • By end-use sector, NLP technologies are making strong gains in healthcare & life sciences, banking & financial services, insurance, legal, IT & telecom, government, media & advertisement, and consumer electronics among others. Examples - Lexalytics
  • A key trend making waves in the market is the growing popularity of cloud-based NLP-as-a-Service solutions Examples – DigitalGenius, Inbenta

Business model

The most famous verticals for NLP are:

  • Banking, Financial Services, and Insurance (BFSI)
  • Manufacturing
  • Healthcare and Life Sciences
  • Retail and Consumer Goods
  • Research and Education
  • High Tech and Electronics
  • Media and Entertainment

The most common business model is NLP-as-a Service. The companies setup online services that rely on their tools, and provide a free number of API calls for those, charging for additional calls (or call-range) to your API. Or it could be a full version, for example chatbot text recognition in customer service with annual payments for the service.

Market assessment

According to Markets and Markets , the NLP market size is estimated to grow from USD 7.63 Billion in 2016 to USD 16.07 Billion by 2021, at a Compound Annual Growth Rate (CAGR) of 16.1%.The major forces driving the NLP market are increase in demand for enhanced customer experience, increase in usage of smart devices, number of emerging options in application areas, increased investment in healthcare industry, increased deployment of web & cloud-based business applications, and growth in machine-to-machine technology

North America is expected to have the largest market share and would dominate the NLP market from 2016 to 2021, due to the presence of a large number of NLP vendors.
East Asia offers potential growth opportunities for the NLP market. The key factors driving the growth rate in this market are increasing data generation in business organizations, growing demand for enhanced customer experience, and increase in the adoption and utility of smart devices.
In the NLP market, solution vendors such as 3M, Apple Incorporation, Dolbey Systems, Google, HP, IBM Incorporation, Microsoft Corporation, NetBase Solutions, SAS Institute Inc., and Verint Systems offer high-end NLP solutions.
All in all, the market is growing rapidly.

What kind of bisuness has a potential in NLP?

  • Assistants – marketing, sales or financial corporate assistant/ examples - Aiden, Einstein
  • Online service consultants – chatbots - Yamichat
  • HR companies – help to automate the calls with the canditates. For example, Amelia and one Russian startup.
  • Corporations dealing with troubled customers or the ones who want to analyze the worker’s attitude towards their occupation - Semantria