Skip to content

What Is DeepPavlov Dream

Jump to bottom
Dilyara Zharikova (Baymurzina) edited this page May 9, 2023 · 1 revision

What Is DeepPavlov Dream?

DeepPavlov Dream is an open-source Multiskill AI Assistant Platform built using DeepPavlov Conversational AI Stack. It works on top of DeepPavlov Agent running as container in Docker. It runs on x86_64 machines, and prefers having NVIDIA GPUs on the machine. You can use it to study how multiskill AI assistants can be built, as a base for research projects, or for commercial systems where running open source system on-premises (or in the cloud) is a requirement. It's also a proud descendant of the 3 previous projects also called DREAM:

  • Original DREAM Socialbot for Alexa Prize 3 (2019-2020). Read the paper here.
  • Dream AI Assistant Platform Demo (2020). Read the post on Medium.
  • DREAM 2 Socialbot for Alexa Prize 4 (2020-2021). Read the paper here.

You can think of it as a Dialogue Operating System of sorts, with the emphasize on Operating System. Like in the Linux world, DeepPavlov Dream has multiple distributions:

  • DREAM Socialbot is the biggest distribution (~30 Skills, ~30 Annotators) adapted to run w/o Amazon Alexa Prize-specific services. This is almost the same version of the DREAM socialbot as at the end of Alexa Prize Challenge 4. DREAM 2 Socialbot blends together almost 40 different chit-chat and task-oriented skills to engage in open domain conversations. It relies on a selection of modern NLP models and components including ~30 annotators, post-annotators, and knowledge graph integration. Some API services are replaced with trainable models. Some services (e.g., News Annotator, Game Skill, Weather Skill) require private keys for underlying APIs, most of them can be obtained for free. If you want to use these services in local deployments, add your keys to the environmental variables (e.g., ./.env). This version of Dream Socialbot consumes a lot of resources because of its modular architecture and original goals (participation in Alexa Prize Challenge). We provide a demo of Dream Socialbot on our website.
  • Dream Mini is a neural generation-based socialbot. Dream Mini can be used a starting version of your custom socialbot because it contains only Dummy Skill (with fallbakcs and topical prompts which help to drive the conersation, and added sometimes to the dialog), DFF-Intent-Responder-Skill to respond to the detected user intents, AIML-based skill as an example, and DialoGPT as a main source of the responses.
  • Deepy Base is a base version of Lunar assistant. Deepy Base contains Spelling Preprocessing annotator, template-based Harvesters Maintenance Skill, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.
  • Deepy Advanced is an advanced version of Lunar assistant. Deepy Advanced contains Spelling Preprocessing, Sentence Segmentation, Entity Linking and Intent Catcher annotators, Harvesters Maintenance GoBot Skill for goal-oriented responses, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.
  • Deepy FAQ is a FAQ version of Lunar assistant. Deepy FAQ contains Spelling Preprocessing annotator, template-based Frequently Asked Questions Skill, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.
  • Deepy GoBot is a goal-oriented version of Lunar assistant. Deepy GoBot Base contains Spelling Preprocessing annotator, Harvesters Maintenance GoBot Skill for goal-oriented responses, and AIML-based open-domain Program-y Skill based on Dialog Flow Framework.

Source Code and Licensing

For many companies and individuals two key questions about any piece of software are availability of the source code and the license. DeepPavlov Dream AI Assistant Platform including its various distributives, is open source. The entire source code is available for anyone to use and modify as they see fit, for academic, personal, or commercial purposes. In particular, companies may use DeepPavlov Dream in whole or in part in products. Furthermore, it is completely free of charge. Support and consultancy may be available for a fee. Companies should contact us at [email protected] for pricing.

DeepPavlov Dream is available under an Apache 2.0 license, which may be attractive to companies since it does not require them to publish changes they make to the system as the GPL does.

History of DeepPavlov Dream

Deepy was originally built in October 2020 as a demo of a simple multiskill AI Assistant to be shown at DeepPavlov's talk at NVIDIA GTC Fall 2020 Conference. However, Deepy is based on DeepPavlov Dream AI Assistant Demo, which in turn is an adaptation of the original DREAM Socialbot created by DeepPavlov's student team for Alexa Prize Grand Socialbot Challenge 3 (2019-2020).

Architecture (by distribs)

Original DREAM Socialbot (2019-2020)

Note: not included in this repository as a distribution.

Dream AI Assistant Platform Demo (2020)

Deepy

image

D3PO

DREAM 2 Socialbot (2020-2021)

Note: not included in this repository as a distribution.

DeepPavlov Dream AI Assistant Platform

Running DeepPavlov Dream on a PC

Getting DeepPavlov Dream up and running depending on the distrib may be quite simple, but it might require having GPU(s) depending on the number of GPU-heavy components you want to use in your solution. Deepy, the simplest distrib, can be run entirely locally on a dev machine (6GB of RAM), while DREAM and DREAM 2 socialbots will require the use of the proxy services provided by DeepPavlov.ai.

Running Deepy

  1. Clone the repository
  2. Change directory to it
  3. Pick the distribution you want to run from /assistant_dists
  4. Copy docker-compose.yml from it to the root directory of your repository (system will ask you to confirm rewriting the existing one; confirm it)
  5. Copy pipeline_conf.json from it to the /agent of your repository (system will ask you to confirm rewriting the existing one; confirm it)
  6. Write the command docker-compose -f docker-compose.yml build to build your distribution but don't press ENTER just yet.
  7. [Optional] If you want to use ASR & TTS modules, add them to the docker-compose command file chain; in this case the command below would look like this: docker-compose -f docker-compose.yml -f ass_tts.yml build
  8. For GPU-intensive services, change lines in your docker-compose.yml and asr_tts.yml (optional) to specify GPUs you want to run them on. Typical GPU-intensive service that uses BERT model needs ~4GB of GPU RAM so plan accordingly.
  9. Build your system by using the line you've formed above
  10. Once done, use the same line but also replace build in the end with up.
  11. Once the agent is up and running, use your favorite tool (e.g., curl or Postman) to talk to Deepy via http://localhost:4242/ endpoint by providing the following content:
{
   "user_id" : "24424252524525",
   "payload" : "Hello!"
}

If everything works correctly, you'll get the response like this:

{
    "dialog_id": "773dc35a567072c142b9d6d8bdea00fb",
    "utt_id": "dfd1c751875439c6d98e3bb1410448f7",
    "user_id": "r234242343",
    "response": "Hello, I'm a lunar assistant Deepy! How are you?",
    "active_skill": "program_y",
    "debug_output": [
        {
            "skill_name": "harvesters_maintenance_skill",
            "annotations": {
                "emotion_classification": [
                    {
                        "anger": 0.46746790409088135,
                        "fear": 0.3528013229370117,
                        "joy": 0.3129902184009552,
                        "love": 0.2804321050643921,
                        "sadness": 0.35413244366645813,
                        "surprise": 0.19576209783554077,
                        "neutral": 0.9979490041732788
                    }
                ]
            },
            "text": "I don't have this information.",
            "confidence": 0.5
        },
        {
            "skill_name": "program_y",
            "annotations": {
                "emotion_classification": [
                    {
                        "anger": 0.38495343923568726,
                        "fear": 0.22263416647911072,
                        "joy": 0.4415707588195801,
                        "love": 0.4192220866680145,
                        "sadness": 0.21526440978050232,
                        "surprise": 0.19943127036094666,
                        "neutral": 0.998478353023529
                    }
                ]
            },
            "text": "Hello, I'm a lunar assistant Deepy! How are you?",
            "confidence": 0.98,
            "ssml_tagged_text": "Hello, I'm a lunar assistant Deepy! How are you?"
        }
    ],
    "human_utt_annotations": {
        "sentseg": {
            "punct_sent": "hello!",
            "segments": [
                "hello!"
            ]
        },
        "spelling_preprocessing": "hello!"
    }
}

Current Status

DeepPavlov Dream is, like all software, under development. However, you can pick any of the distributions (as well call them) from /assistant_dists directory to use it in your own system. We use one of these configs (currently /assistant_dists/deepy_ai_adv/) on our Demo Web Site. Here are some of the features of the current system. Development is ongoing and we hope you will join the community and help out.

DeepPavlov Documentation

Nearly all the documentation is in this wiki. This is a collection of pages with information on many topics relating to DeepPavlov Dream. When you are starting out, you should consult it often. Once you become more experienced, you can edit it, updating pages or adding new ones, just like Wikipedia.

DeepPavlov Dream Videos

Our team has made some videos about DeepPavlov Dream and its various distribs. You can watch them on YouTube as follow.

Deepy

Deepy on the Web

Deepy has appeared on the Web in various places. Here are the references:

DeepPavlov DREAM Socialbot

TBD

DeepPavlov DREAM 2 Socialbot

TBD

Clone this wiki locally